Depth-direction convolution with parallel processing of input and output

Abstract

This provides a systolic array that performs depth-direction convolution operations on multiple channels. [Solution] Depth convolution, which includes parallel processing of input and output, is performed by a plurality of multiplication-accumulation (MAC) units of the systolic array, each MAC unit having a weight register for storing weight values, an activation register for storing activation values, a multiplexer that transmits the activation value received from one of the activation registers and input lines, a multiplier 336 that multiplies the weight values ​​from the weight register and the activation value from the multiplexer, a memory that communicates with the plurality of MAC units, and a controller that transmits the weight values ​​of each MAC unit from the memory to the weight register of the MAC unit and transmits the activation value from the memory to the upstream MAC unit among the plurality of MAC units.

Description

Background Art

[0001] A neural network inference chip performs a convolution operation including a multiply-and-accumulate (MAC) operation. A systolic array can be used to perform a pointwise convolution operation for each point with parallel processing of input channels and output channels. A depthwise convolution operation is performed by separate chip hardware.

Brief Description of the Drawings

[0002] The aspects of the present disclosure are best understood from the following detailed description when read in conjunction with the accompanying drawings. Note that various features are not drawn to scale according to standard practice in the industry. In fact, the dimensions of various features may be arbitrarily increased or decreased for clarity of discussion.

[0003] [Figure 1] FIG. is a diagram showing a system for depthwise convolution with parallel processing of input and output according to at least some embodiments of the present disclosure.

[0004] [Figure 2] FIG. is a schematic diagram of a systolic array according to at least some embodiments of the present disclosure.

[0005] [Figure 3] FIG. is a schematic diagram of a MAC unit according to at least some embodiments of the present disclosure.

[0006] [Figure 4] FIG. is a schematic diagram of a depthwise convolution process according to at least some embodiments of the present disclosure.

[0007] [Figure 5A]This is a schematic diagram of a systolic array in period T1 according to at least some embodiments of the present disclosure.

[0008] [Figure 5B] This is a schematic diagram of a systolic array in period T2 according to at least some embodiments of the present disclosure.

[0009] [Figure 5C] This is a schematic diagram of a systolic array in period T3 according to at least some embodiments of the present disclosure.

[0010] [Figure 6] This figure shows the computational flow for performing convolution using a systolic array according to at least some embodiments of the present disclosure.

[0011] [Figure 7] This figure shows a computational flow for depth convolution with parallel processing of inputs and outputs, according to at least some embodiments of the present disclosure. [Modes for carrying out the invention]

[0012] The following disclosure provides many different embodiments or examples to implement different features of the subject matter provided. Specific examples of components, values, behaviors, materials, arrangements, etc., are described below for the sake of brevity of this disclosure. Naturally, these are merely examples and not intended to be limiting. Other components, values, behaviors, materials, arrangements, etc., are contemplated. In addition, this disclosure may repeat reference numbers and / or letters in various examples. This repetition is for the sake of brevity and clarity and does not in itself define relationships between the various embodiments and / or configurations discussed.

[0013] A systolic array known to the inventor, used to perform point-by-point convolution operations with parallel processing of input and output channels, cannot be used to perform depth-direction convolution operations on multiple channels with parallel processing of input and output channels.

[0014] In at least some embodiments described herein, a systolic array used to perform a point-by-point convolution operation with parallel processing of input and output channels is modified to add an activation register in series to each column, and one multiplexer for each register. This multiplexer directs input from the activation register to the multiplier in the MAC element, instead of an input line. In at least some embodiments, the upstream activation register of each column is connected to the input line of a single row, from which it receives an activation value. In at least some embodiments, during the execution of a depth convolution, the upstream activation register performs the MAC operation and also propagates the received activation value down to the next activation register in the column and receives the next activation value. In at least some embodiments, the downstream activation register performs the same propagation operation, passing it down to the last activation register. In at least some embodiments, the column repeats the receiving and propagating operations until all MAC operations have been performed. Thus, each channel may be applied to a single column, and in at least some embodiments where the systolic array has multiple columns, multiple channels can be processed in parallel.

[0015] In at least some embodiments, depth convolution is enabled to be performed using a small amount of additional hardware to the systolic array, instead of separate, dedicated chip hardware. At least some embodiments of such systolic arrays provide performance equal to or better than that of separate, dedicated chip hardware.

[0016] Figure 1 shows a system for depth convolution with parallel processing of inputs and outputs, according to at least some embodiments of the present disclosure. The system comprises an integrated circuit 100 and a host computer 102.

[0017] The integrated circuit 100 communicates with a host computer 102 and includes a systolic array 110, memory 116, and a controller 118. In at least some embodiments, the integrated circuit 100 is configured to house components for performing depth convolution operations with parallel input and output processing. In at least some embodiments, the integrated circuit 100 is configured for parallel processing of multiple channels using a systolic array architecture such as the systolic array 110. In at least some embodiments, the integrated circuit 100 is configured to communicate with the host computer 102 to receive instructions and data. In at least some embodiments, the integrated circuit 100 is configured for convolutional neural network inference. In at least some embodiments, the integrated circuit 100 is configured for other types of neural network inference operations. In at least some embodiments, the integrated circuit 100 is part of a silicon chip, a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a larger system-on-chip (SoC). In at least some embodiments, the integrated circuit 100 is configured for other computational tasks.

[0018] The host computer 102 communicates with the integrated circuit 100 for depth convolution tasks. In at least some embodiments, the host computer 102 is configured to provide instructions and data for depth convolution tasks to the integrated circuit 100. In at least some embodiments, the host computer 102 transmits data and instructions to the integrated circuit 100 via wired, wireless, network, or any other form of electronic communication. In at least some embodiments, the host computer 102 receives processed data and results from the integrated circuit 100. In at least some embodiments, the host computer 102 performs general computing tasks, user interface management, data storage, etc. In at least some embodiments, the host computer 102 interfaces with peripherals, storage devices, and network components. In at least some embodiments, the host computer 102 is a desktop computer, server, or embedded system. In at least some embodiments, the host computer 102 is used to run applications, manage databases, and perform general computing tasks.

[0019] The systolic array 110 communicates with the controller 118 and the memory 116. In at least some embodiments, the systolic array 110 is configured to perform parallel processing of convolution operations using a plurality of MAC units arranged in a grid. In at least some embodiments, the systolic array 110 is configured for depth convolution with parallel input and output processing by transmitting activation values ​​through columns of MAC units. In at least some embodiments, the systolic array 110 is configured to receive data from the memory 116 and control signals from the controller 118. In at least some embodiments, the systolic array 110 is configured to transmit data to the memory 116 for storing result values. In at least some embodiments, the systolic array 110 is configured for other types of matrix operations with parallel input and output processing, such as point-by-point convolution.

[0020] Memory 116 communicates with the systolic array 110 and the host computer 102. In at least some embodiments, memory 116 is configured to store weight values, activation values, and convolution result values. In at least some embodiments, memory 116 is an on-chip memory directly connected to the systolic array 110. In at least some embodiments, memory 116 communicates with the controller 118 to receive and store data. In at least some embodiments, memory 116 is configured for general data storage and retrieval purposes. In at least some embodiments, memory 116 interfaces with an external memory or storage device of the host computer 102 for larger datasets. In at least some embodiments, the integrated circuit 100 has memory 116 that communicates with multiple MAC units.

[0021] The controller 118 communicates with the systolic array 110, the memory 116, and the host computer 102. In at least some embodiments, the controller 118 is configured to manage the calculations of the systolic array 110 and to coordinate the data flow. In at least some embodiments, the controller 118 is configured to control the transmission of weight values ​​and activation values ​​of the systolic array 110 to the MAC unit. In at least some embodiments, the controller 118 is configured to receive instructions from the host computer 102. In at least some embodiments, the controller 118 is configured to transmit control signals to the systolic array 110 and the memory 116. In at least some embodiments, the controller 118 interfaces with other controllers or processing units for other calculations. In at least some embodiments, the controller 118 is part of a microcontroller or a larger control unit. In at least some embodiments, the controller 118 is of a type used to manage calculations in embedded systems, robotics, and other automated systems. In at least some embodiments, the integrated circuit 100 includes a controller 118 configured to send weight values ​​for each of a plurality of MAC units from memory 116 to the weight register of the MAC unit, send activation values ​​from memory to the upstream MAC unit of the plurality of MAC units, and store the sum of outputs generated by the last MAC unit of the plurality of MAC units in memory 116. In at least some embodiments, the controller 118 is further configured to perform one of point-by-point convolution and depth-direction convolution.

[0022] FIG. 2 is a schematic diagram of systolic array 210 according to at least some embodiments of the present disclosure. Systolic array 210 communicates with memory 216 and includes a plurality of MAC units such as MAC units 220A, 220B, and 220C, activation input lines 222, activation input line connector 223, register input lines 224, intermediate result input lines 226, and output line 228. Memory 216 is substantially similar to memory 116 of FIG. 1 in its structure and function, unless otherwise specifically described.

[0023] Memory 216 communicates with a plurality of MAC units via input lines such as activation input lines 222. In at least some embodiments, memory 216 is configured to transmit weight values and activation values to the plurality of MAC units via the input lines. In at least some embodiments, memory 216 is configured to receive and store intermediate and final results of calculations via output lines such as output line 228.

[0024] A plurality of MAC units, such as MAC units 220A, 220B, and 220C, communicate with memory 216. In at least some embodiments, a plurality of MAC units, such as MAC units 220A, 220B, and 220C, are configured to perform MAC operations for convolutional calculations. In at least some embodiments, each MAC unit is configured to store weight values and activation values in dedicated registers. In at least some embodiments, the plurality of MAC units are configured to receive weight values from memory 216. In at least some embodiments, each MAC unit is configured to receive an activation value from either memory 216 or an upstream MAC unit. In at least some embodiments, each MAC unit is configured to transmit an intermediate result to a downstream MAC unit or return it to memory 216. In at least some embodiments, each MAC unit is configured to perform basic arithmetic operations such as multiplication and addition. In at least some embodiments, the plurality of MAC units are part of another processing unit or an arithmetic logic unit (ALU). In at least some embodiments, the plurality of MAC units are arranged in columns of a systolic array. For example, MAC units 220A, 220B, and 220C are arranged in a column. In at least some embodiments, the systolic array includes a plurality of columns forming a matrix of MAC units. In at least some embodiments, each MAC unit is configured to store weight values and activation values, transmit an activation value received from one of an activation register and an input line storing the activation value, multiply the weight values and the transmitted activation value to generate a product value, and add the product value and an input sum value to generate an output sum value.

[0025] Activation input lines, such as activation input line 222, connect memory 216 to multiple MAC units. In at least some embodiments, the activation input line of each MAC unit among the multiple MAC units is shared among the corresponding MAC units in each column. In at least some embodiments, the corresponding MAC units in each column sharing an activation input line are referred to as a row of MAC units. For example, activation input line 222 is configured for transmitting activation values ​​to MAC unit 220B and other MAC units among the multiple MAC units connected to input line 222. In at least some embodiments, the activation input line is configured for general data transmission. In at least some embodiments, the activation input line is part of a larger data transmission network. In at least some embodiments, the activation input line includes electrical wiring or traces suitable for integrated circuits.

[0026] Activation input line connectors, such as activation input line connector 223, connect the activation input line to an upstream MAC unit. In at least some embodiments, each activation input line connector is configured to route the data transmitted through the activation input line to the MAC unit located at the top of the row of MAC units. In at least some embodiments, the activation input line connector enables the use of all MAC units in the execution of depth convolution with parallel processing of inputs and outputs. In at least some embodiments, the connection between the activation input line connector and the activation input line is fixed.

[0027] Register input lines, such as register input line 224, connect upstream MAC units to downstream MAC units. In at least some embodiments, each register input line is configured to transmit an activation value from an upstream MAC unit to the nearest downstream MAC unit. In at least some embodiments, each register input line is configured to sequentially connect the registers of MAC units.

[0028] Intermediate result input lines, such as intermediate result input line 226, connect upstream MAC units to downstream MAC units. In at least some embodiments, each intermediate result input line is configured to transmit intermediate results from an upstream MAC unit to the nearest downstream MAC unit. In at least some embodiments, each intermediate result input line is configured to connect the adders of sequential MAC units.

[0029] Output lines, such as output line 228, connect MAC units to memory 216. In at least some embodiments, each output line is configured to send the output sum value from the last (most downstream) MAC unit in each column of the systolic array 210 to memory 216. In at least some embodiments, each output line is configured to connect the adder of the last MAC unit to memory 216.

[0030] Figure 3 is a schematic diagram of a MAC unit 320 according to at least some embodiments of the present disclosure. The MAC unit 230 includes an activation register 330, a multiplexer 332, a weight register 334, a multiplier 336, and an adder 338. In at least some embodiments, each MAC unit comprises a weight register configured to store weight values, an activation register configured to store activation values, a multiplexer configured to transmit activation values ​​received from one of the activation registers and input lines, a multiplier configured to multiply the weight values ​​from the weight register and the activation value transmitted from the multiplexer to produce a product, and an adder configured to add the product from the multiplier and the input sum to produce an output sum.

[0031] The activation register 330 communicates with the multiplier 336 via the multiplexer 332. In at least some embodiments, the activation register 330 is configured to store activation values ​​for use in depth convolution with parallel processing of inputs and outputs. In at least some embodiments, the activation register 330 is configured to transmit activation values ​​to the multiplier 336 via the multiplexer 332. In at least some embodiments, the activation register 330 is configured to receive register input activation values, such as register input activation value 324A, from the activation register of the nearest upstream MAC unit. In at least some embodiments, the activation register 330 is configured to transmit register output activation values, such as register output activation value 324B, to the activation register of the nearest downstream MAC unit. In at least some embodiments, the activation register 330 is typically implemented in a flip-flop or latch in an integrated circuit. In at least some embodiments, the activation register 330 is of a type used in various integrated circuits for temporary data storage, such as in registers within a CPU. In at least some embodiments, where MAC unit 320 is the upstream MAC unit of at least one column, the activation register 330 is configured to receive an activation value via the input line of the downstream MAC unit of at least one column.

[0032] The multiplexer 332 is configured to selectively connect the activation register 330 and the multiplier 336. In at least some embodiments, the multiplexer 332 is configured to send either an activation value from the activation register 330 for use in depth convolution with parallel processing of inputs and outputs, or an activation value from the input lines for use in other calculations such as point-by-point convolution, to the multiplier 336. In at least some embodiments, the multiplexer 332 is configured to select between the activation register 330 and the input lines based on a signal from a controller, such as the controller 118 in Figure 1. In at least some embodiments, the multiplexer 332 is configured to form a direct connection from either the activation register 330 or the input lines to the multiplier 336. In at least some embodiments, the multiplexer 332 is a digital multiplexer circuit used to select between multiple input signals, which is typically found in FPGA or ASIC designs. In at least some embodiments, the multiplexer 332 is of a type commonly used in data routing, signal selection, and control systems.

[0033] The weight register 334 is connected to the multiplier 336. In at least some embodiments, the weight register 334 is configured to store weight values ​​for use in depth convolution and other calculations with parallel input and output processing. In at least some embodiments, the weight register 334 is configured to send weight values ​​to the multiplier 336. In at least some embodiments, the weight register 334 is configured to send the same weight values ​​to the multiplier 336 for multiple sequential calculations. In at least some embodiments, the weight register 334 is configured to receive weight values ​​from on-chip memory, such as memory 216 in Figure 2. In at least some embodiments, the weight register 334 is typically implemented in a flip-flop or latch in an integrated circuit. In at least some embodiments, the weight register 334 is of a type used in various integrated circuits for temporary data storage, such as in registers within a CPU.

[0034] The multiplier 336 communicates with the activation register 330 and input lines via the multiplexer 332, and also communicates with the adder 338. In at least some embodiments, the multiplier 336 is configured to multiply an activation value and a weight value to produce a product value. In at least some embodiments, the multiplier 336 is configured to receive the activation value from the multiplexer 332 and the weight value from the weight register 334, and send the product value to the adder 338. In at least some embodiments, the multiplier 336 is configured to multiply two data values. In at least some embodiments, the multiplier 336 is implemented as a digital multiplier circuit typically found in FPGA or ASIC designs. In at least some embodiments, the multiplier 336 is of a type used in digital signal processing, arithmetic units in CPUs, and graphics processing units (GPUs).

[0035] The adder 338 communicates with the multiplier 336. In at least some embodiments, the adder 338 is configured to add the product from the multiplier 336 to the input sum 326A to produce an output sum 326B. In at least some embodiments, the adder 338 is configured to receive the product from the multiplier 336 and the input sum 326A from the adder of the upstream MAC unit. In at least some embodiments, the adder 338 is configured to send the output sum 326B to the adder of the downstream MAC unit or to on-chip memory such as memory 216 in Figure 2. In at least some embodiments, the adder 338 is configured to generally perform the addition of two data values. In at least some embodiments, the adder 338 is implemented as a digital adder circuit, as is typically found in FPGA or ASIC designs. In at least some embodiments, the adder 338 is of the type used in arithmetic logic units (ALUs) within CPUs, digital signal processing, and control systems. In at least some embodiments, for example, in the case of the upstream MAC unit in a column, there is no upstream MAC unit that receives the input sum, so the adder is not included in the MAC unit, and the product generated by the multiplier is sent as the output sum to the adder of the downstream MAC unit.

[0036] Figure 4 is a schematic diagram of a depth convolution process according to at least some embodiments of the present disclosure. The diagram includes channel kernels 440, 441, and 442 and channel activation matrices 444, 445, and 446 throughout periods T2, T3, and T4.

[0037] To perform depth convolution with parallel processing of input and output, the upstream MAC unit, such as MAC unit 220A in Figure 2, receives one activation value A0 for each channel of the systolic array during the first period T0. In the next period T1, the activation value A0 is transmitted from the upstream MAC unit to the nearest downstream MAC unit, such as MAC unit 220B in Figure 2, and the upstream MAC unit receives the activation value A1. No calculations are performed between periods T0 and T1.

[0038] During the next period T2, activation value A1 is transmitted from the upstream MAC unit to the nearest downstream MAC unit, activation value A0 is transmitted from the nearest downstream MAC unit to the next nearest downstream MAC unit, and the upstream MAC unit receives activation value A2. During period T2, the MAC units have activation values ​​and weight values ​​suitable for performing calculations.

[0039] Figure 5A is a schematic diagram of the systolic array 510 in period T2 according to at least some embodiments of the present disclosure. Each column of MAC units in the systolic array 510 stores weight values ​​and activation values ​​for channels. For example, the upstream MAC unit 520A stores the weight value W2 and activation value A2 for channel CH1, the next downstream MAC unit 520B stores the weight value W1 and activation value A1 for channel CH1, and the next downstream MAC unit 520C stores the weight value W0 and activation value A0 for channel CH1. In other words, the systolic array 510 is in a state for performing a depth convolution calculation with parallel processing of inputs and outputs.

[0040] During the next period T3, activation value A2 is transmitted from the upstream MAC unit to the next downstream MAC unit, activation value A1 is transmitted from the next downstream MAC unit to the next downstream MAC unit, and the upstream MAC unit receives activation value A3. During period T3, the MAC units have activation values ​​and weight values ​​suitable for performing depth convolution calculations with parallel processing of inputs and outputs.

[0041] Figure 5B is a schematic diagram of the systolic array 510 in period T3 according to at least some embodiments of the present disclosure. Each column of MAC units in the systolic array 510 stores weight values ​​and activation values ​​for channels. For example, the upstream MAC unit 520A stores the weight value W2 and activation value A3 for channel CH1, the next downstream MAC unit 520B stores the weight value W1 and activation value A2 for channel CH1, and the next downstream MAC unit 520C stores the weight value W0 and activation value A1 for channel CH1.

[0042] During the next period T4, activation value A3 is transmitted from the upstream MAC unit to the nearest downstream MAC unit, activation value A2 is transmitted from the nearest downstream MAC unit to the next nearest downstream MAC unit, and the upstream MAC unit receives activation value A4. During period T4, the MAC units have activation values ​​and weight values ​​suitable for performing depth convolution calculations with parallel processing of inputs and outputs.

[0043] Figure 5C is a schematic diagram of the systolic array 510 in period T4 according to at least some embodiments of the present disclosure. Each column of MAC units in the systolic array 510 stores weight values ​​and activation values ​​for channels. For example, the upstream MAC unit 520A stores the weight value W2 and activation value A4 for channel CH1, the next downstream MAC unit 520B stores the weight value W1 and activation value A3 for channel CH1, and the next downstream MAC unit 520C stores the weight value W0 and activation value A2 for channel CH1.

[0044] Figure 6 shows a computational flow for performing convolution using a systolic array according to at least some embodiments of the present disclosure. In at least some embodiments, the computational flow provides a method for performing convolution using a systolic array according to at least some embodiments of the present disclosure. In at least some embodiments, the method is performed by a controller of an integrated circuit, such as the controller 118 in Figure 1.

[0045] In S650, the controller determines whether the operation is a depth convolution. In response to the controller's determination that the convolution operation is not a depth convolution, the operation flow proceeds in S656 to set a multiplexer on the line input. In response to the controller's determination that the operation is a depth convolution, the operation flow proceeds in S652 to set a multiplexer on the register input. In at least some embodiments, the operation is specified by a host machine, such as the host computer 102 in Figure 1.

[0046] In S652, the controller sets up a multiplexer on the register input. In at least some embodiments, the controller sets up a multiplexer on the register input by configuring the multiplexer to form a connection from the activation register to the multiplier. In at least some embodiments, the controller sets up a multiplexer to route the activation value from the register.

[0047] In S654, the controller performs depth convolution. In at least some embodiments, the controller performs depth convolution with parallel processing of inputs and outputs. In at least some embodiments, the controller performs depth convolution with parallel processing of inputs and outputs using a configured systolic array. In at least some embodiments, the controller performs depth convolution according to the operation flow in Figure 7, which is described below. In at least some embodiments, the controller is configured to perform depth convolution by advancing the activation value from the activation register of the nearest upstream MAC unit to the activation register of each MAC unit, and selecting the activation register for transmission by each multiplexer.

[0048] In S656, the controller sets the multiplexer to a line input. In at least some embodiments, the controller sets the multiplexer to a line input by configuring the multiplexer to form a connection from the input line to the multiplier. In at least some embodiments, the controller sets the multiplexer to route the activation value from the input line.

[0049] In S658, the controller performs point-by-point convolution. In at least some embodiments, the controller performs point-by-point convolution using a configured systolic array. In at least some embodiments, the controller is configured to perform point-by-point convolution by transmitting the activation value of each MAC unit among a plurality of MAC units via the input line of the MAC unit, and by selecting the input line for transmission by each multiplexer.

[0050] Figure 7 shows a computational flow for depth convolution with parallel input and output processing according to at least some embodiments of the present disclosure. In at least some embodiments, the computational flow provides a method for a computational flow for depth convolution with parallel input and output processing according to at least some embodiments of the present disclosure. In at least some embodiments, the method is performed by a controller of an integrated circuit, such as the controller 118 in Figure 1.

[0051] In the S760, the controller or a part thereof sets weight values. In at least some embodiments, the controller transmits the weight values ​​to a weight register. In at least some embodiments, the controller causes memory to transmit the weight values ​​to the weight register of the MAC unit in the systolic array. In at least some embodiments, the controller initializes a MAC unit having weights for a depth-direction convolution operation.

[0052] In S762, the controller or a part thereof inputs an activation value. In at least some embodiments, the controller transmits the activation value to an upstream MAC unit. In at least some embodiments, the controller causes memory to transmit the activation value to the activation register of an upstream MAC unit in a systolic array. In at least some embodiments, the controller transmits each activation value via different input lines of a systolic array. In at least some embodiments, the controller is configured to transmit the activation value to the activation register of an upstream MAC unit via the input lines of an upstream MAC unit. In at least some embodiments, the controller is configured to transmit the activation value to at least some upstream MAC units via an activation input line connector that connects the input line to the upstream MAC unit of a column, such as the activation input line connector 223 in Figure 2. In at least some embodiments, the controller is configured to transmit the activation value to the activation register of an upstream MAC unit of at least one column via the input lines of a downstream MAC unit of at least one column.

[0053] In S764, the controller or a part thereof determines whether the activation value has advanced sufficiently. In response to determining that the activation value has not advanced sufficiently, the arithmetic flow proceeds in S768 to advance the activation value. In response to determining that the activation value has advanced sufficiently, the arithmetic flow proceeds in S765 to execute the MAC operation. In at least some embodiments, the controller determines whether the activation value has advanced sufficiently through the activation register, thereby determining whether each MAC unit having a weight value also has an activation value. In at least some embodiments, the controller determines whether the systolic array is ready to perform the MAC operation. In at least some embodiments, the controller determines whether the systolic array is ready to begin performing a depth convolution with parallel processing of inputs and outputs.

[0054] In S765, the controller or a part thereof performs MAC calculations. In at least some embodiments, the controller causes the MAC unit to multiply a weight value and an activation value to generate a product value. In at least some embodiments, the controller causes the MAC unit to accumulate the product value and the input sum value to generate an output sum value. In at least some embodiments, the controller causes the downstream MAC unit to send the output sum value to memory. In at least some embodiments, the controller causes the output sum value to be stored in memory.

[0055] In S767, the controller or a part thereof determines whether all activation values ​​have been input. In response to determining that not all activation values ​​have been input, the calculation flow proceeds to activation value advancement in S768. In response to determining that not all activation values ​​have been input, the calculation flow terminates. In at least some embodiments, the controller determines whether all activation values ​​of the channel activation matrix have been input. In at least some embodiments, the controller determines whether the depth convolution process is complete or whether more activation values ​​need to be processed. In at least some embodiments, the controller tracks the input of activation values ​​as specified by a host machine, such as the host computer 102 in Figure 1.

[0056] In S768, the controller or a part thereof advances the activation value. In at least some embodiments, the controller advances the activation value from the upstream activation register to the next downstream register. In at least some embodiments, the controller prepares a systolic array for the next set of MAC operations. In at least some embodiments, the controller moves the activation value through the systolic array, thereby enabling parallel processing of multiple channels.

[0057] In at least some embodiments, depth convolution is performed on a kernel having multiple rows of weight values. In at least some embodiments, the computation flow in Figure 7 is performed once for each row of weight values ​​in the kernel. In at least some embodiments, depth convolution is performed on a kernel having more rows of weight values ​​than the number of MAC units per column of the systolic array. In at least some embodiments, such that not all MAC units of the systolic array contain activation registers, depth convolution is performed on a kernel having more rows of weight values ​​than the number of registers per column of the systolic array. In at least some embodiments, the computation flow in Figure 7 is performed an additional number of times until all weight values ​​in one row of the kernel have been processed.

[0058] While embodiments of the present invention have been described, the technical scope of any claimed subject matter is not limited to the embodiments described above. Those skilled in the art will understand that various modifications and improvements are possible to the embodiments described above. Those skilled in the art will also understand that such modified or improved embodiments fall within the technical scope of the present invention, within the scope of the claims.

[0059] The operations, procedures, steps, and stages of each process performed by the apparatus, system, program, and method shown in the claims, embodiments, or figures can be performed in any order, unless their order is indicated by "prior to" or "before," and unless the output from a previous process is used in a later process. Even if a process flow is described in the claims, embodiments, or figures using words such as "first" or "next," such description does not necessarily mean that the processes must be performed in the order described.

[0060] In at least some embodiments, depth convolution with parallel processing of inputs and outputs is performed by a plurality of multiplication-accumulation (MAC) units, each MAC unit having a weight register configured to store weight values, an activation register configured to store activation values, a multiplexer configured to transmit the activation values ​​received from the activation register and one of the input lines, a multiplier configured to multiply the weight values ​​from the weight register and the activation value transmitted from the multiplexer to produce a product, and an adder configured to add the product from the multiplier and the input sum to produce an output sum; and a controller configured to transmit the weight values ​​of each MAC unit from the memory to the weight register of the MAC unit, transmit the activation values ​​from the memory to the upstream MAC unit of the plurality of MAC units, and store the output sum generated by the last MAC unit of the plurality of MAC units in the memory. In at least some embodiments, the controller is further configured to perform one of point-by-point convolution and depth convolution. In at least some embodiments, the controller is configured to perform point-by-point convolution by transmitting the activation value of each MAC unit among a plurality of MAC units via the MAC unit's input line and selecting the input line for transmission by each multiplexer. In at least some embodiments, the controller is configured to perform depth convolution by advancing the activation value from the activation register of the nearest upstream MAC unit to the activation register of each MAC unit and selecting the activation register for transmission by each multiplexer. In at least some embodiments, the plurality of MAC units are arranged in columns of a systolic array. In at least some embodiments, the systolic array includes a plurality of columns that form a matrix of MAC units.In at least some embodiments, the activation input line of each MAC unit among a plurality of MAC units is shared among the corresponding MAC units in each column. In at least some embodiments, the activation register of the upstream MAC unit of at least one column is configured to receive an activation value via the input line of the downstream MAC unit of at least one column. In at least some embodiments, the activation register of the upstream MAC unit is configured to receive an activation value via the input line of the upstream MAC unit.

[0061] In at least some embodiments, depth convolution with parallel input and output processing is performed by a plurality of multiplication (MAC) units, each MAC unit configured to store weight values ​​and activation values, transmit activation values ​​received from one of the activation registers and input lines storing the activation values, multiply the weight values ​​and the transmitted activation values ​​to produce a product, and add the product and the input sum to produce an output sum; and a controller configured to transmit the weight values ​​of each of the plurality of MAC units from the memory to the MAC units, transmit the activation values ​​from the memory to the upstream MAC units of the plurality of MAC units, and store the output sum generated by the last MAC unit of the plurality of MAC units in the memory. In at least some embodiments, the controller is further configured to perform one of point-by-point convolution and depth convolution. In at least some embodiments, the controller is configured to perform point-by-point convolution by transmitting the activation value of each MAC unit among a plurality of MAC units via the MAC unit's input line and selecting the input line for transmission by each multiplexer. In at least some embodiments, the controller is configured to perform depth convolution by advancing the activation value from the nearest upstream MAC unit to each MAC unit and selecting the activation register for transmission by each multiplexer. In at least some embodiments, the plurality of MAC units are arranged in columns of a systolic array. In at least some embodiments, the systolic array includes a plurality of columns that form a matrix of MAC units. In at least some embodiments, the activation input line of each MAC unit among the plurality of MAC units is shared among the corresponding MAC units in each column.In at least some embodiments, the controller is configured to transmit the activation value of the activation register of the upstream MAC unit of at least one column via the input line of the downstream MAC unit of at least one column. In at least some embodiments, the controller is configured to transmit the activation value of the activation register of the upstream MAC unit via the input line of the upstream MAC unit.

[0062] In at least some embodiments, depth convolution with parallel processing of inputs and outputs is performed by the steps of: sending weight values ​​of each MAC unit among a plurality of MAC units of the integrated circuit from the integrated circuit memory to the weight register of the MAC unit; sending activation values ​​from the memory to the activation register of each upstream MAC unit among the plurality of MAC units; sending the activation values ​​from the activation register and one of the input lines to the multiplier of the MAC unit by the multiplexer of each MAC unit among the plurality of MAC units connected to the activation register of the MAC unit; generating a product value by multiplying the weight values ​​from the weight register and the activation values ​​sent from the multiplexer by the multiplexer of the MAC unit and the multiplier of each MAC unit among the plurality of MAC units connected to the weight register of the MAC unit; generating an output sum value by adding the product value from the multiplier and the input sum value by the adder of each downstream MAC unit among the plurality of MAC units connected to the multiplier of the MAC unit; and storing the output sum value generated by the last MAC unit among the plurality of MAC units in the memory. In at least some embodiments, the method includes the steps of advancing the activation value from the nearest upstream MAC unit to each MAC unit, and The process further includes selecting the activation register for transmission by each multiplexer.

[0063] The foregoing outlines the features of several embodiments so that those skilled in the art may better understand the aspects of this disclosure. Those skilled in the art should understand that this disclosure can be readily used as a basis for designing or modifying other processes and structures to perform the same purposes and / or achieve the same advantages as the embodiments introduced herein. Those skilled in the art should also recognize that such equivalent configurations do not depart from the spirit and scope of this disclosure, and that various changes, substitutions, and modifications described herein are possible without departing from the spirit and scope of this disclosure. 。 [Item 1] Multiple multiplication-accumulation (MAC) units, each MAC unit is, A weight register configured to store weight values, An activation register configured to store the activation value, A multiplexer configured to transmit the activation value received from one of the activation registers and input lines, A multiplier configured to multiply the weight values ​​from the weight register and the activation value transmitted from the multiplexer to generate a product value, and An adder configured to generate an output sum by adding the product from the multiplier and the input sum. It has, The memory communicating with the aforementioned multiple MAC units, and The weight values ​​of each MAC unit among the plurality of MAC units are transmitted from the memory to the weight register of the corresponding MAC unit. The activation value is transmitted from the memory to the upstream MAC unit among the plurality of MAC units. The sum of the outputs generated by the last MAC unit among the plurality of MAC units is stored in the memory. Controller configured in this way An integrated circuit equipped with the following features. [Item 2] The integrated circuit according to item 1, wherein the controller is further configured to perform one of point-by-point convolution and depth-direction convolution. [Item 3] The aforementioned controller, The activation value of each MAC unit among the plurality of MAC units is transmitted via the input line of the corresponding MAC unit, and Select the input line for transmission by each multiplexer. The integrated circuit described in item 2, configured to perform point-by-point convolution. [Item 4] The aforementioned controller, The activation value is advanced from the activation register of the nearest upstream MAC unit to the activation register of each MAC unit, and Select the activation register for transmission by each multiplexer. The integrated circuit described in item 2, configured to perform convolution in the depth direction by... [Item 5] The aforementioned multiple MAC units are integrated circuits according to any one of items 1 to 4, arranged in a column of a systolic array. [Item 6] The integrated circuit according to item 5, wherein the systolic array includes a plurality of columns that form a matrix of MAC units. [Item 7] The integrated circuit according to item 6, wherein the input line of each of the plurality of MAC units is shared among the corresponding MAC units in each column. [Item 8] The integrated circuit according to item 7, wherein the activation register of the upstream MAC unit of at least one column is configured to receive the activation value via the input line of the downstream MAC unit of at least one column. [Item 9] The integrated circuit according to any one of items 1 to 4, wherein the activation register of the upstream MAC unit is configured to receive the activation value via the input line of the upstream MAC unit. [Item 10] Multiple multiplication-accumulation (MAC) units, each MAC unit is, Store the weight values ​​and activation values, The activation value received from one of the activation registers and input lines storing the aforementioned activation value is transmitted. The weight value and the transmitted activation value are multiplied to generate the product value, The output sum is generated by adding the aforementioned product value and the input sum value. It is configured in such a way, The memory communicating with the aforementioned multiple MAC units, and The weight values ​​of each MAC unit among the plurality of MAC units are transmitted from the memory to the corresponding MAC unit. The activation value is transmitted from the memory to the upstream MAC unit among the plurality of MAC units. The sum of the outputs generated by the last MAC unit among the plurality of MAC units is stored in the memory. Controller configured in this way An integrated circuit equipped with the following features. [Item 11] The integrated circuit according to item 10, wherein the controller is further configured to perform one of point-by-point convolution and depth-direction convolution. [Item 12] The aforementioned controller, The activation value of each MAC unit among the plurality of MAC units is transmitted via the input line of the corresponding MAC unit, and Select the input line for transmission by each multiplexer. The integrated circuit described in item 11, configured to perform point-by-point convolution. [Item 13] The aforementioned controller, The activation value is advanced from the nearest upstream MAC unit to each MAC unit, and Select the activation register for transmission by each multiplexer. The integrated circuit described in item 11, configured to perform convolution in the depth direction by [Item 14] The plurality of MAC units are integrated circuits according to any one of items 10 to 13, arranged in a column of a systolic array. [Item 15] The integrated circuit according to item 14, wherein the systolic array includes a plurality of columns that form a matrix of MAC units. [Item 16] The integrated circuit according to item 15, wherein the input line of each MAC unit among the plurality of MAC units is shared among the corresponding MAC units in each column. [Item 17] The integrated circuit according to item 16, wherein the controller is configured to transmit the activation value to the activation register of the upstream MAC unit of the at least one column via the input line of the downstream MAC unit of the at least one column. [Item 18] The integrated circuit according to any one of items 10 to 13, wherein the controller is configured to transmit the activation value to the activation register of the upstream MAC unit via the input line of the upstream MAC unit. [Item 19] The step of transmitting the weight values ​​of each MAC unit among the multiple MAC units of the integrated circuit from the memory of the integrated circuit to the weight register of the corresponding MAC unit, The step of sending the activation value from the memory to the activation register of each upstream MAC unit among the plurality of MAC units, A step in which, for each of the plurality of MAC units, a multiplexer connected to the activation register of the corresponding MAC unit transmits the activation value from one of the activation registers and input lines to the multiplier of the corresponding MAC unit. A step in which, for each MAC unit among the plurality of MAC units, a multiplier connected to the multiplexer and the weight register of the corresponding MAC unit multiplies the weight value from the weight register and the activation value transmitted from the multiplexer to generate a product value, A step in which an adder connected to the multiplier of the corresponding MAC unit of each downstream MAC unit among the plurality of MAC units adds the product value from the multiplier and the input sum value to generate an output sum value, and The step of storing the sum of outputs generated by the last MAC unit among the plurality of MAC units into the memory. A method for providing this. [Item 20] The step of passing the aforementioned activation value from the nearest upstream MAC unit to each MAC unit, and The step of selecting the activation register for transmission by each multiplexer. The method described in item 19, further comprising the above.

[0064]

Claims

1. Multiple multiplication (MAC) units, each MAC unit is, A weight register configured to store weight values, An activation register configured to store the activation value, A multiplexer configured to transmit the activation value received from one of the activation registers and input lines, A multiplier configured to multiply the weight values ​​from the weight register and the activation value transmitted from the multiplexer to generate a product value, and An adder configured to generate an output sum by adding the product from the multiplier and the input sum. It has, The memory communicating with the aforementioned multiple MAC units, The weight values ​​of each MAC unit among the plurality of MAC units are transmitted from the memory to the weight register of the corresponding MAC unit. The activation value is transmitted from the memory to the upstream MAC unit among the plurality of MAC units. The sum of the output values ​​generated by the last MAC unit among the plurality of MAC units is stored in the memory. Controller configured in this way Equipped with, The aforementioned controller, The activation value of each MAC unit among the plurality of MAC units is transmitted via the input line of the corresponding MAC unit, and Select the input line for transmission by each multiplexer. This performs point-by-point convolution, and also, The activation value is advanced from the activation register of the nearest upstream MAC unit to the activation register of each MAC unit, and Select the activation register for transmission by each multiplexer. This performs convolution in the depth direction. It is further structured in such a way. Integrated circuit.

2. The integrated circuit according to claim 1, wherein the plurality of MAC units are arranged in a row of a systolic array.

3. The integrated circuit according to claim 2, wherein the systolic array includes a plurality of columns that form a matrix of MAC units.

4. The integrated circuit according to claim 3, wherein the input line of each of the plurality of MAC units is shared among the corresponding MAC units in each row.

5. The integrated circuit according to claim 4, wherein the activation register of the upstream MAC unit of at least one column is configured to receive the activation value via the input line of the downstream MAC unit of at least one column.

6. The integrated circuit according to any one of claims 1 to 4, wherein the activation register of the upstream MAC unit is configured to receive the activation value via the input line of the upstream MAC unit.

7. A plurality of multiplication-accumulation (MAC) units, each MAC unit is A weight register configured to store weight values, An activation register configured to store the activation value, A multiplexer configured to transmit the activation value received from one of the activation registers and input lines, A multiplier configured to multiply the weight values ​​from the weight register and the activation value transmitted from the multiplexer to generate a product value, and An adder configured to generate an output sum by adding the product from the multiplier and the input sum. It has, The memory communicating with the aforementioned multiple MAC units, The weight values ​​of each MAC unit among the plurality of MAC units are transmitted from the memory to the weight register of the corresponding MAC unit. The activation value is transmitted from the memory to the upstream MAC unit among the plurality of MAC units. The sum of the output values ​​generated by the last MAC unit among the plurality of MAC units is stored in the memory. Controller configured in this way Equipped with, The aforementioned MAC units are arranged in a column of a systolic array, The systolic array includes a plurality of columns that form a matrix of MAC units, The input lines of each of the aforementioned MAC units are shared among the corresponding MAC units in each column. The activation register of the upstream MAC unit of at least one column is configured to receive the activation value via the input line of the downstream MAC unit of at least one column. Integrated circuit.

8. A plurality of multiplication-accumulation (MAC) units, each MAC unit is A weight register configured to store weight values, An activation register configured to store the activation value, A multiplexer configured to transmit the activation value received from one of the activation registers and input lines, A multiplier configured to multiply the weight values ​​from the weight register and the activation value transmitted from the multiplexer to generate a product value, and An adder configured to generate an output sum by adding the product from the multiplier and the input sum. It has, The memory communicating with the aforementioned multiple MAC units, The weight values ​​of each MAC unit among the plurality of MAC units are transmitted from the memory to the weight register of the corresponding MAC unit. The activation value is transmitted from the memory to the upstream MAC unit among the plurality of MAC units. The sum of the output values ​​generated by the last MAC unit among the plurality of MAC units is stored in the memory. Controller configured in this way Equipped with, The aforementioned MAC units are arranged in a column of a systolic array, The systolic array includes a plurality of columns that form a matrix of MAC units, The input lines of each of the aforementioned MAC units are shared among the corresponding MAC units in each column. The controller is further configured to transmit the activation value to the activation register of the MAC unit upstream of the MAC unit of at least one column via the input line of the MAC unit downstream of the MAC unit of at least one column. Integrated circuit.

9. Multiple multiplication (MAC) units, each MAC unit is, Store the weight values ​​and activation values, The activation value received from one of the activation registers and input lines storing the aforementioned activation value is transmitted. The weight value and the transmitted activation value are multiplied to generate the product value, The output sum is generated by adding the aforementioned product value and the input sum value. It is configured in such a way, The memory communicating with the aforementioned multiple MAC units, The weight values ​​of each MAC unit among the plurality of MAC units are transmitted from the memory to the corresponding MAC unit. The activation value is transmitted from the memory to the upstream MAC unit among the plurality of MAC units. The sum of the outputs generated by the last MAC unit among the plurality of MAC units is stored in the memory. Controller configured in this way Equipped with, The aforementioned controller, The activation value of each MAC unit among the plurality of MAC units is transmitted via the input line of the corresponding MAC unit, and Select the input line for transmission by each multiplexer. This performs point-by-point convolution, and also, The activation value is advanced from the nearest upstream MAC unit to each MAC unit, and Select the activation register for transmission by each multiplexer. This performs convolution in the depth direction. It is further structured in such a way. Integrated circuit.

10. The integrated circuit according to claim 9, wherein the plurality of MAC units are arranged in a row of a systolic array.

11. The integrated circuit according to claim 10, wherein the systolic array includes a plurality of columns that form a matrix of MAC units.

12. The integrated circuit according to claim 11, wherein the input line of each of the plurality of MAC units is shared among the corresponding MAC units in each row.

13. The integrated circuit according to claim 12, wherein the controller is configured to transmit the activation value to the activation register of the upstream MAC unit of the at least one column via the input line of the downstream MAC unit of the at least one column.

14. The integrated circuit according to any one of claims 9 to 12, wherein the controller is configured to transmit the activation value to the activation register of the upstream MAC unit via the input line of the upstream MAC unit.

15. Multiple multiplication (MAC) units, each MAC unit is Store the weight values ​​and activation values, The activation value received from one of the activation registers and input lines storing the aforementioned activation value is transmitted. The weight value and the transmitted activation value are multiplied to generate the product value, The output sum is generated by adding the aforementioned product value and the input sum value. It is configured in such a way, The memory communicating with the aforementioned multiple MAC units, The weight values ​​of each MAC unit among the plurality of MAC units are transmitted from the memory to the corresponding MAC unit. The activation value is transmitted from the memory to the upstream MAC unit among the plurality of MAC units. The sum of the outputs generated by the last MAC unit among the plurality of MAC units is stored in the memory. Controller configured in this way Equipped with, The aforementioned MAC units are arranged in a column of a systolic array, The systolic array includes a plurality of columns that form a matrix of MAC units, The input lines of each of the aforementioned MAC units are shared among the corresponding MAC units in each column. The activation register of the upstream MAC unit of at least one column is configured to receive the activation value via the input line of the downstream MAC unit of at least one column. Integrated circuit.

16. A plurality of multiplication-accumulation (MAC) units, each MAC unit is Store the weight values ​​and activation values, The activation value received from one of the activation registers and input lines storing the aforementioned activation value is transmitted. The weight value and the transmitted activation value are multiplied to generate the product value, The output sum is generated by adding the aforementioned product value and the input sum value. It is configured in such a way, The memory communicating with the aforementioned multiple MAC units, The weight values ​​of each MAC unit among the plurality of MAC units are transmitted from the memory to the corresponding MAC unit. The activation value is transmitted from the memory to the upstream MAC unit among the plurality of MAC units. The sum of the outputs generated by the last MAC unit among the plurality of MAC units is stored in the memory. Controller configured in this way Equipped with, The aforementioned MAC units are arranged in a column of a systolic array, The systolic array includes a plurality of columns that form a matrix of MAC units, The input lines of each of the aforementioned MAC units are shared among the corresponding MAC units in each column. The controller is further configured to transmit the activation value to the activation register of the MAC unit upstream of the MAC unit of at least one column via the input line of the MAC unit downstream of the MAC unit of at least one column. Integrated circuit.

17. A method, The step of transmitting the weight values ​​of each MAC unit among the multiple MAC units of the integrated circuit from the memory of the integrated circuit to the weight register of the corresponding MAC unit, The step of transmitting the activation value from the memory to the activation register of each upstream MAC unit among the plurality of MAC units, The step of transmitting the activation value from one of the activation registers and input lines to the multiplier of the corresponding MAC unit by a multiplexer connected to the activation register of the corresponding MAC unit of each of the plurality of MAC units, A step in which, in each of the plurality of MAC units, a multiplier connected to the multiplexer and the weight register of the corresponding MAC unit multiplies the weight value from the weight register and the activation value transmitted from the multiplexer to generate a product value, A step in which an adder connected to the multiplier of the corresponding MAC unit of each downstream MAC unit among the plurality of MAC units adds the product value from the multiplier and the input sum value to generate an output sum value, and Step 1: Store the sum of the output values ​​generated by the last MAC unit among the plurality of MAC units in the memory. Equipped with, When performing point-by-point convolution, The step of transmitting the activation value from one of the activation register and the input line to the multiplier of the corresponding MAC unit by the multiplexer is as follows: The multiplexer transmits the activation value from the input line to the multiplier of the corresponding MAC unit by the multiplexer. The aforementioned method, The step of selecting the input line for transmission by each multiplexer. It further includes, When performing convolution in the depth direction, The step of transmitting the activation value from one of the activation register and the input line to the multiplier of the corresponding MAC unit by the multiplexer is as follows: The multiplexer transmits the activation value from the activation register to the multiplier of the corresponding MAC unit. The aforementioned method, The step of advancing the activation value from the activation register of the nearest upstream MAC unit to the activation register of each MAC unit, and The step of selecting the activation register for transmission by each multiplexer. It also has, method.

18. A method, The step of transmitting the weight values ​​of each MAC unit among the multiple MAC units of the integrated circuit from the memory of the integrated circuit to the weight register of the corresponding MAC unit, The step of transmitting the activation value from the memory to the activation register of each upstream MAC unit among the plurality of MAC units, The step of transmitting the activation value from one of the activation registers and input lines to the multiplier of the corresponding MAC unit by a multiplexer connected to the activation register of the corresponding MAC unit of each of the plurality of MAC units, A step in which, in each of the plurality of MAC units, a multiplier connected to the multiplexer and the weight register of the corresponding MAC unit multiplies the weight value from the weight register and the activation value transmitted from the multiplexer to generate a product value, A step in which an adder connected to the multiplier of the corresponding MAC unit of each downstream MAC unit among the plurality of MAC units adds the product value from the multiplier and the input sum value to generate an output sum value, and Step 1: Store the sum of the output values ​​generated by the last MAC unit among the plurality of MAC units in the memory. Equipped with, The aforementioned MAC units are arranged in a column of a systolic array, The systolic array includes a plurality of columns that form a matrix of MAC units, The input lines of each of the aforementioned MAC units are shared among the corresponding MAC units in each column. The aforementioned method, The step of receiving the activation value via the input line of the MAC unit downstream of the at least one column, by the activation register of the MAC unit upstream of the column. A way to further enhance it.

19. A method, The step of transmitting the weight values ​​of each MAC unit among the multiple MAC units of the integrated circuit from the memory of the integrated circuit to the weight register of the corresponding MAC unit, The step of transmitting the activation value from the memory to the activation register of each upstream MAC unit among the plurality of MAC units, The step of transmitting the activation value from one of the activation registers and input lines to the multiplier of the corresponding MAC unit by a multiplexer connected to the activation register of the corresponding MAC unit of each of the plurality of MAC units, A step in which, in each of the plurality of MAC units, a multiplier connected to the multiplexer and the weight register of the corresponding MAC unit multiplies the weight value from the weight register and the activation value transmitted from the multiplexer to generate a product value, A step in which an adder connected to the multiplier of the corresponding MAC unit of each downstream MAC unit among the plurality of MAC units adds the product value from the multiplier and the input sum value to generate an output sum value, and Step 1: Store the sum of the output values ​​generated by the last MAC unit among the plurality of MAC units in the memory. Equipped with, The aforementioned MAC units are arranged in a column of a systolic array, The systolic array includes a plurality of columns that form a matrix of MAC units, The input lines of each of the aforementioned MAC units are shared among the corresponding MAC units in each column. The step of transmitting the activation value from the memory to the activation register of each upstream MAC unit among the plurality of MAC units includes transmitting the activation value to the activation register of the upstream MAC unit of at least one column via the input line of the downstream MAC unit of at least one column. method.

Images