Apparatus and method for pipelined multiply accumulator

Abstract

A circuit may include a vector arithmetic logic unit (ALU) including a multiplier, a first multiplexer, a second multiplexer, and an accumulator. The vector ALU may calculate a dot product of two or more vector inputs. A system may include two or more vector ALUs, and may divide a vector input into a plurality of segments. Each segment may be input to a respective vector ALU via a multiplexer, and the controller may route a partial sum of the respective ALU via one or more feedback paths, and the system may calculate a full dot product of the vector inputs.

Description

priority

[0001] This application claims priority to jointly owned U.S. Patent Application No. 63 / 541,096, filed September 28, 2023, the entire contents of which are incorporated herein by reference for all purposes. Technical Field

[0002] This disclosure relates to multiply-accumulate circuits and methods, and more particularly to an apparatus and method for a pipelined multiply-accumulate. Background Technology

[0003] Mathematical algorithms may require the use of dot product operations. As one example among many, in machine learning applications, dot product operations can encompass a large portion of the arithmetic operations in machine learning algorithms. Processors implemented on application-specific integrated circuits (ASICs) and field-programmable gate arrays (FPGAs) can include support for machine learning and can perform multiple dot product operations. Efficient computation of dot products can be a key performance parameter in these systems.

[0004] The dot product operation is a combination of vector multiplication and addition. The following equation defines the dot product of two vectors A and B of length N:

[0005]

[0006] The corresponding samples of vectors A and B are multiplied, and the product is then added to the previous product in turn to produce a single scalar output. In one example of various examples, the hardware multiplier accumulator can multiply vector values ​​and add the new multiplication outputs in turn to calculate the dot product result.

[0007] Multiplication and accumulation arithmetic operations can consume varying numbers of loops, making it difficult to fully utilize available loops in pipelined operations. In floating-point systems, accumulators can involve both shift and addition operations, which can increase the accumulator's loop time.

[0008] A pipelined multiply-accumulator is needed for high-speed dot product calculations. Summary of the Invention

[0009] The example in this paper implements a system for a pipelined multiply-accumulate.

[0010] According to one aspect, an apparatus may include a vector arithmetic logic unit (ALU) including at least two vector inputs. The vector ALU further includes a multiplier for taking inputs from the at least two vector inputs and for producing a multiplier output including the product of the at least two vector inputs. The vector ALU further includes a first multiplexer for selecting one of the multiplier output, an external input, and a feedback output as a first multiplexer output. The vector ALU further includes a second multiplexer for selecting one of the external input and the feedback output as a second multiplexer output. The vector ALU further includes an accumulator for adding the first multiplexer output to the second multiplexer output to generate the feedback output.

[0011] According to one aspect, a system includes a multiplexer for receiving two or more vector inputs. The system includes at least one vector arithmetic logic unit (ALU) for receiving at least one output of the multiplexer. The at least one vector ALU may include at least two ALU vector inputs provided by the multiplexer. The at least one vector ALU may include a multiplier for taking inputs from the at least two ALU vector inputs and for generating a multiplier output comprising the product of the at least two ALU vector inputs. The at least one vector ALU may include an accumulator for taking inputs from the multiplier output and a feedback output, wherein the accumulator adds the multiplier output and the feedback output. The at least one vector ALU may include a controller having an output feedback signal coupled to the multiplexer, the controller for receiving inputs from the at least one vector ALU and for generating the feedback signal based on the output of the at least one vector ALU.

[0012] According to one aspect, a method includes the steps of: dividing a first vector and a second vector into one or more segments; coupling the segment of the first vector and the second vector to one or more vector ALUs, the vector ALUs including at least a multiplier and an accumulator; computing the dot product of the segment of the first vector and the second vector by pipelined sampling of even-numbered and odd-numbered samples within the one or more vector ALUs; and computing the dot product of the first vector and the second vector by summing the outputs of the one or more vector ALUs. Attached Figure Description

[0013] Figure 1 This is one example of various examples of vector ALU devices.

[0014] Figure 2This example illustrates one of the various examples of a pipelined multiply-accumulate system used to compute dot products.

[0015] Figure 3 The timing diagram of a vector ALU device is shown.

[0016] Figure 4 An example is given of a method for calculating the dot product. Detailed Implementation

[0017] Figure 1 One example of various examples of the vector ALU 100 is shown. The vector ALU 100 may include multiply-accumulate circuitry.

[0018] The vector ALU 100 may include a multiplier 130. A first input 110 may be coupled to a first operand input 133 of the multiplier 130. A second input 120 may be coupled to a second operand input 134 of the multiplier 130. The multiplier 130 may include a clock signal input 137.

[0019] In operation, multiplier 130 can generate a multiplier output. The multiplier output can be a product 135, which includes the arithmetic product of the first operand input 133 and the second operand input 134. The vertical dashed line 131 can represent the cycle time of multiplier 130. Product 135 can be output one clock cycle after the first input 110 and the second input 120 are input to multiplier 130. The first input 110 and the second input 120 can be input to multiplier 130 at the first clock cycle, and product 135 can be valid at the second clock cycle. In other examples, product 135 can be valid after a different number of cycles.

[0020] The first input 110 and the second input 120 can be a sequence of samples, each sample arriving at the multiplier 130 at a rate of one sample per clock signal input 137.

[0021] External input 160 can be coupled to a first input of the first multiplexer 141. Product 135 can be coupled to a second input of the first multiplexer 141. First selection signal 143 can select one of the inputs of the first multiplexer 141 to couple to the first multiplexer output 148.

[0022] External input 160 can be coupled to a first input of the second multiplexer 142. Feedback output 180 can be coupled to a second input of the second multiplexer 142. A second selection signal 144 can select one of the inputs of the second multiplexer 142 to be coupled to the second multiplexer output 149.

[0023] The accumulator 150 may include a first operand input 155 and a second operand input 156. The accumulator 150 may include a clock signal input 157.

[0024] In operation, accumulator 150 can output accumulator output 190, which includes the arithmetic sum of first operand input 155 and second operand input 156. Vertical dashed lines 152 and 153 represent the cycle time of accumulator 150, and... Figure 1 The example illustrated may indicate that accumulator 150 may include a 3-cycle accumulator. Therefore, accumulator output 190 may be output two clock cycles after the first multiplexer output 148 and the second multiplexer output 149 are input to accumulator 150. Accumulator 150 may output feedback output 180, which includes the arithmetic sum of first operand input 155 and second operand input 156. Feedback output 180 may be output two clock cycles after the first multiplexer output 148 and the second multiplexer output 149 are input to accumulator 150. In other examples, the timing of feedback output 180 may differ. Figure 1 Examples illustrated in .

[0025] In operation, the vector ALU 100 performs dot product operations. As one example among various examples, the vector ALU 100 can compute the dot product of a first vector A and a second vector B. The first vector A can be a first sequence of samples. The second vector B can be a second sequence of samples. The vector ALU 100 can process sampled data, i.e., continuous data samples processed within a discrete time period defined by the clock signal input 137.

[0026] During a first time period, a first sample of vector A can be coupled to a first input 110, and a first sample of vector B can be coupled to a second input 120. Multiplier 130 can compute the product 135 of the first sample of vector A and the first sample of vector B as product 135. The product 135 of the first sample of vector A and the first sample of vector B is valid during a second time period. The second time period can occur after the first time period. First multiplexer 141 can couple product 135 to first multiplexer output 148. During the second time period, external input 160 can be set to logic zero, and second multiplexer 142 can couple external input 160 to second multiplexer output 149. Accumulator 150 can compute the sum of first multiplexer output 148 and second multiplexer output 149, and provide the computed sum at feedback output 180. Feedback output 180 can represent the product of the first sample of vector A and the first sample of vector B, and is valid during a fourth time period.

[0027] During a second time period (which precedes the third time period), a second sample of vector A can be coupled to the first input 110, and a second sample of vector B can be coupled to the second input 120. Multiplier 130 can compute the product of the second samples of vector A and vector B as product 135. Product 135 of the second samples of vector A and vector B is valid during the third time period. First multiplexer 141 can couple product 135 to first multiplexer output 148. During the third time period, first selection signal 143 can guide first multiplexer 141 to select product 135, and second multiplexer 142 can couple external input 160 to second multiplexer output 149. External input 160 can be set to zero. Accumulator 150 can compute the sum of first multiplexer output 148 and second multiplexer output 149, and provide the computed sum at feedback output 180. The feedback output 180 can represent the product of the second sample of vector A and the second sample of vector B, and it is valid in the fifth time period.

[0028] During the third time period (which precedes the fourth time period), a third sample of vector A can be coupled to the first input 110, and a third sample of vector B can be coupled to the second input 120. Multiplier 130 can compute the product of the third samples of vector A and vector B as a product 135. The product 135 of the third samples of vector A and vector B is valid during the fourth time period. First multiplexer 141 can couple the product 135 to first multiplexer output 148. During the fourth time period, feedback output 180 can represent the sum of the first samples of vector A and vector B. At the fourth time period and subsequent time periods or subsequent clock cycles, second multiplexer 142 can couple feedback output 180 to second multiplexer output 149. Accumulator 150 can compute the sum of first multiplexer output 148 and second multiplexer output 149 as feedback output 180. The feedback output can represent the sum of two values: the product of the first sample of vector A input from the second multiplexer output 149 and the first sample of vector B, and the product of the third sample of vector A input from the first multiplexer output 148 and the third sample of vector B. In this way, the dot product of odd-numbered samples (first, third, fifth, and subsequent samples) of vectors A and B can be computed in a pipelined operation, and the dot product of even-numbered samples (second, fourth, sixth, and subsequent samples) of vectors A and B can also be computed in a pipelined operation.

[0029] Similar multiplication and addition operations can continue for additional samples of vectors A and B. The pipeline can compute the partial sum of even-numbered samples in one pipeline stage and the partial sum of odd-numbered samples in a second pipeline stage.

[0030] During the first time period, the first samples of vectors A and B can be processed by vector ALU 100. During the second time period, the second samples of vectors A and B can be processed by vector ALU 100. During the fourth time period, feedback output 180 can represent the sum of the values ​​processed during the first time period, since feedback output 180 is valid after two clock cycles. Similarly, during the fifth time period, feedback output 180 can represent the sum of the values ​​processed during the second time period, since feedback output 180 is valid after two clock cycles. In this way, vector ALU 100 can implement a pipelined multiply-accumulate. In the alternating loop, vector ALU 100 can process odd-numbered samples of the two input vectors and even-numbered samples of the two input vectors.

[0031] When the last sample of vectors A and B is input to vector ALU 100, even-numbered samples and odd-numbered samples can be added together to compute the full dot product of vectors A and B. In the penultimate loop, feedback output 180 can be the partial sum of all odd-numbered samples. In the final loop (the last loop after the penultimate loop), feedback output 180 can be the partial sum of all even-numbered samples. In the final loop, first multiplexer 141 can select the output of delay unit 158 ​​and couple the output of delay unit 158 ​​to first multiplexer output 148. The output of delay unit 158 ​​can be the previous value of feedback output 180, specifically the partial sum of all odd-numbered samples. The output of delay unit 158 ​​can be a delayed signal. Accumulator 150 can add the partial sum of odd-numbered samples to the partial sum of even-numbered samples to generate a dot product.

[0032] Figure 1 An example with a feedback output that is valid after two clock cycles is shown.

[0033] The operation of the vector ALU 100 can be controlled by software operating in a microcontroller or microprocessor, or by dedicated hardware.

[0034] Figure 2 This illustrates one example of various examples of a pipelined multiply-accumulate system 200 used for calculating dot products. Figure 2 In the example illustrated, the dot product operation can be divided into eight parallel arithmetic logic units (ALUs) 210, 220, 230, 240, 250, 260, 270, and 280. ALUs 210, 220, 230, 240, 250, 260, 270, and 280 can be as shown in the reference. Figure 1The vector ALUs described and illustrated. ALUs 210, 220, 230, 240, 250, 260, 270, and 280 may include other arithmetic and computation units not specifically disclosed or described. A clock signal (not shown) may control the operation of system 200, and a clock cycle may be defined as the period of the clock signal.

[0035] Inputs to the corresponding ALU can be provided from the first vector input 201 and the second vector input 202. The inputs to the corresponding ALU can be ALU vector inputs. Each portion of the first vector input 201 and the second vector input 202 can be routed to one or more ALUs by the multiplexer 299. Each portion of the first vector input 201 and the second vector input 202 can include one or more samples.

[0036] The multiplexer 299 may include feedback inputs 205 and 206, which can be routed to one or more of ALU vector inputs 211, 212, 221, 222, 231, 232, 241, 242, 251, 252, 261, 262, 271, 272, 281, and 282.

[0037] ALU vector inputs 211 and 212 can be coupled to a first ALU 210. The first ALU 210 can perform one or more arithmetic operations, including but not limited to multiplication-accumulation operations, and can generate a first ALU output 213. The first ALU output 213 can be input to controller circuit 290. ALU vector inputs 221 and 222 can be coupled to a second ALU 220. The second ALU 220 can perform one or more arithmetic operations, including but not limited to multiplication-accumulation operations, and can generate a second ALU output 223. The second ALU output 223 can be input to controller circuit 290. ALU vector inputs 231 and 232 can be coupled to a third ALU 230. The third ALU 230 can perform one or more arithmetic operations and can generate a third ALU output 233. The third ALU output 233 can be input to controller circuit 290. ALU vector inputs 241 and 242 can be coupled to a fourth ALU 240. The fourth ALU 240 can perform one or more arithmetic operations and can generate a fourth ALU output 243. The fourth ALU output 243 is input to the controller circuit 290.

[0038] ALU vector inputs 251 and 252 can be coupled to a fifth ALU 250. The fifth ALU 250 performs one or more arithmetic operations and generates a fifth ALU output 253. The fifth ALU output 253 can be input to controller circuit 290. ALU vector inputs 261 and 262 can be coupled to a sixth ALU 260. The sixth ALU 260 performs one or more arithmetic operations and generates a sixth ALU output 263. The sixth ALU output 263 can be input to controller circuit 290. ALU vector inputs 271 and 272 can be coupled to a seventh ALU 270. The seventh ALU 270 performs one or more arithmetic operations and generates a seventh ALU output 273. The seventh ALU output 273 can be input to controller circuit 290. ALU vector inputs 281 and 282 can be coupled to an eighth ALU 280. The eighth ALU 280 can perform one or more arithmetic operations and can generate an eighth ALU output 283. The eighth ALU output 283 can be input to the controller circuit 290.

[0039] ALU vector inputs 211, 212, 221, 222, 231, 232, 241, 242, 251, 252, 261, 262, 271, 272, 281, and 282 can be driven simultaneously, allowing ALUs 210, 220, 230, 240, 250, 260, 270, and 280 to operate in parallel.

[0040] The controller circuit 290 may include arithmetic circuitry, a multiplexer, and other circuitry to perform arithmetic operations on one or more inputs and route one or more inputs to outputs 295 and 296. Outputs 295 and 296 may be connected via multiplexer 299 to any of the ALU vector inputs 211, 212, 221, 222, 231, 232, 241, 242, 251, 252, 261, 262, 271, 272, 281, and 282. Outputs 295 and 296 may be referred to as feedback signals.

[0041] In operation, system 200 can implement the dot product. The input vector can be divided into 8 segments and distributed in parallel by multiplexers 299 among the first ALU 210, second ALU 220, third ALU 230, fourth ALU 240, fifth ALU 250, sixth ALU 260, seventh ALU 270, and eighth ALU 280. In one example of various examples, system 200 can implement the dot product of two input vectors (vector A and vector B). Vector A can have 1024 samples, but this is not intended to be limiting. Vector B can have 1024 samples, but this is not intended to be limiting. Vector A and vector B can have an equal number of samples. The individual samples of vector A and vector B can be addressed by index values. As an example of various examples, the first sample of vector A can be addressed by index value 1, and the first sample of vector B can be addressed by index value 1. In other examples, the first sample of vector A can be addressed by index 0, and the first sample of vector B can also be addressed by index 0. Vector A can be divided into 8 segments of 128 samples, and the segments can be input to one ALU by multiplexer 299. Vector B can be divided into 8 segments of 128 samples, and the segments can be input to one ALU by multiplexer 299. The segments can be formed sequentially as samples arrive, but this is not intended to be restrictive, such that the first segment of vector A can contain the first sample of vector A, the second segment can contain the second sample of vector A, and so on sequentially through all samples and all segments of vector A. Similarly, the first segment of vector B can contain the first sample of vector B, the second segment can contain the second sample of vector B, and so on sequentially through all samples and all segments of vector B.

[0042] A partial sum can be defined as the dot product of a segment of a vector input. Figure 2 In the example shown, vectors A and B can be divided into 8 segments, and the sum of the eight segments can be calculated. Figure 2 In the example shown, the corresponding ALU can compute the partial sum of corresponding segments of vectors A and B.

[0043] In one example of various examples, a first segment of vector A may be coupled to ALU vector input 211. The first segment of vector A may consist of samples from index 1 to index 128. A first segment of vector B may be coupled to ALU vector input 212. The first segment of vector B may consist of samples from index 1 to index 128. A first ALU 210 may compute the dot product of the first segments of vector A and vector B, and may output this dot product at a first ALU output 213. The dot product at the first ALU output 213 may be a partial sum. The first ALU 210 may perform a pipelined multiply-accumulate and may process even and odd samples in separate streams.

[0044] A second segment of vector A can be coupled to ALU vector input 221. This second segment of vector A can consist of samples from index 129 to index 256. A second segment of vector B can be coupled to ALU vector input 222. This second segment of vector B can consist of samples from index 129 to index 256. A second ALU 220 can compute the dot product of the second segments of vector A and vector B, and output this dot product at second ALU output 223. The dot product at second ALU output 223 can be a partial sum. The second ALU 220 can perform a pipelined multiply-accumulate and can process even and odd samples in separate streams.

[0045] A third segment of vector A can be coupled to ALU vector input 231. This third segment of vector A can consist of samples from index 257 to index 384. A third segment of vector B can be coupled to ALU vector input 232. This third segment of vector B can consist of samples from index 257 to index 384. The third ALU 230 can compute the dot product of the third segments of vector A and vector B, and output this dot product at output 233. The dot product at the third ALU output 233 can be a partial sum. The third ALU 230 can perform a pipelined multiply-accumulate and can process even and odd samples in separate streams.

[0046] The fourth segment of vector A can be coupled to ALU vector input 241. The fourth segment of vector A can consist of samples from index value 385 to index value 512. The fourth segment of vector B can be coupled to ALU vector input 242. The fourth segment of vector B can consist of samples from index value 385 to index value 512. The fourth ALU 240 can compute the dot product of the fourth segments of vector A and the fourth segments of vector B, and can output the dot product at the fourth ALU output 243. The dot product at the fourth ALU output 243 can be a partial sum. The fourth ALU 240 can perform a pipelined multiply-accumulate and can process even and odd samples in separate streams.

[0047] The fifth segment of vector A can be coupled to ALU vector input 251. The fifth segment of vector A can consist of samples from index 513 to index 640. The fifth segment of vector B can be coupled to ALU vector input 252. The fifth segment of vector B can consist of samples from index 513 to index 640. The fifth ALU 250 can compute the dot product of the fifth segments of vector A and vector B, and can output the dot product at the fifth ALU output 253. The dot product at the fifth ALU output 253 can be a partial sum. The fifth ALU 250 can perform a pipelined multiply-accumulate and can process even and odd samples in separate streams.

[0048] The sixth segment of vector A can be coupled to ALU vector input 261. The sixth segment of vector A can consist of samples from index 641 to index 768. The sixth segment of vector B can be coupled to ALU vector input 262. The sixth segment of vector B can consist of samples from index 641 to index 768. The sixth ALU 260 can compute the dot product of the sixth segments of vector A and vector B, and can output the dot product at the sixth ALU output 263. The dot product at the sixth ALU output 263 can be a partial sum. The sixth ALU 260 can perform a pipelined multiply-accumulate and can process even and odd samples in separate streams.

[0049] The seventh segment of vector A can be coupled to ALU vector input 271. The seventh segment of vector A can consist of samples from index 769 to index 896. The seventh segment of vector B can be coupled to ALU vector input 272. The seventh segment of vector B can consist of samples from index 769 to index 896. The seventh ALU 270 can compute the dot product of the seventh segments of vector A and vector B, and can output the dot product at the seventh ALU output 273. The dot product at the seventh ALU output 273 can be a partial sum. The seventh ALU 270 can perform a pipelined multiply-accumulate and can process even and odd samples in separate streams.

[0050] The eighth segment of vector A can be coupled to ALU vector input 281. The eighth segment of vector A can consist of samples from index 897 to index 1024. The eighth segment of vector B can be coupled to ALU vector input 282. The eighth segment of vector B can consist of samples from index 897 to index 1024. The eighth ALU 280 can compute the dot product of the eighth segments of vector A and vector B, and can output the dot product at the eighth ALU output 283. The dot product at the eighth ALU output 283 can be a partial sum. The eighth ALU 280 can perform a pipelined multiply-accumulate and can process even and odd samples in separate streams.

[0051] exist Figure 2 In the example illustrated, each of ALUs 210, 220, 230, 240, 250, 260, 270, and 280 can compute the dot product of corresponding segments of vectors A and B. Controller circuit 290 can control the outputs of ALUs 210, 220, 230, 240, 250, 260, 270, and 280, and can drive outputs 295 and 296 to compute the dot product of the entire vector A and vector B.

[0052] Controller circuit 290 can receive input from the output of a corresponding ALU. The output of the corresponding ALU can be a partial sum of dot products, representing the dot product of one or more segments of the vector input. Controller circuit 290 can route signals back to one or more ALUs using outputs 295 and 296 via feedback inputs 205 and 206 of multiplexer 299, and can use one or more of these ALUs to perform addition to generate a final dot product output. In one example of various examples, first ALU output 213 can represent the dot product of the first 128 samples of vectors A and B, and second ALU output 223 can represent the dot product of the second 128 samples of vectors A and B. Controller circuit 290 can couple first ALU output 213 to output 295, and multiplexer 299 can route output 295 to ALU vector input 211 via feedback input 205. The controller circuit 290 can couple the second ALU output 223 to the output 296, and the multiplexer 299 can route the output 296 to the ALU vector input 212 via the feedback input 206. The first ALU 210 can sum the ALU vector inputs 211 and 212.

[0053] Similarly, the third ALU output 233 can represent the dot product of the third 128 samples of vectors A and B, and the fourth ALU output 243 can represent the dot product of the fourth 128 samples of vectors A and B. The controller circuit 290 can couple the third ALU output 233 to output 295, and the multiplexer 299 can route output 295 to ALU vector input 221 via feedback input 205. The controller circuit 290 can couple the fourth ALU output 243 to output 296, and the multiplexer 299 can route output 296 to ALU vector input 222 via feedback input 206. The second ALU 220 can sum ALU vector input 211 and ALU vector input 212.

[0054] Similarly, the fifth ALU output 253 can represent the dot product of the fifth 128 samples of vectors A and B, and the sixth ALU output 263 can represent the dot product of the sixth 128 samples of vectors A and B. Controller circuit 290 can couple the fifth ALU output 253 to output 295, and multiplexer 299 can route output 295 to ALU vector input 231 via feedback input 205. Controller circuit 290 can couple the sixth ALU output 263 to output 296, and multiplexer 299 can route output 296 to ALU vector input 232 via feedback input 206. The third ALU 230 can sum the ALU vector inputs 231 and 232.

[0055] Similarly, the seventh ALU output 273 can represent the dot product of the seventh 128 samples of vectors A and B, and the eighth ALU output 283 can represent the dot product of the eighth 128 samples of vectors A and B. Controller circuit 290 can couple the seventh ALU output 273 to output 295, and multiplexer 299 can route output 295 to ALU vector input 241 via feedback input 205. Controller circuit 290 can couple the eighth ALU output 283 to output 296, and multiplexer 299 can route output 296 to ALU vector input 242 via feedback input 206. The fourth ALU 240 can sum the ALU vector inputs 241 and 242.

[0056] The outputs of ALU 210, 220, 230, 240, 250, 260, 270 and 280 can be computed simultaneously and in parallel, so that ALU outputs 213, 223, 233, 243, 253, 263, 273 and 283 can be computed simultaneously.

[0057] The controller circuit 290 can feed back the output of the corresponding ALU to the input of one or more ALUs. In this way, the controller circuit 290 can control the multiplexer 299 and can compute the complete dot product of the entire vector A and vector B in one or more stages.

[0058] In one example of various examples, during the first phase, controller circuitry 290 may couple a first ALU output 213 to output 295, and multiplexer 299 may route output 295 to ALU vector input 211 via feedback input 206. Controller circuitry 290 may couple a second ALU output 223 to output 296, and multiplexer 299 may route output 296 to ALU vector input 212 via feedback input 206. First ALU 210 may compute the sum of ALU vector input 211 and ALU vector input 212, and this sum may be output to first ALU output 213. First ALU output 213 may represent the dot product of the first 256 samples of vectors A and B.

[0059] Controller circuit 290 couples a third ALU output 233 to output 295, and multiplexer 299 routes output 296 to ALU vector input 231 via feedback input 206. Controller circuit 290 couples a fourth ALU output 243 to output 296, and multiplexer 299 routes output 296 to ALU vector input 232. Third ALU 230 calculates the sum of ALU vector input 231 and ALU vector input 232, and this sum can be output to third ALU output 233. Third ALU output 233 can represent the dot product of the second 256 samples of vector A and vector B.

[0060] Controller circuit 290 couples the fifth ALU output 253 to output 295, and multiplexer 299 routes output 295 to ALU vector input 251 via feedback input 205. Controller circuit 290 couples the sixth ALU output 263 to output 296, and multiplexer 299 routes output 296 to ALU vector input 252 via feedback input 206. Fifth ALU 250 calculates the sum of ALU vector input 251 and ALU vector input 252, and this sum can be output to fifth ALU output 253. Fifth ALU output 253 can represent the dot product of the third 256 samples of vectors A and B.

[0061] Controller circuit 290 couples the seventh ALU output 273 to output 295, and multiplexer 299 routes output 295 to ALU vector input 271 via feedback input 205. Controller circuit 290 couples the eighth output 283 to output 296, and multiplexer 299 routes output 296 to ALU vector input 272 via feedback input 206. Seventh ALU 270 calculates the sum of ALU vector input 271 and ALU vector input 272, and this sum can be output to seventh ALU output 273. Seventh ALU output 273 can represent the dot product of the fourth 256 samples of vectors A and B.

[0062] Similarly, during the second phase following the first phase, the first ALU output 213, the third ALU output 233, the fifth ALU output 253, and the seventh ALU output 273 can be summed to calculate the dot product of 1024 samples of vectors A and B. Controller circuitry 290 can couple the first ALU output 213 to output 295, and multiplexer 299 can route output 295 to ALU vector input 211 via feedback input 205. Controller circuitry 290 can couple the third ALU output 233 to output 296, and multiplexer 299 can route output 296 to ALU vector input 212 via feedback input 206. First ALU 210 can calculate the sum of ALU vector input 211 and ALU vector input 212, and this sum can be output to first ALU output 213. First ALU output 213 can represent the dot product of the first 512 samples of vectors A and B. Controller circuit 290 couples the fifth ALU output 253 to output 295, and multiplexer 299 routes output 295 to ALU vector input 221 via feedback input 205. Controller circuit 290 couples the seventh ALU output 273 to output 296, and multiplexer 299 routes output 296 to ALU vector input 222 via feedback input 206. Second ALU 220 calculates the sum of ALU vector input 221 and ALU vector input 222, and this sum can be output to second ALU output 223. Second ALU output 223 can represent the dot product of the second 512 samples of vector A and vector B.

[0063] During the third phase following the second phase, controller circuit 290 may couple the first ALU output 213 to output 295, and multiplexer 299 may route output 295 to ALU vector input 211 via feedback input 205. Controller circuit 290 may couple the second ALU output 223 to output 296, and multiplexer 299 may route output 296 to ALU vector input 212 via feedback input 206. First ALU 210 may calculate the sum of ALU vector input 211 and ALU vector input 212, and this sum may be output to first ALU output 213. First ALU output 213 may represent the dot product of all samples of vector A and vector B.

[0064] The examples illustrated above are not intended to be limiting. In the examples illustrated above, three stages are required to compute the complete dot product, but this is not intended to be limiting. Each of the first, second, and third stages can be defined as one or more clock cycles. The dot product of vectors of sizes other than 1024 samples can be computed. The ALU can be utilized with sequences different from those described.

[0065] The computation time can be reduced by dividing the computation into eight simultaneous 128-element dot products.

[0066] Figure 2 The example shown includes 8 vector ALUs, but this is not intended to be limiting. Other examples may include more or fewer ALUs.

[0067] In another example, the first and second vectors can be divided into two segments. The corresponding segments of the first and second vectors can be coupled to corresponding ALUs. The first ALU can compute a first partial sum, and the second ALU can compute a second partial sum. The controller can control the computation of the sum of the first and second partial sums to compute the dot product of the first and second vectors.

[0068] Figure 3 A timing diagram 300 for a vector ALU is illustrated. In one example of various examples, timing diagram 300 may be represented as shown in the reference. Figure 2 The timing diagrams for one of the ALUs 210, 220, 230, 240, 250, 260, 270, and 280 are described and illustrated. A vector ALU computes the dot product of a first vector and a second vector. In this illustration, the first vector may also be referred to as vector A, and the second vector may also be referred to as vector B.

[0069] Trajectory 305 may represent the clock cycle count of the operation in the device used to calculate the dot product. Trajectory 310 may represent the first vector input of the device, with the number indicating the index of the first vector input. Figure 3 In the example illustrated, the index values ​​can start from zero, allowing the 1024 sample vector inputs to be indexed from 0 to 1023. In other examples, the index values ​​can start from one, or they can start with another number. Trajectory 320 can represent the device's second vector input, with the number representing the index of the second vector input. Trajectory 330 can represent the multiplier output, which is used to multiply the values ​​of trajectories 310 and 320. Trajectory 340 can represent the feedback output, which is used to represent the sum of the multiplier output and the feedback output. The feedback output can also be referred to as an accumulator.

[0070] At time 360, at clock cycle 0, trajectory 310 can be the first sample of the first vector. This sample can also be represented as A[0] or A0. Trajectory 320 can be the first sample of the second vector. This sample can also be represented as B[0] or B0. Trajectories 330 and 340 can be zero during clock cycle zero because the multiplier and accumulator are multi-cycle operations and do not produce output until a later clock cycle.

[0071] At time 361, at clock cycle 1, trajectory 310 can be the second sample of the first vector. This sample can also be represented as A[1] or A1. Trajectory 320 can be the second sample of the second vector. This sample can also be represented as B[1] or B1. Trajectory 330 can be the product of the first sample of the first vector and the first sample of the second vector. This product is... Figure 3 The value can be represented as "0*0". Trajectory 340 can be zero during clock cycle 1 because the accumulator is a multi-cycle operation and may not produce an output until a later clock cycle.

[0072] At time 362, at clock cycle 2, trajectory 310 can be the third sample of the first vector. This sample can also be represented as A[2] or A2. Trajectory 320 can be the third sample of the second vector. This sample can also be represented as B[2] or B2. Trajectory 330 can be the product of the second sample of the first vector and the second sample of the second vector. This product is at Figure 3 This can be represented as "1*1". Trajectory 340 can be zero during clock cycle two because the accumulator is a multi-cycle operation and may not produce an output until a later clock cycle.

[0073] At time 363, in clock cycle 3, trajectory 310 can be the fourth sample of the first vector. This sample can also be represented as A[3] or A3. Trajectory 320 can be the fourth sample of the second vector. This sample can also be represented as B[3] or B3. Trajectory 330 can be the product of the third sample of the first vector and the third sample of the second vector. This product can be valid in clock cycle 3 because the multiplier is a 2-loop operation. This product is valid in clock cycle 3. Figure 3 This can be represented as "2*2". Trajectory 340 can be the product of the first sample of the first vector and the first sample of the second vector, because for this sample, the accumulator feedback value is set to zero. This product is... Figure 3 The value can be represented as "0*" to make the graph more readable.

[0074] At time 364, at clock cycle 4, trajectory 310 can be the fifth sample of the first vector. This sample can also be represented as A[4] or A4. Trajectory 320 can be the fifth sample of the second vector. This sample can also be represented as B[4] or B4. Trajectory 330 can be the product of the fourth sample of the first vector and the fourth sample of the second vector. This product is at Figure 3 This can be represented as "3*3". Trajectory 340 can be the product of the second sample of the first vector and the second sample of the second vector, because for this sample, the accumulator feedback value is set to zero. This product is... Figure 3 The value can be represented as "1*" to make the graph more readable.

[0075] At time 365, at clock cycle 5, trajectory 310 can be the sixth sample of the first vector. This sample can also be represented as A[5] or A5. Trajectory 320 can be the sixth sample of the second vector. This sample can also be represented as B[5] or B5. Trajectory 330 can be the product of the fifth sample of the first vector and the fifth sample of the second vector. This product is at Figure 3 This can be represented as "4*4". Trajectory 340 can be the sum of the product of the third sample of the first vector and the third sample of the second vector, and the sum of the product of the first sample of the first vector and the first sample of the second vector, because the pipelined operation sets the accumulator feedback value to the output of the accumulator from clock cycle 4. This sum is in Figure 3 The graph can be represented as "0*+2*" to make it more readable.

[0076] At time 366, at clock cycle 6, trajectory 310 can be the seventh sample of the first vector. This sample can also be represented as A[6] or A6. Trajectory 320 can be the seventh sample of the second vector. This sample can also be represented as B[6] or B6. Trajectory 330 can be the product of the sixth sample of the first vector and the sixth sample of the second vector. This product is... Figure 3 This can be represented as "5*5". Trajectory 340 can be the sum of the product of the fourth sample of the first vector and the fourth sample of the second vector, and the sum of the products of the second sample of the first vector and the second sample of the second vector, because the pipelined operation sets the accumulator feedback value to the output of the accumulator from clock cycle 4. This sum is in Figure 3 The diagram can be represented as "1*+3*" to make it more readable.

[0077] At time 367, at clock cycle 7, trajectory 310 could be the eighth sample of the first vector. This sample could also be represented as A[7] or A7. Trajectory 320 could be the eighth sample of the second vector. This sample could also be represented as B[7] or B7. Trajectory 330 could be the product of the seventh sample of the first vector and the seventh sample of the second vector. This product is... Figure 3 This can be represented as "6*6". Trajectory 340 can be the sum of the product of the fifth sample of the first vector and the fifth sample of the second vector, the product of the third sample of the first vector and the third sample of the second vector, and the product of the first sample of the first vector and the first sample of the second vector, because the pipelined operation sets the accumulator feedback value to the output of the accumulator from clock cycle 6. This sum is in Figure 3 This can be represented as "0*+2*+4*" to make the graph more readable. In this way, trajectory 340 can be accumulated by summing the dot product of even-numbered samples of the first and second vectors.

[0078] At time 368, in clock cycle 8, trajectory 310 could be the ninth sample of the first vector. This sample could also be represented as A[8] or A8. Trajectory 320 could be the ninth sample of the second vector. This sample could also be represented as B[8] or B8. Trajectory 330 could be the product of the seventh sample of the first vector and the seventh sample of the second vector. This product could be valid in clock cycle 8 because of the fact that the multiplier is a 2-loop operation. This product in Figure 3 This can be represented as "7*7". Trajectory 340 can be the sum of the product of the sixth sample of the first vector and the sixth sample of the second vector, the product of the fourth sample of the first vector and the fourth sample of the second vector, and the product of the second sample of the first vector and the second sample of the second vector, because the pipelined operation sets the accumulator feedback value to the output of the accumulator from clock cycle 7. This sum is in Figure 3 This can be represented as "1*+3*+5*" to make the graph more readable. In this way, trajectory 340 can be accumulated by summing the dot product of even-numbered samples of the first and second vectors.

[0079] Once all input samples have been computed, the accumulator output for even-numbered samples can be added to the accumulator output for odd-numbered samples, producing a dot product of the input vectors, as shown in the reference. Figure 1 The description and examples are as follows.

[0080] In this way, dot products can be computed in a pipelined device that computes even-numbered samples in one stream and odd-numbered samples in a second stream.

[0081] Figure 3 The example is illustrated with specific delays in the multiplier and accumulator, but this is not intended to be limiting. Other multipliers and accumulators are available.

[0082] Figure 4 An example is given of a method for calculating the dot product.

[0083] At operation 410, the first vector and the second vector can be divided into one or more segments. As an example of various examples, the first vector may include 1024 samples and can be divided into 8 segments, each containing 128 samples, and the second vector may include 1024 samples and can be divided into 8 segments, each containing 128 samples.

[0084] At operation 420, corresponding segments of the first and second vectors can be input to one or more corresponding vector ALU devices. As one example among various examples, a first segment of the first vector and a first segment of the second vector can be coupled to a first vector ALU device.

[0085] At operation 430, the vector ALU device can compute the dot product of segments of the first and second vectors by pipelined processing of even-numbered and odd-numbered samples. The vector ALU device can compute the partial sum of even-numbered samples during the first clock cycle and the partial sum of odd-numbered samples during the second clock cycle.

[0086] At operation 440, the dot product of the complete first vector and the complete second vector can be calculated by summing the dot products of one or more pipelined multiply-accumulate devices.

[0087] Examples of this disclosure may include a device. The device may include an analog-to-digital (ALU). The ALU may include at least two vector inputs. The ALU may include a multiplier for taking inputs from the at least two vector inputs and for producing a multiplier output comprising the product of the at least two vector inputs. The ALU may include a first multiplexer for selecting one of a multiplier output, an external input, and a delayed signal as a first multiplexer output. The ALU may include a second multiplexer for selecting one of an external input and a feedback output as a second multiplexer output. The ALU may include an accumulator for adding the first multiplexer output to the second multiplexer output to generate a feedback output.

[0088] Combining any of the examples above, the two vector inputs can include multiple samples, which can be defined by clock signal inputs.

[0089] Combining any of the examples above, the second multiplexer can select an external input at the first clock cycle and a feedback output at the second clock cycle.

[0090] Combining any of the examples above, the first multiplexer can select a delayed signal at the last clock cycle.

[0091] In conjunction with any of the examples above, the accumulator may include a pipeline for processing even-numbered samples of at least two vector inputs during a first clock cycle and for processing odd-numbered samples of at least two vector inputs during a second clock cycle following the first clock cycle.

[0092] Examples of this disclosure may include a system. This system may include any of the devices described in the examples above. The system may include a controller for providing an output feedback signal coupled to a multiplexer. The controller may be used to receive input from at least one vector ALU and to generate a feedback signal based on the output of at least one vector ALU.

[0093] Combining any of the examples above, a vector ALU can be used as a pipeline to process samples with at least two vector inputs. The pipeline can be used to process even-numbered samples with at least two vector inputs during a first clock cycle, and to process odd-numbered samples with at least two vector inputs during a second clock cycle, which follows the first clock cycle.

[0094] Combining any of the examples above, a multiplexer can be used to divide two or more vector inputs into two or more segments and to couple the respective segments to at least one corresponding vector ALU.

[0095] Combining any of the examples above, the system may include two vector inputs and two vector ALUs, and the system is used to compute a first partial sum in the first vector ALU and a second partial sum in the second vector ALU. A controller may be used to couple the first and second partial sums to a feedback signal.

[0096] Combining any of the examples above, the controller can be used to couple the first part and the second part to the feedback signal, and the multiplexer can be used to couple the first part and the second part to one of the at least two vector ALUs.

[0097] Combining any of the examples above, at least one of the two vector ALUs can be used to compute the sum of a first partial sum and a second partial sum, which includes the dot product of the first vector input and the second vector input.

[0098] Examples of this disclosure may include a method. This method may include operations of any of the examples above. The method may include: dividing a first vector and a second vector into one or more segments; coupling the segment of the first vector and the second vector to one or more vector ALUs, the vector ALUs including at least a multiplier and an accumulator; computing the dot product of the segment of the first vector and the second vector by pipelined sampling of even-numbered and odd-numbered samples within the one or more vector ALUs; and computing the dot product of the first vector and the second vector by summing the outputs of the one or more vector ALUs.

[0099] Combining any of the examples above, summing the dot products of one or more vector ALUs can be used to feed the output of a vector ALU back to its input.

[0100] Combining any of the examples above, one or more segments can consist of an equal number of samples from the first vector and the second vector.

Claims

1. An apparatus, the apparatus comprising: A vector arithmetic logic unit (ALU), the vector arithmetic logic unit (ALU) comprising: At least two vector inputs; A multiplier, the multiplier being used to obtain input from the at least two vector inputs and to produce a multiplier output comprising the product of the at least two vector inputs; A first multiplexer is configured to select one of the multiplier output, the external input, and the delayed signal as the output of the first multiplexer. A second multiplexer, configured to select one of the external input and the feedback output as its output; and An accumulator is used to add the output of the first multiplexer to the output of the second multiplexer to generate the feedback output.

2. The device according to claim 1, wherein the at least two vector inputs comprise a plurality of samples, the samples being defined by a clock signal input.

3. The device according to claim 2, wherein the second multiplexer is configured to select the external input at a first clock cycle and select the feedback output at a second clock cycle.

4. The device according to any one of claims 2 to 3, wherein the first multiplexer is used to select the delayed signal at the last clock cycle.

5. The device according to any one of claims 2 to 4, wherein the accumulator includes a pipeline for processing even-numbered samples of the at least two vector inputs during a first clock cycle and for processing odd-numbered samples of the at least two vector inputs during a second clock cycle, the second clock cycle following the first clock cycle.

6. A system comprising: A multiplexer for receiving two or more vector inputs; At least one vector arithmetic logic unit (ALU) is provided for receiving at least one output of the multiplexer, the at least one vector ALU comprising: At least two ALU vector inputs, which are provided by the multiplexer; A multiplier, the multiplier being used to obtain input from the at least two ALU vector inputs and to produce a multiplier output comprising the product of the at least two ALU vector inputs; An accumulator, which takes input from the multiplier output and the feedback output, wherein the accumulator adds the multiplier output and the feedback output; and A controller having an output feedback signal coupled to the multiplexer, the controller being configured to receive input from the at least one vector ALU and to generate the feedback signal based on the output of the at least one vector ALU.

7. The system of claim 6, wherein the two or more vector inputs comprise a plurality of samples, the samples being defined by a clock signal input.

8. The system of claim 7, wherein the vector ALU is used to process samples of the at least two vector inputs as a pipeline, the pipeline being used to process even-numbered samples of the at least two vector inputs during a first clock cycle and to process odd-numbered samples of the at least two vector inputs during a second clock cycle, the second clock cycle following the first clock cycle.

9. The system according to any one of claims 7 to 8, wherein the multiplexer is used to divide the two or more vector inputs into two or more segments and to couple the respective segments to at least one corresponding vector ALU.

10. The system according to any one of claims 7 to 9, the system comprising two vector inputs and two vector ALUs, the system being configured to compute a first portion sum in the first vector ALU and a second portion sum in the second vector ALU, and the controller being configured to couple the first portion sum and the second portion sum to the feedback signal.

11. The system of claim 10, wherein the controller is configured to couple the first portion and the second portion to the feedback signal, and the multiplexer is configured to couple the first portion and the second portion to one of the at least two vector ALUs.

12. The system of claim 11, wherein one of the at least two vector ALUs is used to compute the sum of the first partial sum and the second partial sum, the sum comprising the dot product of the first vector input and the second vector input.

13. A method, the method comprising: Divide the first vector and the second vector into one or more segments; Couple the segments of the first vector and the second vector to one or more vector ALUs, the vector ALUs including at least a multiplier and an accumulator; The dot product of the segments of the first vector and the second vector is calculated by pipelined sampling of even-numbered and odd-numbered samples within the one or more vector ALUs. as well as The dot product of the first vector and the second vector is calculated by summing the outputs of the one or more vector ALUs.

14. The method of claim 13, wherein adding the dot products of the one or more vector ALUs includes the controller feeding back the output of the vector ALU to the input of the vector ALU.

15. The method according to any one of claims 13 to 14, wherein the one or more segments consist of an equal number of samples of the first vector and the second vector.

Images