ACCELERATED MATHEMATICAL ENGINE

Abstract

Several forms of disclosure relate to an accelerated mathematical engine; in certain forms, the accelerated mathematical engine is applied to image processing to accelerate the convolution of an image by using a two-dimensional array processor comprising sub-circuits including an ALU, an output register, and a shadow register; this architecture supports a two-dimensional timed architecture in which image data and weights are multiplied in a synchronized manner to allow a large number of mathematical operations to be carried out in parallel.

Description

ACCELERATED MATHEMATICAL ENGINE CROSS-REFERENCE WITH RELATED APPLICATIONS This application claims priority for U.S. provisional patent application no. 62 / 536,399 (20150-2154P (P0822-1PUS)), filed July 24, 2017, entitled Accelerated Mathematical Engine, which lists Peter Joseph Bannon, Kevin Altair Hurd, and Emil Talpes as inventors. The aforementioned patent document is incorporated herein in its entirety by reference. This application also claims priority for U.S. patent application No. 15 / 710,433 (20150-2154 (N0822-1NUS)), filed on September 20, 2017, entitled "Accelerated Mathematical Engine," which lists Peter Joseph Bannon, Kevin Altair Hurd, and Emil Talpes as inventors. Each of the aforementioned patent documents is incorporated herein in its entirety by reference. TECHNICAL FIELD This disclosure relates to an accelerated mathematical engine for operating on large amounts of data and, more particularly, to an accelerated mathematical engine for performing complex convolution operations based on matrix multiplication operations. BACKGROUND OF THE INVENTION An expert in the field will recognize the ever-increasing demands for speed and performance on general-purpose processors and systems used to implement urgent and complex mathematical operations. Because these general-purpose systems are used to process large amounts of data and perform complex mathematical operations, computing resources and calculation speed are limited by the capabilities of existing general-purpose hardware designs that perform those calculations. For example, general-purpose computer devices and processors that perform matrix operations may be unable to perform these operations in a timely manner under certain circumstances.Many conventional multipliers that perform digital signal processing operations rely on a series of software and hardware matrix manipulation steps (address generation, transpositions, bitwise addition and subtraction) and can represent a bottleneck within a time-sensitive system. Often, these manipulation steps require the use of arithmetic functions from a processor to generate immediate results at the expense of wasted computing time due to the additional steps of storing and retrieving intermediate results from various locations to complete an operation. Figure 1 shows an example of a conventional multiplier system. The multiplier system 100 is a scalar machine comprising a computing unit 102, registers 104, cache 106, and memory 108. In operation, the computing unit 102 uses registers 104 and cache 106 to retrieve data stored in memory 108. Conventionally, the computing unit 102 is a microprocessor, such as a CPU or GPU, capable of performing various computing procedures, including matrix multiplication or processing input matrices to obtain a resulting matrix, for example, by converting multiplications into additions and projecting the result into some internal register. For example, a dot product representing an output pixel of an image is typically generated by dot-multiplying individual array elements of two arrays to obtain partial results, which are then aggregated to obtain the final dot product. Multiplication of individual array elements, i.e., scalar multiplication, is typically performed on individual data elements by breaking down dot multiplication into a series of individual sub-operations. As a result, the partial products must be stored and retrieved from one or more registers 104, cache 106, and memory 108 to complete a single arithmetic operation. Computationally demanding applications, such as convolution, often require a software function to be integrated into the compute unit 102 and used to convert convolution operations into alternating matrix multiplication operations. This is achieved by rearranging and reformatting the data into two matrices that can be multiplied by raw matrix. However, there is no mechanism for efficiently sharing or reusing data on the scalar machine 100, so the data needed to perform each scalar operation must be re-stored and re-retrieved from the registers multiple times. The complexity and administrative overhead of these operations become significantly greater as the amount of image data subject to convolution operations increases. The inability to reuse much of the data in the scalar machine 100 coupled with the added and inefficient steps of storing and retrieving intermediate results from registers 104, cache 106 and memory 108 to complete an arithmetic operation, are just some of the existing drawbacks, such as the multiplier system 100. Consequently, what is needed are high-performance computer systems and methods that can perform matrix mathematical operations quickly and efficiently. iviA / a / zuzo / ui U4U / BRIEF DESCRIPTION OF THE DRAWINGS Reference will be made to the embodiments of the invention, examples of which are illustrated in the accompanying figures. These figures are intended to be illustrative but not limiting. Although the invention is generally described in the context of these embodiments, it should be understood that the scope of the invention is not intended to be limited to these particular embodiments. Elements in the figures may not be to scale. Figure 1 shows an example of a conventional multiplier system. Figure 2 illustrates an exemplary array processor architecture for performing arithmetic operations according to various modalities of this disclosure. Figure 3 illustrates details of an exemplary configuration of the array processor architecture illustrated in Figure 2. Figure 4 illustrates an exemplary multiplication and addition circuit implementation of the logic circuit shown in Figure 3. Figure 5 illustrates an exemplary convolution operation according to several modalities of the present disclosure. Figures 6 through 8 illustrate details of an exemplary convolution operation according to various modalities of this disclosure. Figure 9 illustrates an exemplary deconvolution operation according to several modalities of the present disclosure. Figure 10 illustrates a process for performing arithmetic operations to make convolutional neural networks faster, according to various modalities of this disclosure. DETAILED DESCRIPTION OF THE INVENTION In the following description, for explanatory purposes, specific details are set out to provide an understanding of the invention. However, it will be evident to a person skilled in the art that the invention can be practiced without these details. Furthermore, a person skilled in the art will recognize that the embodiments of the present invention, described below, can be implemented in various ways, such as a process, apparatus, system, device, or method in a tangible medium that can be read by a computer. The components or modules shown in diagrams are illustrative of exemplary embodiments of the invention and are intended to avoid obscuring the invention. It should also be understood that throughout this explanation, these components may be described as separate functional units, which may comprise sub-units, but those skilled in the art will understand that the specific components may be described as such. MAS.ZUZÓU 1 U4UZ will recognize that various components or portions thereof can be divided into separate components or can be integrated together, including within a single system or component. It should be noted that the functions or operations explained here can be implemented as components. Components can be implemented in software, hardware, or a combination of both. Various components are formed through the interconnection of various sub-components. Sub-components that are logically different in operation from what is shown here can be selected, where these logically different sub-components, which can be combined in the aggregate with other sub-components, provide similar or identical functionality at the aggregate component level described here (e.g., active high signals can be active low, AND gates replaced with inverted-input NOR gates, etc.). Furthermore, the connections between the components or systems within the figures are not intended to be limited to direct connections. On the contrary, data between these components can be modified, reformatted, or otherwise changed through intermediary components. Fewer or additional connections can also be used. It should also be noted that the terms coupled, connected, or coupled in communication should be understood to include direct connections, indirect connections through one or more intermediary devices, and wireless connections. References in the specification to an embodiment, preferred embodiment, or embodiments mean that a particular feature, structure, or function described in connection with the embodiment is included in at least one embodiment of the invention and may be present in more than one embodiment. Furthermore, the appearance of the aforementioned phrases in various places in the specification does not necessarily mean they all refer to the same embodiment or embodiments. The use of certain terms in various places in the specification is for illustrative purposes and should not be interpreted as limiting. A service, function, or resource is not limited to a single service, function, or resource; the use of these terms may refer to a grouping of services, functions, or resources, which may be distributed or aggregated. The terms "includes," "which includes," "comprises," and "comprising" should be understood as open-ended terms, and any lists that follow are examples and are not intended to be limited to the enumerated items. They may include subsets or supersets of the items along with additional items. Any headings used herein are for organizational purposes only and should not be used to limit the scope of the description or any claims. Each document referenced in this patent document is incorporated herein in its entirety by reference. Furthermore, an expert in the technique will recognize that: (1) certain steps can be performed optionally; (2) the steps may not be limited to the specific order set out here; (3) certain steps can be carried out in different orders; and (4) certain steps can be performed simultaneously. Although the embodiments are explained here primarily in the context of convolutions, a person skilled in the art will appreciate that a deconvolution and other matrix operations can also be structured as a matrix-matrix multiplication operation, and thus the principles of the present invention are equally applicable to deconvolutions. Furthermore, other types of mathematical operations can be implemented according to various embodiments of this disclosure. Figure 2 illustrates an exemplary array processor architecture for performing arithmetic operations according to various embodiments of this disclosure. The system 200 comprises a logic circuit 232, 234, cache / buffer 224, data formatter 210, weight formatter 212, data input array 206, weight input array 208, array processor 240, output array 226, post-processing units 228, and control logic 250. The array processor 240 comprises a plurality of subcircuits 242 containing Arithmetic Logic Units (ALUs), registers, and, in some embodiments, encoders (such as cabinet encoders). The logic circuit 232 may be a circuit representing N input operators and data registers. Logic circuit 234 can be a circuit system that inputs M weight operands into array processor 240.Logic circuit 232 can be a circuit system that inputs image data operands into array processor 240. The weight input array 208 and the data input array 206 can be stored in various types of memory, including SRAM devices. A person skilled in the art will recognize that various types of operands can be input into array processor 240. According to certain modalities, during operation, the System 200 accelerates convolution operations by reducing redundant operations within the systems and by implementing specific hardware logic to perform certain mathematical operations on the entire data and weight set. This acceleration is a direct result of methods (and corresponding hardware components) that retrieve and input image data and weights into the Matrix Processor 240, as well as performing large-scale timing mathematical operations within the Matrix Processor 240.In the various configurations, formatters 210 and 212, which in the example in Figure 2 are implemented as inline formatters, are used. In some configurations, formatters 210 and 212 are discrete components, while in others, they are integrated together with one or more other components. Each is implemented in hardware and converts a matrix into a vector of operands to be operated on within the matrix processor 240. In other configurations, formatters 210 and 212 are implemented in software, although this typically results in a loss of speed. Data formatter 210 converts two-dimensional or three-dimensional data (e.g., a 3 x 3 x 3 cube) comprising an input data matrix 206 into a single vector or sequence that can be represented by a row or column, thus linearizing or vectorizing the input data matrix 206.In detail, the formatter 210 receives the data input matrix 206 and prepares the input data to be processed by the matrix processor 240. In the modes, this is achieved by mapping parameters of the data input matrix 206 into a suitable format according to the hardware requirements of the matrix processor 240 so that the matrix processor 240 can efficiently perform a matrix multiplication as part of a convolution calculation when generating output pixels. As an example, assuming a 240-matrix processor comprises 96 rows and 96 columns, data mapped in a 96 x 96 format would cause a 240-matrix processor to be used at its full computing capacity and thus provide preferred efficiency. In that case, the 210 formatter will produce output that is 96 columns wide. Similarly, the 212 formatter will produce output that is 96 rows wide based on the 208-weight input matrix. In these modes, formatter 210 uses a number of multiplexers or switches to retrieve some or all of the input data arrays 206 and selects different elements from them to produce data that is then aligned according to the columns of array processor 240. In these modes, the selection ensures that the appropriate data from the input data array 206 is passed to each of the columns in defined time cycles. In these modes, if the weights are static, they can be preformatted offline, stored in memory, retrieved only once, and fed directly to array processor 240 in a modified, vectorized format without using formatter 212. In other modes, the weights can be dynamically adjusted and fed to array processor 240 according to various formatting and retrieval operations.In these modes, the 240-matrix processor allows column and row entries to vary in size. That is, the 240-matrix processor is designed to perform NxM calculations of arbitrary size. In other modes, if the number of columns of the array processor 240 is limited (for example, to N columns) so that the number of columns in the data input array 206 (for example, X) is greater than the number of columns of the array processor 240 (i.e., X>N), then the control logic 250 can divide the data input array 206 into multiple sub-arrays, each sub-array computed by an array processor 240. In such cases, each array processor 240 can run in a different chain. For example, if the data input array 206 consists of 192 x 96 data points, and the array processor has 96 columns and 96 rows (i.e., 96x96 calculations can occur in one timer cycle), control logic 250 can split the data input array 206 into two sub-arrays (such as the left half of the data input array 206 and the right half of the data input array 206).Each sub-array will consist of 96x96 data points. Each array processor 240 in a separate chain can calculate the output channels for the sub-array sent to it, with the results placed in the final output array 260, which must be large enough to hold the values ​​of all the channels (i.e., 192 values). More generally, the data input array 206 can be divided into any number of sub-arrays and sent to different array processors 240, each running in a separate chain. As with the output array 226, the data input array 206, the data formatter 210, the cache / buffer 224, the logic circuitry 232, and the post-processing unit 228 must all be equally capable of accommodating larger datasets. In alternative modes, a CNN can be computed across multiple array processors 240 by having the control logic 250 divide the computations along the inner product. The segments of the inner product are computed, each on a different array processor 240, and then the input products are aggregated together to compute the output vector, which is then stored in the output array 260. Unlike common software implementations of formatting functions that are performed by a CPU or GPU to convert a convolution operation into a matrix multiplication by rearranging the data into an alternate format suitable for fast matrix multiplication, several hardware implementations in this disclosure reformat the data on the fly and make it available for execution—for example, 96 pieces of data per cycle. This effectively allows for the parallel processing of a large number of matrix elements, thus efficiently mapping data for a matrix operation. In these implementations, for every 2^N input data retrieved, 2^N2 data can be computed in a single timed cycle.This architecture results in a significant improvement in processing speeds by significantly reducing the number of read or retrieval operations used in a conventional processor architecture, as well as providing a parallel, efficient, and synchronized process when performing a large number of mathematical operations on a plurality of data inputs. In the modes, to increase the efficiency of a 240 matrix processor that can have any arbitrary number of columns and rows, the 212, 214 formatter can IVIA / a / ZUZÓ / UI U4Ur reformats different forms of input array data into columns and rows suitable for array processor 240. In the modes, formatting is performed dynamically to allow processing of arrays that have different input sizes. In the modes, the reformatted arrays comprising input channels are fed into cache / buffer 224. The cache / buffer 224 can retrieve data from the data input array 206 only 1 / k times, since multiple pieces of data can be reused, where k is the convolution core width. For example, for any given cycle, once a row is retrieved, certain columns will have access to all the data in that row. In some configurations, cache / buffer 224 can be a local buffer that stores a local copy of data that can be reused by a convolution without having to re-access and read data from SRAM. Once the array processor 240 has completed a calculation, a set of results can be swapped, for example, from the accumulators in the bottom row of the array processor 240, to output to flip-flops (not shown) that effectively form a shift register receiving a dot product. In some modes, extracting or swapping results to the output array 226, for example, one per timed cycle, from a row corresponding to an output channel, can be accomplished by a state machine (not shown). The state machine can perform additional operations on the output channel, for example, before sending data to the SRAM and / or the post-processing unit 228. The internal operation of the array processor 240 will be described in more detail later. In the modes, the 240 array processor comprises shadow resistors that allow parallel processing by storing a copy of the results that are passed through the 240 array processor to the 226 output array. In the modes, moving an operation result from an output register to the shadow register involves loading the next set of values ​​into the ALUs. Once accumulation is complete, a convolution can begin, and accumulation can start before all the data from a previous convolution has been output to array 226. As a result, in each timer cycle, the data in array processor 240 can be moved down one row, so that, for each cycle, the last row can be output to array 226. In fact, this mode of operation ensures that a new calculation can be performed in each consecutive cycle without any interruption and independently of any additional processing operations, such as storing data in SRAM, etc. The post-processing unit 228 can comprise or interact with a number of devices (not shown), such as a hardware-accelerated pooling unit, a DRAM that may be part of a direct memory access (DMA) system that retrieves data from memory and stores data (e.g., weights and results) in SRAM, etc. The devices may be partially or fully controlled by the control logic 250, which may also handle the formatters 210, 212, and other components within the system 200. Although not shown in Figure 2, there are auxiliary devices that perform management functions, such as a sequencer that generates addresses to read data, writes the results, and keeps track of where system 200 is in the convolution to calculate where to go and how to execute the data to be used in a later step of the convolution. In certain configurations, the 208-digit weight input array is physically split and feeds the weights from two different sides of the 240-digit array processor, so that the two-dimensional array is divided into two regions (for example, a far-left side and a far-right side), each of which receives a portion of the data in the 208-digit weight input array. This implementation reduces data latency by taking advantage of the fact that the weights are known. In some configurations, to reduce peak power consumption, the timing of operations can be chosen so that the weight and data multiplications are spread out over a certain number of cycles.This efficient timing of operations results in a reduction of energy-consuming steps, including a decrease in the number of operations read and carried out by the array processor and improved efficiency of data movement within the array (e.g., between sub-circuits). In these modes, a state machine (not shown) can be used that is configured to identify redundant data. The identified redundant data can then be reused in the columns, so the data does not need to be retrieved again. The state machine can be configured to determine how and where change data should be executed, for example, based on inputs related to image size, filter size, advance, number of channels, and similar parameters. In these modes, a single encoder is shared across a number of elements in the 240-matrix processor's multiplication architecture. The encoder can be any known encoder in the art and can be used to multiply two numbers and encode one of the two numbers, for example, from an 8-bit value to 12 bits, or any other value that makes multiplication operations in the multiplier logic easier and therefore faster. In these modes, the encoder can be applied in parallel across the entire row to share the same encoded weight and alternate it across all columns. By loading an operand into all columns, a multiplication can be performed in a single timer cycle across an entire row.The cost of driving re-coding to share the same data (e.g., weights) across N computing elements is therefore paid only once for each column (or row). In comparison, in existing computing architectures, each scalar requires a separate encoder for each multiplication operation. Figure 3 illustrates details of an exemplary configuration of the array processor architecture shown in Figure 2. In the available modes, the array processor 300 can receive a predetermined vector length on each axis. As illustrated in Figure 3, the array processor 300 can comprise an array of 6 x 6 panels arranged in a matrix format. Each panel 302 can comprise an array 320, which, in turn, comprises subcircuit circuits 350. As explained in detail later, with reference to Figure 4, each subcircuit circuit 350 can be a cell capable of performing arithmetic operations. In the available modes, the subcircuit circuit 350 performs simultaneous multiplication, accumulation, and shift operations. In some modes, arithmetic operations are parallelized by utilizing multiple rows and columns of the 300-matrix processor to generate an N x N panel output. For example, a given row size of 96 and a corresponding column size of 96 facilitates an output of 2 * 9216 mathematical calculations. In other modes, the number of rows and columns can be different. That is, there can be N rows and M columns, generating an N x M panel output. For example, with a row size of 96 and a corresponding column size of 192, an output of 2 * 18,432 calculations is generated in a single timer cycle. Figure 4 illustrates an exemplary multiplication-addition circuit implementation of the subcircuit shown in Figure 3. As illustrated in Figure 4, the multiplication-addition circuit 400 comprises a multiplier 430, an adder 432, logic 434, 436, 438, an accumulator 424, a shadow register 428, and an output register 440. In some configurations, the accumulator 424 can be implemented as an accumulation register. In the modes, the accumulator 424 may comprise an ALU set comprising registers and shadow register 428 that may be configured to receive the outputs of the ALUs. During operation, multiplier 430 receives and multiplies weights 402 and data 404 to generate products. Each product can be provided to adder 432, which, upon receiving the product from multiplier 430, adds the product to the current value of accumulator 424. In the modes, accumulator 424 generates an accumulated value that is stored, for example, in output register 440. The accumulated value is the result of a convolution and, as mentioned with reference to Figure 2, may correspond to the dot product of two formatted matrices. iviA / a / zuzo / ui U4U / In these modes, a copy of the result in output register 440 can be provided to the shadow register 428, which can then output the result to register 450, allowing access to accumulator 424 to begin new calculations. In these modes, the multiplication-addition circuit 400 in Figure 4 can perform a multiplication, an addition, and a change operation simultaneously, that is, within a single cycle, thus doubling the total number of operations that occur in each cycle. In these modes, the ClearAcc 408 signal clears the contents of accumulator 424, for example, when multiplier 430 performs a multiplication operation, so that the accumulation operations can begin again. In these modes, the ResultEnable 412 signal is activated in response to a determination that data 404 is valid. It is understood that accumulator 424 can accumulate and store data, accumulate and clear data, or only clear data. In these modes, the results are moved from an output register 440 to a shadow register 428 in a single timer cycle, i.e., without the need to execute and save intermediate operations. Figure 5 illustrates an exemplary convolution operation according to several embodiments of the present disclosure. The convolution 500 comprises input image IC channels 502, weights 532, dot product 514, output OC channels, and an accumulator 540. In some modalities, the convolution operation 500 applies individual filters (i.e., weights) 532 to the input image 502, for example, to detect small features within the input image 502. By analyzing a sequence of different features in a different order, macro-features can then be identified in the input image 502. In other modalities, the input 502 is non-image data. For example, the input 502 can be non-image sensor data, such as ultrasonic, radar, LIDAR, or other sensor data. The input 502 can also be general mathematical calculations or any other type of data known to someone skilled in the art. Convolution 500 can use a different set of weights 532 for each IC input channel, since each IC input channel can contain a different set of information; each weight matrix 532 can be designed to identify a different feature. In the modalities, convolution 500 multiplies a rectangular input matrix 504 by a rectangular weight matrix 532 to obtain partial dot products. The partial dot products can then be summed by the adder 546 to generate a cumulative dot product 514 (i.e., an integer) that represents an output pixel 514 in the output image. In the modes, each pixel in the OC output channel is generated by the multiplier 542 and the adder 544. In the modes, the value of the partial dot products iviA / a / zuzo / ui U4U / corresponds to the application of the weight matrix 532 in its entirety to the area 504 of the input image 502. In other words, each weight 532 is multiplied dot by multiplier 542 with area 504 to produce a partial dot product; then, the partial dot products are accumulated in accumulator 540 to generate an accumulated output that represents the convolution. One or more IC input channels can be used, for example, one for each color (e.g., RGB). For instance, each convolution might use 532 weights representing three different matrices, one for each color. Each OC output channel 512 can be generated using a different filter or 532 weight representing a different feature in the 502 input data. The number of output channels can depend on the number of features. The number of convolutions equals the number of OC output channels multiplied by the number of IC input channels, and each convolution can have N convolutions for each IC input channel. A person skilled in the technique will recognize that the number and type of input channels can vary and may include color and / or erase inputs. As illustrated in Figure 5, the input matrix 504 is a Kx x Ky (i.e., 3x3) matrix that can be combined with a 3 x 3 weight matrix 532 through 3 input channels, i.e., 3 x 3 x IC, so that the depths match and produce a single element, dot product 514, in the output plane. Each dot product 514 in the output channel 512 is the result of a dot multiplication. Figures 6 through 8 illustrate details of an exemplary convolution operation according to various embodiments of this disclosure. The convolution 600 comprises an input data array 602, a weight data array 604, an array 606, and a dot product 630. In the embodiments, the array 606 is an array processor architecture as shown in Figures 2 and 3. The input data matrix 602 in Figure 6 comprises a column 610, which, depending on the modality, can be obtained by linearizing an input matrix, such as the rectangular input matrix 504 shown in Figure 5, to obtain a vectorized form of the input matrix. Similarly, the weight data matrix 604 comprises a row 620, which can be a vectorized form of a weight matrix, such as the rectangular weight matrix 532 in Figure 5. As an example, a 3 x 3 input matrix and 3 input channels can be reformatted into a vector comprising 3 x 3 x 3 = 27 elements, from which a 27-element column 610 can be produced for use in the input data matrix 602. Conversely, a 3 x 3 weight matrix for the same 3 input channels can be used to generate a 27-element row 620 for use in the weight data matrix 604.An expert in the technique will recognize that the sizes of the input arrays and the number of input channels can vary in different applications. iviA / a / zuzo / ui U4U / In the various modes, the input channels and input weights, represented as rectangles in Figure 5, are reformatted—for example, using a formatter explained with reference to Figure 2—into vector formats (e.g., vectors with 96 elements). These vectors are then fed to a matrix multiplier / processor (indicated as element 240 in Figure 2), enabling a parallel 96 x 96 dot product operation. Specifically, the input data 504 and input weights 532, shown as rectangles in Figure 5 for each input channel, are reformatted into vector formats. In the modes, the resulting vector formats, illustrated in Figure 6, as input data 602 and input weights 604 (e.g., each comprising 96 elements), are provided to a matrix processor or matrix multiplier 240, which performs a 96 x 96 dot product operation in parallel. In the modes, in the output channel calculation, the same output pixels are produced using the same set of input data but a different set of weights (i.e., filters), so that by reading the input data once, the output channels can be generated immediately. As stated above, it is understood that the number of input and output channels can be chosen arbitrarily. Furthermore, it is understood that the input data matrix 602, the weight data matrix 604, and the array 606 can have different numbers of columns and rows than those illustrated in Figure 6. In particular, the shapes of the input data matrix 602 and the weight data matrix 604 can be formatted to receive the columns and rows of any arbitrary configuration of the array 606. Moreover, in circumstances where the weight data matrix 604 is known, then row 620 can be generated and stored in a vectorized format without the use of a formatter. In the modes, the dot product 630 in Figure 6 is generated by dot-multiplying a vector corresponding to column 610 with a vector corresponding to row 620. In the modes, as shown in Figure 7, the next dot product 632 can be obtained by dot-multiplying a vector corresponding to column 612 with the vector corresponding to row 620. As the expert in the art will recognize, once all the dot products in the first row of array 606 are filled, the dot product of the second row of array 606 can be calculated by dot-multiplying the elements in the first column 610 of the input data matrix 602 with the second row of the weight data matrix 604, and so on. It is important to mention that Figures 6 through 8 are for illustrative purposes only, and that the dot multiplications mentioned above can be performed simultaneously to generate a one-time matrix-matrix multiplication operation. iviA / a / zuzo / ui U4U / Figure 9 illustrates an exemplary deconvolution operation according to several modalities of this disclosure. The deconvolution system 900 comprises input image channels 902 (IC), weights 922, dot products 904 and 906, and output channels OC. A person skilled in the art will recognize that the deconvolution operation 900 is, in effect, a mathematical transposition (approximately the inverse) of the convolution operation, for example, the convolution shown in Figure 5. A person skilled in the art will further recognize that a neural network can be used to learn the deconvolution operation 900 by applying procedures similar to those used by ordinary convolutional neural networks. For the sake of brevity, the description or functions of components similar to those in Figure 5 are not repeated here. In the modalities, the deconvolution operation 900 in Figure 9 reconstructs the matrices 912 by deconstructing the dot product 904, 906 using the weights 922. As with a convolution operation, deconvolution 900 can use a different set of weights 922 for each IC input channel. In the modalities, deconvolution 900 can be advantageously applied to an image to perform image deconvolution, for example, to improve robustness against artifacts. Other applications may include image data analysis and restoration, etc. Figure 10 illustrates a process for performing arithmetic operations to accelerate convolutional neural networks according to various modalities of this disclosure. The process 1000 for performing arithmetic operations begins at step 1002 when a first set of operands, which may be representative of a row in a data array, is received from a first logic circuit. This first set of operands can be vectorized so that the operands are aligned with inputs in an array processor. In certain configurations, the size of the vectorized operands is directly related to the number of inputs in an array processor along the axis. In step 1004, a second set of operands, which may represent a column in a weight matrix, is received from a second logic circuit. This second set of operands can be vectorized so that the operands are aligned within corresponding inputs in the matrix processor. In certain configurations, the size of the vectorized operands is directly related to the number of inputs in the matrix process along a different axis. In step 1006, the first set of operands is dot-multiplied with the second set of operands to obtain one or more dot products. In certain configurations, this set-through operation across the sets of operands is performed in a single timer cycle. In step 1008, dot products can be used to convolve an image with a filter to produce a convolution result. In step 1010, the convolution result is further processed to improve the image output. This additional processing can occur through the use of a nonlinear function, a normalization operation, or a grouping operation. A person skilled in the art will recognize that no computer system or programming language is critical to implementing the present invention. A person skilled in the art will also recognize that a number of the elements described above can be physically and / or functionally separated into sub-modules or combined. It should be noted that the elements of the following claims may be arranged differently, including having multiple dependencies, configurations, and combinations. For example, in the modalities, the subject matter of several claims may be combined with other claims. Those skilled in the art will appreciate that the preceding examples and embodiments are illustrative and do not limit the scope of the present invention. It is intended that all permutations, improvements, equivalents, combinations, and enhancements thereof that are apparent to those skilled in the art after reading the specification and studying the drawings are included within the true spirit and scope of the present invention.

Claims

1. A matrix processor comprising: a first input circuit configured to receive sensor data; a second input circuit configured to receive one or more filters from a plurality of filters; and a plurality of sub-circuits configured to receive the sensor data and filters, wherein each sub-circuit comprises an arithmetic logic unit, and wherein the sub-circuits are configured to convolve the sensor data and filters, and wherein, to convolve the sensor data and filters, the sub-circuits are configured to sequentially convolve, by means of the sub-circuits, individual subsets of the sensor data with the one or more filters, wherein one or more of the remaining filters are subsequently received for convolution.

2. The array processor according to claim 1, further characterized in that the sensor data comprises image data, LIDAR data, ultrasonic data, or radar data.

3. The array processor according to claim 1, further characterized in that the receiving sensor data comprises reformatted operands representing linearized sensor data.

4. The matrix processor according to claim 1, further characterized in that one or more of the sub-circuits additionally comprises encoders.

5. The matrix processor according to claim 4, further characterized in that at least a portion of the sub-circuits share a particular encoder, and wherein the particular encoder is a cockpit encoder.

6. The array processor according to claim 1, further characterized in that the array processor implements a state machine configured to identify redundant data.

7. The array processor according to claim 6, further characterized in that identifying redundant data is based on an input comprising respective sizes associated with individual filters of the plurality of filters and / or individual steps of one or more steps.

8. The array processor according to claim 1, further characterized in that convolving a first subset comprises: iviA / a / zuzo / ui U4U / determining a convolution of the first subset with one or more of the filters, wherein one or more of the remaining subsets are sequentially convolved with the one or more of the remaining filters.

9. The matrix processor according to claim 1, further characterized in that the matrix processor comprises an arrangement of panels, and wherein the panels comprise respective subsets of the sub-circuits.

10. The array processor according to claim 1, further characterized in that each row of an output array comprises individual subsets of the sensor data convolved with a respective filter from the plurality of filters.

11. A system comprising: a first logic circuit configured to format sensor data and provide the formatted sensor data to an array processor; a second logic circuit configured to provide one or more filters from a plurality of filters to the array processor; and the array processor comprising a plurality of sub-circuits, the sub-circuits being configured to sequentially convolve individual subsets of the sensor data with the one or more filters, wherein one or more of the remaining filters are subsequently received for convolution.

12. The system according to claim 11, further characterized in that sequentially convolving individual subsets of the sensor data comprises: determining a convolution of a first set of the sensor data with one or more of the filters, wherein one or more of the remaining filters from the plurality of filters are received for convolution with the first subset.

13. The system according to claim 11, further characterized in that the system comprises an output array, wherein each row of the output array comprises individual subsets of the sensor data convolved with a respective filter from the plurality of filters.

14. The system according to claim 11, further characterized in that the first logic circuit comprises a plurality of data registers that store portions of the sensor data, the plurality of data registers having a first width that corresponds to the size of an input region obtained from the sensor data.

15. The system according to claim 14, further characterized in that the input region corresponds to an individual subset of the sensor data. iviA / a / zuzo / ui U4U / 16. The system according to claim 11, further characterized in that the system comprises a data formatter configured to linearize sensor data into a plurality of vectors, each vector representing a respective subset of the sensor data.

17. A method implemented by means of a matrix processor, the method comprising: receiving, from a first logic circuit, sensor data comprising a plurality of subsets; receiving, from a second logic circuit, one or more filters from a plurality of filters; and using a plurality of sub-circuits of the matrix processor to sequentially convolve individual subsets with the one or more filters, wherein one or more of the remaining filters are subsequently received for sequential convolution with the individual subsets.

18. The method according to claim 17, further characterized in that after convolving a first subset with one or more filters, a second subset is convolved with one or more filters.

19. The method according to claim 17, further characterized in that the array processor sequentially receives a respective from one or more filters from the plurality of filters.

20. The method according to claim 17, further characterized in that it additionally comprises: outputting, in each row of an output array, individual subsets of the sensor data convolved with a filter associated with the row.