System, method and apparatus for reducing power consumption

Abstract

The present invention discloses a system, device and method for generating a two-dimensional array from three-dimensional input data using a transposition function. The use of the transposition function reduces redundant elements in the obtained two-dimensional array, thereby improving efficiency and reducing power consumption.

Description

Technical Field

[0001] The present technology relates to a system, method and apparatus for data manipulation, thereby providing a reduction in local memory bandwidth by exploiting redundancy / duplication in data mapping. Background Art

[0002] Machine learning techniques continue to advance and already support many aspects of modern society. One of the most useful applications of machine learning is image classification. Image classification can be achieved by using neural networks to efficiently manipulate large amounts of data. Other uses of machine learning include web search, content filtering, automatic recommendations for merchant websites, automated games, object detection, speech recognition, machine translation, and drug discovery. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] The accompanying drawings provide a visual representation that will be used to more fully describe the various representative embodiments and can be used by those skilled in the art to better understand the disclosed representative embodiments and their inherent advantages. In the accompanying drawings, like reference numerals represent corresponding elements.

[0004] Figure 1 Shown is the computation of a single output feature map of a convolutional neural network (CNN) layer.

[0005] Figure 2 The im2col function of a convolutional neural network (CNN) is shown.

[0006] Figure 3 A hardware accelerator for use with a convolutional neural network (CNN) is shown.

[0007] Figure 4 An IM2COL operation for a 3x3 core is shown.

[0008] Figure 5 It is shown that each loop generates the transpose of a single 4-element column.

[0009] Figure 6 The hardware arrangement for implementing the transposition is shown.

[0010] Figure 7 An implementation of the IM2COL transformation for an interleaved data stream is shown.

[0011] Figure 8 The hardware arrangement for implementing the transformation is shown.

[0012] Fig. 9 A flow chart according to an embodiment is shown.

[0013] Fig.10 A flow chart according to an embodiment is shown. DETAILED DESCRIPTION

[0014] Although the present disclosure is capable of many different forms of embodiments, specific embodiments are shown in the drawings and will be described in detail herein, it should be understood that the present disclosure should be considered as an example of the principles described, and is not intended to limit the present disclosure to the specific embodiments shown and described. In the following description, like reference numerals are used to describe the same, similar or corresponding parts in the several views of the drawings.

[0015] In this document, relational terms, such as first and second, top and bottom, etc., may be used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms "comprises," "comprising," "containing," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitations, an element beginning with "comprises" does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0016] Reference throughout this document to "one embodiment," "certain embodiments," "an embodiment," or similar terms means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, the appearances of such phrases in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments without limitation.

[0017] As used herein, the term "or" will be interpreted as an inclusive or meaning any one or any combination. Thus, "A, B, or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B, and C". An exception to this definition will only occur if a combination of elements, functions, or actions are inherently mutually exclusive in some way.

[0018] For simplicity and clarity of description, reference numerals may be repeated in the accompanying drawings to indicate corresponding or similar elements. Many details are set forth to provide an understanding of the embodiments described herein. The embodiments may be practiced without these details. In other cases, well-known methods, procedures, and components are not described in detail to avoid obscuring the embodiments. This specification should not be considered to be limited to the scope of the embodiments described herein.

[0019] As used herein, a "module" describes a component or portion of a program or device that may include hardware or software or a combination of hardware and software. In a module that includes software, the software may include one or more routines or subroutines. One or more modules may constitute a program and / or device.

[0020] Neural networks (NNs) are often used to map or classify a set of input patterns into a set of output patterns. Neural network-based systems have evolved into a popular machine learning foundation and have been successfully used for practical applications in a variety of fields.

[0021] In order to classify input patterns with sufficient accuracy, the neural network first needs to undergo a learning exercise, which is called a training phase. During the training phase, paired training samples of input x and corresponding output or label y (e.g., depicted as (x, y)) are provided to the neural network, which then learns or establishes how to associate or map a given input x with the correct output y.

[0022] Neural networks and related systems may be represented as distributed processing elements that perform summation, multiplication, exponentiation or other functions on elements of incoming messages / signals. Such networks may be enabled and implemented in a variety of implementations.

[0023] For example, the system can be implemented as a network of electrically coupled functional node components. The functional node components can be logic gates arranged or configured in a processor to perform a specified function. As a second example, the system can be implemented as a network model programmed or configured to operate on a processor.

[0024] The network model is preferably electronically stored software that encodes the operations and communications between the nodes of the network.Neural networks and related systems can be used in a variety of applications and can use a variety of data types as input, such as images, video, audio, natural language text, analytical data, widely distributed sensor data, or other suitable forms of data.

[0025] While neural networks have desirable characteristics, a convolutional neural network (CNN) can be thought of as an incompletely connected neural network (each neuron is only connected to a few neurons in the previous layer), and neurons share weights.

[0026] A convolutional neural network (CNN) is a subclass of neural networks with at least one convolutional layer. CNNs are often used to capture local information (e.g., neighboring pixels in an image or surrounding words in a text) as well as to reduce model complexity (faster training, requires fewer samples, reduces the chance of overfitting).

[0027] A convolutional neural network (CNN) is a feed-forward neural network that includes layers that are not fully connected. In a CNN, neurons in a convolutional layer are connected to neurons in a subset or neighborhood of earlier layers. This enables spatial features in the input to be preserved in at least some CNNs.

[0028] Thus, a convolutional neural network (CNN) may include a set of layers, the first layer being an input layer configured to receive input. The input layer includes neurons connected to neurons in a second layer, which may be referred to as a hidden layer. The neurons of the hidden layer may be connected to another hidden layer or an output layer.

[0029] CNNs have repeating blocks of neurons that are applied across space (for images) or time (for audio signals, etc.). For images, these blocks of neurons can be interpreted as 2D convolution kernels, repeatedly applied to each patch of the image. For speech, they can be seen as 1D convolution kernels applied across a time window. At training time, the weights of these repeating blocks are "shared", that is, the gradients of the weights learned over the various image patches are averaged.

[0030] Specifically, convolutional neural networks (CNNs) can be used to perform inference on data where feature recognition is independent of one or more dimensions of the data; for example, when detecting a shape in an image, the detected shape does not depend on its position in the image, and the same features used to detect a square in one part of the image can also be used to detect a square in another part of the image. These dimensions may be spatial dimensions (as in the 2D image example), but may additionally or alternatively be temporal dimensions or any suitable dimensions (e.g., frequency dimensions for audio or multi-spectral light data). According to several embodiments, the process may be performed by a "cloud" server system. In further embodiments, the process may be performed on a user device.

[0031] The embodiments described herein are applicable in part or in whole to any network layer, including CNN, FC or other layers in a network.

[0032] Convolutional Neural Networks (CNNs) are often processed by converting multi-dimensional convolution operations into general matrix multiplications (GEMMs) using a simple transformation often called image-to-column (IM2COL).

[0033] Convolutional Neural Networks (CNNs) are currently very popular for implementing tasks such as image classification. CNNs are challenging to implement in constrained devices because they require a large number of MAC operations operating on very large data structures.

[0034] As described herein, the IM2COL transform is used to convert the input from an image (which is effectively a 3D array) into a 2D array, which can be viewed as a matrix. Each kernel of the 3D array image to which the transform is applied is a three-dimensional cube within the image, so each of those cubes of input values ​​is copied into the matrix as a single column.

[0035] As described herein, embodiments of the present disclosure present a hardware implementation of IM2COL that results in a substantial reduction in local SRAM (Static Random Access Memory) bandwidth by exploiting redundancy / duplication in the IM2COL mapping.

[0036] MAC (Multiply Accumulate) operations are used in computational operations, especially DSP (Digital Signal Processing). A MAC operation is a general operation that calculates the product of two numbers and adds the product to an accumulator. The hardware unit that performs this operation is called a multiplier-accumulator (MAC or MAC unit); the operation itself is also often referred to as a MAC or MAC operation.

[0037] Figure 1 The basic CNN layer calculation is shown in for a 3×3 filter size.

[0038] like Figure 1 As shown, system 100 includes weights 102, which include weights for each layer 103 (a) ... 103 (n), where "n" is any suitable number. Weights 102 may generally be established for each layer 103, either established, or a combination of fixed and programmable.

[0039] The input feature map 104 has a plurality of layers 105(a)...105(n), where "n" is any suitable number.

[0040] The output feature map 106 is generated by performing a logical multiplication or other suitable operation on the weights 102 and the input feature map (IFM) 104 using the respective layers 103 and 105. Figure 1 An example of a 3x3 filter is shown, but any suitable structure may be used.

[0041] A common way to implement convolution in CPUs (central processing units), GPUs (graphics processing units), and dedicated hardware accelerators is to convert it into a general matrix multiplication (GEMM) operation.

[0042] GEMMs can be used in convolutional layers because convolutional layers treat their input as a two-dimensional image with multiple channels per pixel, much like a classic image with width, height, and depth.

[0043] In software, GEMM operations are performed by calling library functions. In hardware, GEMM operations are often efficiently implemented as 2D MAC arrays. For example, TPU and MIT The chips are two examples of specialized hardware accelerators for CNNs.

[0044] like Figure 2 As shown, to convert a convolution into a GEMM, an IM2COL transform can be used. The IM2COL transform is used to "flatten" a 3D data structure into a 2D structure that can be efficiently processed with GEMM-like operations. This provides a good location in the basic 1D memory system.

[0045] Figure 2 A system 200 is shown that includes an input image 208. The input image 208 has a three-dimensional (3D) data structure shown as elements 209(a) ... 209(n), where "n" is any suitable number of structures 208. An IM2COL transform 210 is used to transpose or transform or flatten the 3D data structure 208 into a two-dimensional (2D) data structure 212. The 3D elements 209(a) ... 209(n) of the structure 208 are respectively transposed into corresponding elements 213(a) ... 213(n) of the 2D structure 212, where "n" is any suitable number.

[0046] For software implementation, such as Figure 2 As shown, the IM2COL operation 210 is typically done in memory and results in an increased memory footprint. The memory cost of IM2COL 210 exceeds the efficiency and memory locality achievable with a software GEMM implementation.

[0047] Figure 3 A hardware accelerator 300 is shown for use with a convolutional neural network (CNN). Figure 3 The accelerator 300 is shown having GPV registers 320 , a sequencer module 326 , a command buffer 332 , a TBU 342 , a DMA 348 , a data buffer 352 , an IM2COL module 368 , a systolic array 374 , and a scalar module 376 .

[0048] In a hardware accelerator, IM2COL 368 can be done within the accelerator before the MAC array, and thus typically does not result in an increase in memory usage. However, the redundancy in the flattened representation results in multiple reads per word, which increases power consumption due to local SRAM read power. For context, Figure 3 A typical hardware accelerator for a CNN is shown, indicating the IM2COL transform 368 before the MAC array, which reads data from the local SRAM within the accelerator.

[0049] Figure 4The IM2COL operation for a 3×3 core is shown. Two iterations of the patch 420 are shown (420(a) and 420(b)). The 3×3 patch 420(a) and 420(b), typically 420 is flattened into rows in the output matrix 430 (shown as two iterations 430(a) and 430(b)). In the second iteration 420(b), 430(b), IM2COL actually reads only three new elements (D, J, P), six of the nine elements having already been read in the previous cycle (B, H, N, C, I, O) 420(a). This is clearly inefficient and can be optimized to reduce the number of words read from the SRAM, thereby reducing power consumption.

[0050] Additional enhancements can be achieved by using two optimization algorithms and hardware implementations for implementing IM2COL. The first is applicable to SIMD accelerators and "non-pipelined" MAC arrays, which typically utilize "square-aligned" arrays. The second targets systolic arrays, which typically utilize skew / diagonal patterns in data movement.

[0051] Figure 5 Transposed IM2COL 510 (510(a) and 510(b) iterations) is shown to generate a single 4-element column representation per cycle 500. Average internal memory bandwidth is reduced by a factor of two (load 6B instead of 12B).

[0052] Figure 5 The method in is to use IM2COL 510(a) to transform from the patch 533(a) of 520(a) to the row 535(a) of 530(a). Therefore, transposing 510(a) turns the small row 533(a) into a single column 535(a).

[0053] This transposition 510(a) generates a single output column 535(a) per cycle, rather than a full row, which means more buffering. For the initial operation, this involves reading a 4-element row, which becomes a 4-element column in the output array. However, after this initial operation, the subsequent cycles shown as part (b) require only a single 1B load and shift operation.

[0054] like Figure 5 As shown, element 533(b) of diagram 520(b) is transposed by IM2COL 510(b) to place the element shown as 535(b) in diagram 530(b). Figure 5 As shown, this is a total internal bandwidth reduction of one-half.

[0055] Figure 6 A hardware arrangement 600 for implementing the transposition is shown, which is shown as IM2COL transposition ( Figure 5 ).like Figure 6As shown, arrangement 600 includes one or more IFM SRAMs shown generally as 602, a transpose module 604, a temporary register 606, a shift module 608, and a module 620 which may be a SIMD unit or a non-pipelined MAC array.

[0056] The IFM SRAM 602 is a memory register for storing input feature maps and layer information as well as map information provided to or accessed by registers 606, as shown at 612. The IFM SRAM 602 also receives input from the transpose module 604, as shown at 614.

[0057] The transposition module 604 may store a transposition function for transposing the 3×3 input data, such as IM2COL. The transposition function is applied to the input data using an algorithm, such as Fig. 9 and Fig.10 The transposition module 604 can also be regarded as a control and address generation module.

[0058] The register 606 (which may be a temporary register) is used to store the input feature map (IFM) accessed from the SRAM 602. The temporary register 606 may be a suitable buffer or storage medium for storing the content accessed from the SRAM 602.

[0059] Shifter (shift register) 608 is operatively coupled to register 606 as indicated at 616. Shift register 608 shifts data to the left or right from register 606. Control and address generation module 604 provides input to shifter 608 as indicated at 624.

[0060] Module 620 may be a SIMD module or alternatively a non-pipelined MAC array.

[0061] Module 620 may be a SIMD (Single Instruction Multiple Data) module that performs the same operation on multiple data points simultaneously or substantially simultaneously.

[0062] Alternatively, module 620 may be a non-pipelined MAC array, which may be a module comprising a multiplier implemented in combinatorial logic, followed by an adder and an accumulator register to store the result.

[0063] In the MAC array, the output of the register is fed back to one input of the adder so that on every clock cycle, the output of the multiplier is added to the register.

[0064] Figure 7 An implementation 700 of the IM2COL transformation for interleaved data streams is shown. Figure 7As shown, IM2COL converts the interleaved data stream as required by the systolic array. This achieves a one-half reduction in internal bandwidth and does not use any more than 2B / cycle.

[0065] Systolic arrays usually require an interleaved data pattern. Interleaved patterns are actually already inherent in IM2COL, which results in a pattern with many repeating elements next to each other and is therefore implemented efficiently.

[0066] Figure 7 Transposed IM2COL 710 (710(a) and 710(b) iterations) is shown to generate a single 4-element column representation per cycle 700. Average internal memory bandwidth is reduced by a factor of two (load 6B instead of 12B).

[0067] Figure 7 The method in FIG. 7 is to transpose the 3×3 IFMAP 740 (740(a) and 740(b)) to FIG. 750 (750(a) and 750(b)) using IM2COL 710(a) and 710(b). Therefore, the patch "E" 742(a) of FIG. 740(a) and the patch "H" 746(a) of FIG. 740(a) are transposed to "E", "E" 744(a) of FIG. 750(a) and "H", "H" 748(a) of FIG. 750(a), respectively.

[0068] like Figure 7 As shown, the element "F" 742(b) of Figure 740(b) is transposed by IM2COL 710(b) to place the element "F" shown as element 742(b) into the "F" shown as 744(b) in Figure 750(b). The element "I" 746(b) of Figure 740(b) is transposed by IM2COL 710(b) into three elements "I", "I", "I" shown as 748(b) in Figure 750(b).

[0069] Figure 8 A hardware arrangement 800 for implementing the transformation is shown. Figure 8 As shown, use the mux crossbar instead Figure 6 The shift register shown.

[0070] Specifically, Figure 8 A hardware arrangement 800 for implementing the transposition is shown, which is shown as IM2COL transposition ( Figure 7 ).like Figure 8 As shown, arrangement 800 includes one or more IFM SRAMs shown generally as 802, a transpose module 804, temporary registers 806, a crossbar 809 which may be a multiplexing (mux) crossbar, and a module 820 which may be a SIMD unit or a non-pipelined MAC array.

[0071] IFM SRAMS 802 are memory registers for storing input feature maps and layer information as well as map information provided to or accessed by registers 806, as shown at 812. IFM SRAM 802 also receives input from the transpose module (also known as the control and address generation module 804), as shown at 814.

[0072] The transposition module 804 may store a transposition function, such as IM2COL, for transposing the 3×3 input data. The transposition function is applied to the input data using an algorithm, such as Fig. 9 and Fig.10 The transposition module 804 can also be regarded as a control and address generation module.

[0073] The register 806 (which may be a temporary register) is used to store the input feature map (IFM) accessed from the SRAM 802. The temporary register 806 may be a suitable buffer or storage medium for storing the content accessed from the SRAM 802.

[0074] A crossbar or multiplexer 809 is operatively coupled to the registers 806 as shown at 816. The crossbar 809 multiplexes the data received from the registers 806. The control and address generation module 804 provides an input to the crossbar or multiplexer 809 as shown at 824.

[0075] Module 820 may be a SIMD module, or alternatively a non-pipelined MAC array, as described herein with respect to Figure 6 described.

[0076] Arrangement 800 shows a Figure 6 Compared to modified hardware to support interleaved data patterns, this arrangement has the characteristic of requiring only at most two new operands per cycle.

[0077] Fig. 9 A flowchart 900 according to an embodiment is shown. Algorithm 900 starts (902). A three-dimensional input data representation is accessed (904). The three-dimensional data representation can be an image, audio, a combination of images and audio, or any type of data that can be stored. The input can be an input feature map, as described herein.

[0078] A transposition function, such as the IM2COL function, is accessed (908). The transposition function (eg, the IM2COL function) is applied to the three-dimensional data (910) and used to "flatten" the three-dimensional input data representation so that it can be represented as a corresponding two-dimensional data representation.

[0079] Application of the transpose function identifies input data rows (912). These identified input data rows are read (914) and one or more output columns are generated per cycle (918). Another embodiment of the present disclosure is that higher throughput will result in more output columns per cycle. In practice, the throughput level determines the output columns per cycle.

[0080] A transposition function (eg, IM2COL function) identifies redundant elements based on the transposition of the input data (920). The redundant element locations (924) and quantities (926) are part of this identification.

[0081] Redundant elements identified by the transposition are reduced (930). It is determined whether additional data elements exist (932). If so (934), the transposition is applied to the additional input data elements. If not (936), the resulting array is generated (940). The algorithm ends (942).

[0082] Fig.10 A flowchart 1000 is shown according to an embodiment. The algorithm 1000 starts (1002). A three-dimensional input data representation is accessed (1004). The three-dimensional data representation can be an image, audio, a combination of images and audio, or any type of data that can be stored. The input can be an input feature map, as described herein.

[0083] A transposition function, such as the IM2COL function, is accessed (1006). The transposition function (eg, the IM2COL function) is applied to the three-dimensional data (1008) and used to "flatten" the three-dimensional input data representation so that it can be represented as a corresponding two-dimensional data representation.

[0084] Application of the transpose function identifies input data rows (1010). These identified input data rows are read (1012) and one or more output columns are generated per cycle (1014). Another embodiment of the present disclosure is that higher throughput will result in more output columns per cycle. In practice, the throughput level determines the output columns per cycle.

[0085] A transposition function (eg, IM2COL function) identifies redundant elements based on the transposition of the input data (1016). The redundant element locations (1018) and quantities (1020) are part of this identification.

[0086] Redundant elements identified by the transposition are reduced (1022). The subsequent loop is accessed (1024) and a single load and shift operation is performed (1026).

[0087] A determination is made as to whether additional data elements exist (1028). If so (1030), the transposition is applied to the additional input data elements. If not (1032), the resulting array is generated (1040). The algorithm ends (1050).

[0088] A series of operations 900( Fig. 9 ) and 1000( Fig.10 ) may be stored in non-volatile memory or other suitable storage or computer usable media and in any suitable programming language.

[0089] Any combination of one or more computer usable or computer readable media may be utilized. Computer usable or computer readable media may be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, equipment, or propagation media. More specific examples of computer readable media (a non-exhaustive list) would include the following: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CDROM), an optical storage device, a transmission medium (such as those supporting the Internet or an intranet), or a magnetic storage device.

[0090] The computer usable or computer readable medium may even be paper or another suitable medium on which the program is printed, as the program may be captured electronically, for example, via optical scanning of the paper or other medium, and then compiled, interpreted, or otherwise processed in a suitable manner as desired, and then stored in a computer memory. In the context of this document, a computer usable or computer readable medium may be any medium that may contain, store, convey, propagate, or transmit the program for use by or in connection with an instruction execution system, apparatus, or device. The computer usable medium may include a propagated data signal having the computer usable program code embodied therewith, either in baseband or as part of a carrier wave. The computer usable program code may be transmitted using any suitable medium, including but not limited to wireless, wireline, fiber optic cable, RF, etc.

[0091] The computer program code for performing the operation of the present disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smallalk, C++, C#, and conventional procedural programming languages ​​such as "C" programming language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any type of network (including a local area network (LAN) or a wide area network (WAN)), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0092] The present disclosure is described below with reference to the flowchart and / or block diagram of the method, device, system and computer program product according to the embodiment.It should be understood that each frame of the flowchart and / or block diagram and the combination of frames in the flowchart and / or block diagram can be implemented by computer program instructions.

[0093] These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a device for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram. These computer program instructions may also be stored in a computer-readable medium, which may instruct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable medium produce an article including an instruction device, which implements the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0094] The computer program instructions may also be loaded onto a computer or other programmable data processing device to cause a series of operations to be performed on the computer, or loaded onto other programmable devices to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide a process for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0095] The computer usable or computer readable medium may even be paper or another suitable medium on which the program is printed, as the program may be captured electronically, for example, via optical scanning of the paper or other medium, and then compiled, interpreted, or otherwise processed in a suitable manner as desired, and then stored in a computer memory. In the context of this document, a computer usable or computer readable medium may be any medium that may contain, store, convey, propagate, or transmit the program for use by or in connection with an instruction execution system, apparatus, or device. The computer usable medium may include a propagated data signal having the computer usable program code embodied therewith, either in baseband or as part of a carrier wave. The computer usable program code may be transmitted using any suitable medium, including but not limited to wireless, wireline, fiber optic cable, RF, etc.

[0096] The present embodiment is described below with reference to the flowchart and / or block diagram of the method, apparatus, system and computer program product according to the embodiment.It should be understood that each frame of the flowchart and / or block diagram and the combination of frames in the flowchart and / or block diagram can be implemented by computer program instructions.

[0097] Unless otherwise stated, terms such as "first" and "second" are used to arbitrarily distinguish between the elements such terms describe. Therefore, these terms are not necessarily intended to indicate temporal or other prioritization of such elements.

[0098] As will be appreciated by those skilled in the art, the present disclosure may be embodied as a system, method, or computer program product. Thus, embodiments may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, which may be collectively referred to herein as "circuits," "modules," or "systems." Furthermore, embodiments may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium.

[0099] It can be seen that the systems and methods presented herein provide an advancement over the prior art.

[0100] Therefore, some of the embodiments disclosed in the present invention are listed in the following items:

[0101] 1. A method comprising: accessing a three-dimensional representation of input data; accessing a transpose function; applying the transpose function to the input data; identifying one or more rows of the input data; generating one or more columns for each cycle of the input data; identifying redundant characteristics of the input data based on each generated column; reducing the redundant characteristics of each column; performing load and shift operations on subsequent cycles of the input data; and generating a resulting array based on the reduced columns.

[0102] 2. The method according to item 1 also includes performing the same operation on multiple input data points of the input data substantially simultaneously.

[0103] 3. The method of claim 1, wherein the transposition function is an image-to-column transposition function.

[0104] 4. The method according to item 1 also includes performing multiple multiplication and accumulation operations on the input data.

[0105] 5. The method of claim 1, wherein the storage blocks of the resulting array are aligned.

[0106] 6. The method according to item 1 also includes reading four-element rows of the input data to generate four-element columns in the resulting array.

[0107] 7. The method according to item 1, wherein the resulting array is a two-dimensional array.

[0108] 8. A method comprising: accessing a three-dimensional representation of input data; accessing a transpose function; applying the transpose function to the input data; identifying one or more rows of the input data; generating one or more columns for each cycle of the input data; identifying redundant characteristics of the input data based on each generated column; reducing the redundant characteristics of each column; and generating a resulting array based on the reduced columns.

[0109] 9. The method of claim 8, wherein each subsequent loop utilizes at most two new operands per loop.

[0110] 10. The method according to item 8 further comprises performing the same operation on multiple input data points of the input data substantially simultaneously.

[0111] 11. The method of clause 8, wherein the memory blocks of the resulting array are interleaved.

[0112] 12. The method according to item 8, wherein the resulting array is a two-dimensional array.

[0113] 13. The method of claim 8, wherein the function is an image-to-column transpose function.

[0114] 14. A device, comprising: an input module, the input module being suitable for storing a three-dimensional input data representation; a transposition module, the transposition module being suitable for storing a transposition function, wherein: the transposition function is applied to the three-dimensional input data representation; the transposition function identifies one or more rows of the input data; the transposition function generates one or more columns for each cycle of the input data; the transposition function identifies redundant characteristics of the input data based on each single column generated; the transposition function reduces the redundant characteristics of each column; and the transposition function generates a resulting array based on the reduced columns.

[0115] 15. The apparatus of clause 14, wherein the transposition function is an image-to-column transposition function.

[0116] 16. The device according to item 14, wherein the resulting array is a two-dimensional array.

[0117] 17. The apparatus of clause 14, wherein the memory blocks of the resulting array are aligned.

[0118] 18. The apparatus of clause 14, wherein the memory blocks of the resulting array are interleaved.

[0119] 19. An apparatus according to item 14, wherein the transposition function performs the same operation on multiple input data points of the input data substantially simultaneously.

[0120] 20. A system comprising: a memory; and a processor, the processor being coupled to the memory and adapted to execute instructions stored in the memory, the instructions comprising: accessing a three-dimensional representation of input data; accessing a transposition function; applying the transposition function to the input data; identifying one or more rows of the input data; generating one or more columns for each cycle of the input data; identifying redundant characteristics of the input data based on each generated column; reducing the redundant characteristics of each column; and generating a resulting array based on the reduced columns.

[0121] 21. A system according to item 20, wherein the instruction also includes performing load and shift operations on subsequent cycles of the input data.

[0122] 22. The system of item 20, wherein the transpose function is an image-to-column transpose function.

[0123] 23. The system of item 20, wherein the storage blocks of the resulting array are aligned.

[0124] 24. The system of item 20, wherein the memory blocks of the resulting array are interleaved.

[0125] 25. The system of item 20, wherein the resulting array is a two-dimensional array.

[0126] 26. A system according to item 20, wherein the instructions also include performing the same operation on multiple input data points of the input data substantially simultaneously.

[0127] Various representative embodiments have been described in detail herein by way of example and not limitation. It will be appreciated by those skilled in the art that various changes may be made to the form and details of the embodiments described to yield equivalent embodiments within the scope of the appended claims.

Claims

1. A method for optimizing a convolutional neural network (CNN) model, the method comprising: accessing a three-dimensional representation of input image data; Access the transpose function; applying the transpose function to the input image data; identifying one or more rows of the input image data; generating one or more columns of input image data during each clock cycle; identifying redundant characteristics of the input image data based on each of the generated columns; reducing said redundant characteristics of each column; Performing loading and shifting operations on the input image data in subsequent clock cycles; and The resulting array is generated based on columns with the redundant properties reduced.

2. The method according to claim 1 also includes performing the same operation on multiple input image data points of the input image data at the same time or performing the same operation on multiple input image data points of the input image data by executing a single instruction. The method of claim 1 , wherein the transpose function is an image-to-column transpose function.

4. The method of claim 1, further comprising performing a plurality of multiplication and accumulation operations on the input image data. The method of claim 1 , wherein the memory blocks of the resulting array are aligned.

6. The method of claim 1, further comprising reading four-element rows of the input image data to generate four-element columns in a resulting array.

7. The method of claim 1, wherein the resulting array is a two-dimensional array.

8. A method for optimizing a convolutional neural network (CNN) model, the method comprising: accessing a three-dimensional representation of input image data; Access the transpose function; applying the transpose function to the input image data; identifying one or more rows of the input image data; At each clock cycle, one or more columns of input image data are generated; identifying redundant characteristics of the input image data based on each of the generated columns; reducing said redundant characteristics of each column; and The resulting array is generated based on columns with the redundant properties reduced.

9. The method of claim 8, wherein each subsequent clock cycle utilizes at most two new operands per cycle.

10. The method of claim 8, further comprising performing the same operation on a plurality of input image data points of the input image data simultaneously or performing the same operation on a plurality of input image data points of the input image data by executing a single instruction.

11. The method of claim 8, wherein the memory blocks of the resulting array are interleaved.

12. The method of claim 8, wherein the resulting array is a two-dimensional array.

13. The method of claim 8, wherein the function is an image to column transpose function.

14. A device for optimizing a convolutional neural network (CNN) model, the device comprising: an input module adapted to store a three-dimensional input image data representation; A transposition module, the transposition module being adapted to store a transposition function, wherein: The transposition function is applied to the three-dimensional input image data representation; The transpose function identifies one or more rows of the input image data; The transpose function generates one or more columns of input image data in each clock cycle; The transposition function identifies redundant characteristics of the input image data based on each individual column generated; The transposition function reduces the redundant nature of each column; and The transpose function produces a resulting array based on the columns that reduces the redundant properties. The apparatus of claim 14 , wherein the transposition function is an image-to-column transposition function.

16. The device of claim 14, wherein the resulting array is a two-dimensional array.

17. The apparatus of claim 14, wherein the memory blocks of the resulting array are aligned.

18. The apparatus of claim 14, wherein the memory blocks of the resulting array are interleaved.

19. The apparatus of claim 14, wherein the transposition function performs the same operation on a plurality of input image data points of the input image data simultaneously or performs the same operation on a plurality of input image data points of the input image data by executing a single instruction.

20. A system for optimizing a convolutional neural network (CNN) model, the system comprising: Memory; and A processor, coupled to the memory, adapted to execute instructions stored in the memory, the instructions comprising: accessing a three-dimensional representation of input image data; Access the transpose function; applying the transpose function to the input image data; identifying one or more rows of the input image data; generating one or more columns of input image data during each clock cycle; identifying redundant characteristics of the input image data based on each of the generated columns; reducing said redundant characteristics of each column; and The resulting array is generated based on columns with the redundant properties reduced.

21. The system of claim 20, wherein the instructions further comprise performing load and shift operations on the input image data in subsequent clock cycles.

22. The system of claim 20, wherein the transpose function is an image-to-column transpose function.

23. The system of claim 20, wherein the memory blocks of the resulting array are aligned.

24. The system of claim 20, wherein the memory blocks of the resulting array are interleaved.

25. The system of claim 20, wherein the resulting array is a two-dimensional array.

26. The system of claim 20, wherein the instructions further comprise performing the same operation on multiple input image data points of the input image data simultaneously or performing the same operation on multiple input image data points of the input image data by executing a single instruction.

Images