Circuits, methods, and systems for convolving input data

Abstract

The embodiments described herein provide systems, circuits, and methods for convolving a filter ("kernel") with input data in the form of an input array by reusing calculations of duplicated data items in the input array due to movement from one convolution step to the next. In one embodiment, in order to compute a convolution of an input matrix and a filter matrix, instead of expanding data items from the input matrix of each convolution step into an input vector, only non-duplicated new data items of each convolution step may be added to the input vector. An input mapping circuit implementing the input parameter mapping matrix may then iteratively map data items of the input vector to different weight registers corresponding to weights in the filtering matrix.

Description

Technical Field

[0001] Embodiments of the present invention generally relate to the field of semiconductors, and more particularly, to circuits, methods, and systems for convolving input data. Background Art

[0002] Artificial intelligence (AI) accelerators include specialized hardware components and / or devices for accelerating the execution of AI and machine learning workloads. Example workloads include convolution operations, which are often performed in deep learning and convolutional neural networks, such as in tasks such as image recognition, natural language processing, computer vision, etc. Convolution involves applying a filter (also called a "core") matrix to an input data matrix to extract features. This calculation also requires that, for each position of the input matrix and the core matrix, the corresponding term values ​​be multiplied and added together. Traditional AI accelerators perform multiplication operations for each convolution step by expanding the input parameters from the input matrix and expanding them into vector form, even though some input parameters may be repeated in different convolution steps. Therefore, a large number of input registers are often required to store the expanded input vectors from the input matrix, which results in a considerable circuit area and computing power being occupied. Summary of the invention

[0003] According to an embodiment of the present invention, there is provided a circuit for convolving input data and a weight matrix, comprising: an input register configured to store an input vector of non-repeated data items from an input data array; an input mapping circuit configured to receive a first data item from the input register and selectively send the first data item to a first output based on a control signal indicating a stride of the convolution, wherein the first output is connected to a first weight register at a first computing unit, and the first computing unit performs multiplication of the first data item with a first weight corresponding to the first weight register for the convolution. 2. The circuit according to claim 1, wherein the input vector of non-repeated data items is obtained by expanding the non-repeated data items from the input data array based on the stride of the convolution.

[0004] According to another embodiment of the present invention, a method for convolving input data and a weight matrix is ​​provided, comprising: obtaining, at an input register, an input vector of non-repeating data items from an input data array; receiving a first data item of the input vector from the input register at a first input of an input mapping circuit; selectively transmitting the first data item to a first output of a first weight register connected to a first computing unit within the input mapping circuit based on a control signal indicating a stride of the convolution; and performing multiplication of the first data item with a first weight corresponding to the first weight register for the convolution.

[0005] According to another embodiment of the present invention, a system for convolving input data and a weight matrix is ​​provided, comprising: a memory storing a plurality of instructions; one or more hardware processors executing the plurality of instructions to perform operations, the operations comprising: obtaining an input vector of non-repeating data items from an input data array at an input register; receiving a first data item of the input vector from the input register at a first pin of the input mapping circuit; selectively transmitting the first data item to a first output of a first weight register connected to a first computing unit within the input mapping circuit based on a control signal indicating the stride of the convolution; and performing multiplication of the first data item with a first weight corresponding to the first weight register for the convolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Various aspects of the present invention will be best understood from the following detailed description when read in conjunction with the accompanying drawings. It should be noted that, in accordance with standard practice in the industry, the various components are not drawn to scale. In fact, the sizes of the various components may be arbitrarily increased or reduced for clarity of discussion.

[0007] Figure 1 An example of a convolutional layer performing a convolution operation in a convolutional neural network (CNN) according to one or more embodiments described herein is shown.

[0008] FIG. 2A to FIG. 2B An embodiment according to one or more embodiments described herein is shown. Figure 1 An example of a convolution operation is shown.

[0009] FIG. 3A to FIG. 3B According to one or more embodiments described herein, FIG. 2A to FIG. 2B In the example shown, an example of expanding an input data array into an input vector to perform convolution with a filter matrix is ​​shown without and with data reuse.

[0010] FIG. 3C to FIG. 3D The corresponding embodiment according to the present invention is shown. Figure 3B Instance input data reuse rate.

[0011] Figure 4 According to one or more embodiments described herein, using Figure 3B An example structure of a circuit for computing a convolution operation using a memory efficient data unrolling scheme is shown.

[0012] Figure 5 An embodiment according to one or more embodiments described herein is shown. Figure 4 An example structure of a stride-aware input mapping circuit is shown, which uses a shift register for selective input.

[0013] Figure 6 The present invention shows a method according to one or more embodiments described herein. Figure 4 An example structure of a stride-aware input mapping circuit using input multiplexers for selective inputs is shown.

[0014] Figure 7 The present invention shows a method according to one or more embodiments described herein. Figure 5 to Figure 6 An example circuit structure of the stride matrix is ​​shown.

[0015] FIG. 8A to FIG. 8B An example stride matrix for a stride-1 convolution is shown in accordance with one or more embodiments described herein.

[0016] FIG. 9A to FIG. 9B An example stride matrix for a stride-2 convolution is shown in accordance with one or more embodiments described herein.

[0017] FIG. 10A to FIG. 10C According to one or more embodiments described herein, FIG. 8A to FIG. 9B Example stride matrices for stride-1 and stride-2 of the stride matrix design shown.

[0018] FIG. 11A to FIG. 11B An illustrative example is provided showing an embodiment according to the present invention. Fig. 10B The hardware implementation of the stride matrix 1006 is superimposed.

[0019] FIG. 12A to FIG. 12C According to one or more embodiments described herein, using Figure 6 An example of an input multiplexer in the Input Multiplexer Map.

[0020] Fig.13 is an example logic flow diagram according to an embodiment described herein, showing the use of Figure 1 to Figure 1 2 The circuit structure performs the convolution process of the input array and the filter matrix.

[0021] Fig.14 According to one embodiment described herein, using Figures 1 to 13 A simplified diagram of a computing device implementing a convolution operation using the input mapping circuit described in FIG. DETAILED DESCRIPTION

[0022] The present invention provides many different embodiments or examples for realizing the different features of the present disclosure. Specific examples of components and arrangements are described below to simplify the present invention. Of course, these are merely examples and are not intended to limit the present invention. For example, in the following description, forming a first component above or on a second component may include an embodiment in which the first component and the second component are formed in direct contact, and may also include an embodiment in which an additional component may be formed between the first component and the second component so that the first component and the second component may not be in direct contact. In addition, the present invention may repeat reference numerals and / or characters in various examples. This repetition is for the purpose of simplicity and clarity, and does not itself indicate the relationship between the various embodiments and / or configurations discussed.

[0023] Furthermore, for ease of description, spatially relative terms such as "below," "beneath," "lower," "above," "upper," etc. may be used herein to describe the relationship of one element or component to another (or additional) elements or components as illustrated in the figures. The spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. The device may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein should likewise be interpreted accordingly.

[0024] As used herein, the term "network" may include any hardware or software based framework, including any artificial intelligence network or system, neural network or system and / or any training or learning model implemented thereon or using the same.

[0025] As used herein, the term "module" may include a hardware or software-based framework that performs one or more functions. In some embodiments, the module may be implemented on one or more neural networks.

[0026] The present application relates to computing circuits, and more specifically, to methods and devices for convolving input data. The embodiments described herein provide systems, devices and methods for convolving a filter ("kernel") with input data in the form of an input array by reusing the calculation of repeated data items in the input array caused by moving from one convolution step to the next convolution. In one embodiment, in order to calculate the convolution of an input matrix and a filter matrix, instead of expanding the data items from the input matrix of each convolution step into an input vector, only the non-repeated new data items at each convolution step can be added to the input vector. The resulting input vector can then be input to an input register. Then, an input mapping circuit that implements an input parameter mapping matrix can iteratively map the data items of the input vector to different weight registers corresponding to the weights in the filter matrix. Then, the computing unit can multiply the mapped data items and the corresponding weights, and add these multiplication results to perform the convolution step.

[0027] In this way, as the convolution moves, data items from the original input data array are reused in different convolution steps, so fewer macro external memory accesses can be performed, which improves memory bandwidth efficiency. In addition, with less data movement between memory and / or input registers, the dynamic energy efficiency of the input buffer can be improved.

[0028] In one embodiment, a system that applies an input parameter mapping matrix to perform convolution between an input data matrix and a filter matrix, such as an AI accelerator that performs a convolutional neural network (CNN) operation, a communication and / or voice / video processing system that applies a filter to input data, etc., can improve computation, memory, and power efficiency by reducing memory bandwidth requirements and / or buffer dynamic energy requirements. Therefore, AI, communication, and voice technologies and / or other types of technologies are improved.

[0029] Figure 1 An example of a convolution layer in a CNN performing a convolution operation according to one or more embodiments described herein is shown. In one embodiment, the neural network may include multiple layers 107, 109, etc. Layer 107 may perform a convolution operation on an input data array (matrix) 102 using a core and / or filter matrix 103 having weights. The convolution output 108, e.g., the result of convolving the input matrix 102 with the core matrix 103, may then be passed to the next layer 109. In one embodiment, the convolution output 108 may take the form of a feature map representing features extracted from the input matrix 102.

[0030] FIG. 2A to FIG. 2B An embodiment according to one or more embodiments described herein is shown. Figure 1 In at least one embodiment, the convolution operation can be performed in one or more iterations. For example, Figure 2A As shown, at the first iteration of the convolution of the input matrix 102 and the core matrix 103, the portion within the sliding window 201 may be dot-producted with the core matrix 103, and the results are combined to obtain the sum "-14" as the corner data item of the feature matrix 108. In one embodiment, since the sliding window 201 contains the corner data items of the input matrix 102, the iteration may be referred to as the "corner" mode of the convolution.

[0031] exist Figure 2B In the second iteration of the convolution, the sliding window may be moved to position 202 to repeat similar operations of matrix dot products and product sums, thereby obtaining "-1" as the second entry in the resulting feature matrix 108. Since the sliding window 202 of the second iteration does not contain the corners of the input matrix 102, this iteration may be referred to as a "non-corner mode of convolution".

[0032] In one embodiment, the sliding window may continue to move from left to right until each entry of the 4×4 feature matrix 108 is calculated using similar operations of matrix dot products and sums of products as described above.

[0033] In one embodiment, FIG. 2A to FIG. 2B In each of the examples shown, Figure 2A The first iteration in Figure 2B In the second iteration of , the sliding window moves one column from left to right. This amount of movement is called the "stride", for example, in FIG. 2A to FIG. 2B In the example in , stride = 1. In some embodiments, the stride can be set to 1, 2, 3, ..., and in CNN, the stride can be set to 1 or 2. In one embodiment, the neural network can include multiple convolutional layers that can use different strides.

[0034] FIG. 3A to FIG. 3B According to one or more embodiments described herein, FIG. 2A to FIG. 2B In the example shown, the input data array 102 is expanded into an input vector to perform a convolution with the filter matrix 103 without and with data reuse. Figure 3A As shown, in one embodiment, a conventional convolutional network expands the input data array without any data reuse, for example, before being combined with the filter matrix 103 (such as FIG. 2A to FIG. 2B All input parameters used when performing adjacent 3×3 convolutions are expanded into an input vector. Therefore, assuming that four computational units (e.g., Figure 4 The convolution is calculated using the multiply-accumulate (MAC) units 421 to 424 shown in FIG. 4 , and a total of 9×4=36 parameters are expanded into the input vector and are thus stored in the input register.

[0035] like Figure 3B As shown, instead of expanding all input parameters from the input data array 102 to perform adjacent 3×3 convolutions, only the different and non-repeated input parameters used in four adjacent 3×3 convolutions are expanded into the input vector. In this way, the input vector contains a total of 9+3×3=18 input parameters for storage in the input register. Therefore, through data reuse, less input register capacity and / or less macro external memory accesses can be achieved to read data into the input register, which improves memory efficiency.

[0036] FIG. 3C to FIG. 3D The corresponding embodiment according to the present invention is shown. Figure 3BInstance input data reuse rate. When only non-repeated data items from the input data array are expanded into the input vector, the previously expanded data items can be reused in multiple computations. The input reuse rate (IRR) can be defined as the number of computations per memory fetch. Figure 3C As shown in , for stride-1 convolution, when the input data array is significantly larger than the filter size, the input data reuse ratio is close to 3, so a large number of iterations need to be performed. Figure 3D As shown, for stride-2 convolution, the reuse ratio of input data is close to 1.5 when the input data array is significantly larger than the filter size.

[0037] Figure 4 4 shows an example structure of a circuit 400 for computing a convolution operation using Figure 3B In one embodiment, the circuit 400 may include an input register array 402, a stride-aware input mapping circuit 410, a plurality of computation units (eg, MAC units) placed in parallel, and a memory array 415.

[0038] In one embodiment, the input register array 402 may be configured to store Figures 1 to 2B 4. The input data array 102 is shown as a macro external memory unit. The input vector 405 is formed by performing adjacent convolutions by expanding only non-repeating data items, for example, as Figure 3B The input vector 405 may be sent to a stride-aware input mapping circuit 410 .

[0039] In one embodiment, stride-aware input mapping circuit 410 may map input vector 405 to output vector 408, e.g., each data item in input vector 405 is selectively mapped to a specific location in output vector 408, such that the mapped data item is passed to a specific weight register in one of MAC units 421 to MAC 424. MAC units 421 to MAC 424 may load weight vector 420 (e.g., with FIG. 2A to FIG. 2B The corresponding MAC unit can then perform the product of the mapped data item and the specific weight stored in the specific weight register to calculate the convolution operation, such as FIG. 2A to FIG. 2B shown.

[0040] It should be noted that the circuit 400 includes four MAC units 421 to 424, corresponding to FIG. 2A to FIG. 2BThe convolution between the 6×6 input matrix 102 and the 3×3 filter matrix 103 shown in FIG. 4 is for illustration purposes only. In other examples, the circuit 400 may include other numbers of MAC units, depending on the size of the input matrix and / or the filter matrix.

[0041] Figure 5 A method for selectively inputting a signal using a shift register according to one or more embodiments described herein is shown. Figure 4 An example structure of the stride-aware input mapping circuit 410 is shown. In one embodiment, the stride-aware input mapping circuit 410 may include an input register 510 and a stride matrix structure 520 that is communicatively connected to a plurality of MAC units 421 to 424 placed in parallel.

[0042] In one embodiment, input register 510 may be from Figure 4 The macro external memory 402 is shown to load the input vector 405 (e.g., 432 bits) and then selectively input at least a first portion 514 (e.g., 216 bits) of the input vector into the stride matrix 520. The input register 510 can be controlled by a clock signal 511, a reset signal 512, and a mode signal 513. For example, the mode signal 513 can control the input register 510 to load the portion 514 of the input vector that is sent to the stride matrix 520, for example, the first 216 bits of the 432-bit input vector 405, and then the second 216 bits of the bits in the 432-bit input vector 405 can be left shifted and loaded into the stride matrix 520. At each iteration, the input register is left shifted to "pop" the input parameter. Other details are as follows FIG. 8A to FIG. 8B and 9A to Fig. 9B shown.

[0043] In one embodiment, the stride matrix structure 520 can be implemented to map selected input parameters 514 (e.g., portions from the input vector 405) to their corresponding MAC units. The stride matrix structure 520 can perform input mapping based on the stride mode signal 515. For example, the stride mode signal 515 can include 2 bits, e.g., taking values ​​from {00, 01, 10, 11}, to select which of four stride matrix mappings, e.g., stride = 1 and angular convolution, stride = 1 and non-angular convolution, stride = 2 and angular convolution, stride = 2 and non-angular convolution.

[0044] In one embodiment, the stride matrix is ​​designed to reuse data items so that there may not be a 1-to-1 mapping between input 514 and output 408a to 408d. Figure 5 In the example shown, 216 input bits 514 may be mapped to 72 x 4 = 288 bits 408a to 408d.

[0045] Figure 6 615 is shown to selectively input a Figure 4 1. An example structure of the stride-aware input mapping circuit 410 is shown. In one embodiment, an input multiplexer 615 is placed between the input register 510 and the stride matrix 520 to selectively input portions of the input vector to the stride matrix based on the control signal 513. For example, the input multiplexer 615 can be a 4-to-1 multiplexer that transmits one of the input register groups, such as 216 bits [431:216], 144 bits [335-192], 216 bits [239:24], 144 bits [143:0], to the stride matrix 520. Additional examples of the input multiplexer 615 that selects the data items to be input to the stride matrix can be combined FIG. 12A to FIG. 12C Provide explanation.

[0046] In one embodiment, control signal 513 may be a 2-bit signal, such as 00, 01, 10, 11, for selecting which of the four input register groups is to be sent to stride matrix 520. For example, control signal 513 may be generated by a processor (e.g., Fig.14 Processor 1410 in is generated according to the state of the current convolution mode.

[0047] Figure 7 The present invention shows a method according to one or more embodiments described herein. Figure 5 to Figure 6 The example circuit structure of the stride matrix 520 is shown. It should be noted that Figure 7 Shows the use of Figure 5 The example circuit structure of the stride matrix 520 of the shift-type input register 510 is shown, but it is for illustration purposes only. The input multiplexer 615 can be added between the input register 510 and the stride matrix 520, as described in detail in relation to FIG. Figure 6 discussed.

[0048] In one embodiment, the stride matrix 520 may include a plurality of multiplexers 701 to 703. For example, each multiplexer may be a 4-to-1 multiplexer that selects which input data should be passed from a register to a particular weight register at a particular MAC unit. The selection may be controlled by a mode stride control signal 515 that indicates which of the four convolution modes is being implemented: stride = 1 with angular convolution, stride = 1 with non-angular convolution, stride = 2 with angular convolution, stride = 2 with non-angular convolution.

[0049] For example, for multiplexer 701, the selected output is connected to the MAC0_A input at the MAC unit. The connections between multiplexers 701 to 703 and different inputs at different MAC units can be designed based on a mapping matrix, such as FIG. 11A to FIG. 11BFurther described in .

[0050] FIG. 8A to FIG. 8B An example stride matrix for a stride-1 convolution according to one or more embodiments described herein is shown. Fig. 8A As shown, diagram 801 shows an example 9×9 input data array that will be convolved with a 3×3 filter matrix. When the convolution moves with a stride=1, non-repeating data items are expanded from the 9×9 output data array to form an input vector 405, which is loaded in the input register 510.

[0051] In one embodiment, a stride matrix 802 for a stride-1 angular convolution for the first iteration is shown. In the stride matrix 802, each "x" mark in the stride-1 angular matrix represents a connection from an input register to a MAC weight register to perform a multiplication of a data item and a corresponding weight. For example, IN_REG[a,b] represents a data item in the input register corresponding to a data item on the ath row and bth column in the input data array. The first row of the stride matrix 802 connects the first data item IN_REG[1,1] of the input register to the MAC register MAC_0[A] to perform IN_REG[1,1]×MAC_0[A]; perform IN_REG[1,2]×MAC_0[B] and IN_REG[1,2]×MAC_1[A], etc.

[0052] like Figure 8B As shown, in the second iteration, the 12 input parameters that have been convolved in the first iteration can be "popped out", for example, the input vector can be shifted left so that position 405a can be moved to the beginning of the vector. Figure 803 shows the non-repeating data items expanded from the 9×9 input data array for the second iteration. In the second iteration of the regular non-angular convolution, the stride matrix 804 can be used to map the input registers to the MAC weight registers to perform the product of the data items and the corresponding weights. For example, the first row of the stride matrix 802 maps the first data item of the input register, IN_REG[1,1] to the MAC register MAC_0[A] to perform IN_REG[1,1]×MAC_0[A]; maps IN_REG[1,2] to the MAC register MAC_0[D] to perform IN_REG[1,2]×MAC_0[D], etc.

[0053] Notice, FIG. 8A to FIG. 8B For illustration purposes only. To complete the convolution, the input register 510 may continue to left-shift and pop the input parameters in one or more subsequent convolution iterations and map the current input parameters to different MAC registers using the stride matrix 802 or 804 depending on the convolution mode.

[0054] FIG. 9A to FIG. 9BAn example stride matrix for a stride-2 convolution according to one or more embodiments described herein is shown. Fig.9A As shown in FIG901 , when the convolution moves with stride=2, non-repeating data items are expanded from the 9×9 input data array to form an input vector 405 , which is loaded at the input register 510 .

[0055] In one embodiment, a stride matrix 902 for a stride-2 angular convolution at the first iteration is shown. In stride matrix 902, each "x" mark in the stride-2 angular matrix represents a connection from an input register to a MAC weight register to perform a multiplication of a data item and a corresponding weight. For example, the first row of stride matrix 902 maps the first data item of the input register IN_REG[1,1] to the MAC register MAC_0[A] to perform IN_REG[1,1]×MAC_0[A]; IN_REG[1,2] is mapped to the MAC register MAC_0[B] to perform IN_REG[1,2]×MAC_0[B], etc. It should be noted that the upper left corner of the stride-1 angular matrix 802 can be similar to the upper left corner of the stride-2 angular matrix 902.

[0056] like Fig. 9B As shown, in the second iteration, the 24 input parameters that have been convolved in the first iteration can be "popped out", for example, the input vector can be shifted left so that position 905a can be moved to the beginning of vector 905b. In the second iteration of the regular non-angular convolution with stride = 2, the stride matrix 904 can be used to map the input registers to the MAC weight registers to perform the product of the data items and the corresponding weights. For example, the first row of the stride matrix 802 connects the first data item IN_REG[1,1] of the input register to the MAC register MAC_0[A] to perform IN_REG[1,1]×MAC_0[A]; connects IN_REG[1,2] to the MAC register MAC_0[D] to perform IN_REG[1,2]×MAC_0[D], etc.

[0057] Notice, FIG. 9A to FIG. 9B For illustration purposes only. To complete the convolution, the input register 510 may continue to left shift and pop the input parameters in one or more subsequent convolution iterations and map the current input parameters to different MAC registers using the stride matrix 902 or 904 depending on the convolution mode.

[0058] FIG. 10A to FIG. 10C shows a combination according to one or more embodiments described herein FIG. 8A to FIG. 9B The stride matrix shown is an example stride matrix of stride-1 and stride-2 for the stride matrix design. Fig. 10A As shown, Fig. 8A The stride-1 angle matrix 802 and Fig.9A The stride-2 angle matrices 902 shown may be superimposed to form a stride angle matrix 1002, which may be used to map input parameters from input registers according to a control signal indicating a stride mode (1 or 2).

[0059] For example, as shown in table 1004, the "x" entries in the stride matrix 1002 map the input parameters to the corresponding MAC registers for stride-1 and stride-2 angular convolutions. The "1" entries in the stride matrix 1002 map the input parameters to the corresponding MAC registers for stride-1 angular convolutions only. The "2" entries in the stride matrix 1002 map the input parameters to the corresponding MAC registers for stride-2 angular convolutions only.

[0060] like Fig. 10B As shown, Figure 8B The stride-1 regular (non-angular) matrix 804 and Fig. 9B The stride-2 regular (non-angular) matrix 904 shown can be further superimposed on top of the stride matrix 1002 to form a stride matrix 1006, which can be used to map input parameters from input registers based on control signals indicating the stride mode (1 or 2) and the convolution mode (regular or angular).

[0061] For example, Fig. 10C As shown in Table 1008 in FIG. 1006 , different numbers “0” to “10” in stride matrix 1006 indicate whether each entry is applied to one or more of stride-1 angular convolution, stride-2 angular convolution, stride-1 regular convolution, or stride-2 regular convolution. For example, when a control signal (e.g., Figure 5 to Figure 6 515) indicates that when the current iteration is for stride-1 angular convolution, based on the stride matrix 1006, IN_REG[1,1] is mapped to MAC_0[A] because in table 1008, the "0" entry is applicable to any of the four convolution modes. Another example is that IN_REG[1,2] is mapped to MAC_0[B] because in table 1008, the "5" entry is applicable to stride-1 angle or stride-2 angle, but the "10" entry is not applicable to stride-1 angle, so under stride-1 angular convolution, only the "5" entry connects IN_REG[1,2] to MAC_0[B].

[0062] FIG. 11A to FIG. 11B An illustrative example is provided showing an embodiment according to the present invention. Fig. 10B The hardware implementation of the superposition stride matrix 1006 is shown in FIG. Figure 7 As shown, the stride matrix can be implemented by multiple multiplexers. In one embodiment, each row of the stride matrix 1006 can be implemented by a multiplexer, for example, a 4-to-1 multiplexer such as 1102 or 1103. The two-bit control signal (e.g., Figure 5 to Figure 6 515) can take the values ​​"00" = stride -1 angle, "01" = stride -2 angle, "10" = stride -1 rule and "11" = stride -2 rule.

[0063] For example, multiplexer 1102 represents mapping the data items in the input register to the first row of MAC_0[A]. In stride matrix 1006, In_REG[1,1] is mapped to MAC_0[A] in any of the four convolution modes according to the "0" entry defined in table 1008. Therefore, the four inputs of multiplexer 1102 are connected to IN_REG[1,1], so that the output of multiplexer 1102 is connected to MAC_0[A] regardless of the value of control signal 515.

[0064] For another example, multiplexer 1103 represents that the data items in the input register are mapped to the second row of MAC_0[B]. In stride matrix 1006, In_REG[1,2] under the "5" item and In_REG[2,1] under the "10" item are mapped to MAC_0[B] in the second row. In table 1008, item "5" is applicable to stride-1 angle and stride-2 angle, and item "10" is applicable to stride-1 rule and stride-2-rule. Therefore, IN_REG[1,2] is connected to the input of "00" (stride-1 angle) and "01" (stride-2 angle) of multiplexer 1103, and IN_REG[2,1] is connected to the input of "10" (stride-1 rule) and "11" (stride-2 rule) of multiplexer 1103. In this way, the output of multiplexer 1103 is connected to MAC_0[B], which selects from one of the four inputs according to control signal 515.

[0065] For another example, multiplexer 1104 indicates that the data items in the input register are mapped to row 10 of MAC_1[A]. In stride matrix 1006, IN_REG[1,2] under item "1", IN_REG[1,3] under item "2", IN_REG[2,1] under item "3", and IN_REG[3,1] under item "4" are mapped to MAC_1[A] according to the corresponding convolution mode in row 10. In table 1008, item "1" applies to stride-1 angle; item "2" applies to stride-2 angle; item "3" applies to stride-1 rule; item "4" applies to stride-2 rule. Thus, IN_REG[1,2] is connected to the "00" (stride-1 corner) input of multiplexer 1104; IN_REG[1,3] is connected to the "01" (stride-2 corner) input of multiplexer 1104; IN_REG[2,1] is connected to the "10" (stride-1 regular) input of multiplexer 1104; and IN_REG[3,1] is connected to the "11" (stride-2 regular) input of multiplexer 1104. In this way, multiplexer 1104 selects one of the four inputs based on control signal 515, and its output is connected to MAC_1[A].

[0066] For another example, multiplexer 1105 indicates that the data items in the input register are mapped to row 13 of MAC_1[D]. In stride matrix 1006, In_REG[2,2] under the "6" item, In_REG[2,3] under the "2" item, and In_REG[3,2] under the "4" item are mapped to MAC_1[D] in row 13. In table 1008, the "6" item applies to stride-1 angle and stride-1 rule, the "2" item applies only to stride-2 angle, and the "4" item applies only to stride-2 rule. Therefore, IN_REG[2,2] is connected to the "00" (stride-1 corner) and "10" (stride-1 regular) inputs of multiplexer 1105, and IN_REG[2,3] is connected to the "01" (stride-2 corner) input of multiplexer 1105 and IN_REG[3,2] is connected to the "11" (stride-2 regular) input of multiplexer 1105. In this way, multiplexer 1105 selects one of the four inputs according to control signal 515, and its output is connected to MAC_1[D].

[0067] like FIG. 10A to FIG. 10B and FIG. 11A to FIG. 11BAs shown, the stride matrix can be stored and / or implemented in various different embodiments. For example, in one implementation, different stride matrices can be stored separately for different strides and / or convolution modes (e.g., angular or regular), for example, a total of four different stride matrices can be stored and used separately, namely stride-1 angular 802, stride-1 regular 804, stride-2 angular 902, and stride-2 regular 904. In another implementation, the superimposed version 1002 of two matrices, for example, the stride-1 angular matrix 802 and the stride-2 angular matrix 902, can be superimposed and implemented together. In another implementation, the superimposed version 1006 of all four stride matrices can be stored and implemented together.

[0068] FIG. 12A to FIG. 12C The use of one or more embodiments described herein is shown Figure 6 615 in. In one embodiment, instead of a shift-type input register that is left-shifted to pop the input parameter after each convolution iteration, the input multiplexer 615 can be used to select the input parameter from the input register 510 to save dynamic power consumption.

[0069] like Fig. 12A As shown, mapping matrix 1202 shows that under stride-1 convolution, the input multiplexer can select one of the input data parameter groups 431:216, 335:192, 239:24 and 143:0 to output to stride matrix 520.

[0070] like Fig. 12B As shown, mapping matrix 1204 shows that under stride-2 convolution, the input multiplexer can select one of the input data parameter 431:216 and 239:24 groups to output to stride matrix 520.

[0071] like Fig. 12C As shown, the mapping matrix 1206 may be configured based on a control signal indicating a stride and a convolution mode (e.g., Figure 6 513 in the figure), mapping matrix 1202 and mapping matrix 1204 are combined to map the input data parameters. For example, when control signal 513 indicates "00" = stride-1 angle, input parameter 431:216 is selected to be output to the stride matrix; when control signal 513 indicates "01" = stride-2 angle, input parameter 335:192 is selected to be output to the stride matrix; when control signal 513 indicates "10" = stride-1 rule, input parameter 239:24 is output to the stride matrix; and when control signal 513 indicates "11" = stride-2 rule, input parameter 143:0 is output to the stride matrix.

[0072] Fig.13 is an example logic flow diagram according to an embodiment described herein, showing the use of Figure 1 to Figure 1 2 The circuit structure performs a process 1300 of convolving an input array with a filter matrix. One or more of the processes of method 1300 may be implemented at least in part in the form of executable code stored on a non-transitory, tangible, machine-readable medium, which executable code, when executed by one or more processors, may cause the one or more processors to perform multiple of the processes. In some embodiments, method 1300 corresponds to the operation of convolution module 1431, which uses Figures 4 to 6 One or more circuit structures discussed in the present invention are used to perform the convolution of the input data array with the filter matrix.

[0073] As shown, method 1300 includes a plurality of enumerated steps, but aspects of method 600 may include additional steps before, after, and between the enumerated steps. In some aspects, one or more of the enumerated steps may be omitted or performed in a different order. Note that while operations are described as performing one convolution iteration, etc., the steps described herein may be performed iteratively and / or in parallel for multiple convolution iterations.

[0074] At step 1302, an input register (eg Figure 5 510) can be obtained from an input data array (e.g. Figure 4 402 in the example to obtain an input vector of non-repeated data items (e.g. Figure 4 For example, Figure 3B As shown, non-repeating data items are expanded from the input data array based on the stride of the convolution.

[0075] At step 1304, a first input (eg, Figure 5 The stride matrix 520 in FIG. 5 may be obtained from an input register (eg, Figure 5 510) receives an input vector (eg, Figure 5 432-bit input vector 405) in the first data item (e.g., Figure 5 data items of the input vector 514 of at least 216 bits).

[0076] In one embodiment, the input register may be used as a shift-type operation: outputting at least a first data item of the input vector to a first output in a current iteration, then shifting the input vector left by a number of units, and outputting at least a second data item of the shifted input vector to the input mapping circuit in a next iteration. FIG. 8A to FIG. 9B shown.

[0077] In one embodiment, an input multiplexer (e.g., Figure 6 615) can be obtained from an input register (e.g., Figure 6The selection of the input multiplexer is as follows: FIG. 12A to FIG. 12C shown.

[0078] At step 1306, the input mapping circuit (e.g., Figure 5 The stride matrix 520 in FIG. 5 may be configured within the input mapping circuit based on a control signal indicating the stride of the convolution (eg, Figure 5 ), selectively sending the first data item to a first computing unit (e.g., Figure 5 The first weight register (such as Figure 5 The first output of any input of MAC units 421 to 424 of MAC_0[A] in Figure 5 any one of 408a to 408d).

[0079] For example, the convolution step size and the angular mode or non-angular mode of the convolution may be determined based on one or more control signals (e.g., Figure 6 The matrix structure (eg, see stride matrices 802, 804, 902, and 904) is implemented by an input mapping circuit that selectively maps a set of inputs to a set of outputs.

[0080] In one embodiment, a matrix structure (e.g., Fig. 10A The stride matrix 1002 in FIG. 1 may be a superposition of a first matrix structure corresponding to the first stride and angle pattern (e.g., stride matrix 802) and a second matrix structure corresponding to the second stride and angle pattern (e.g., stride matrix 902). The matrix structure (e.g., stride matrix 1006) may also be a superposition of a first matrix structure corresponding to the first stride and non-angle pattern (e.g., stride matrix 804) and a second matrix structure corresponding to the second stride and non-angle pattern (e.g., stride matrix 904). The matrix structure is composed of a plurality of multiplexers (e.g., FIG. 11A to FIG. 11B 1102 to 1104) in, where each multiplexer corresponds to a row of the matrix structure.

[0081] At step 1308, the first data item (eg, Fig. 10B The convolution is performed by performing a multiplication operation on IN_REG[1,1] in MAC_0 and the first weight corresponding to the first weight register (e.g., MAC_0[a]).

[0082] The method 1300 may be iterated for different iterations of the convolution.

[0083] Fig.14 According to one embodiment described herein, using Figures 1 to 13 A simplified diagram of a computing device that implements a convolution operation using the input mapping circuit described in . Fig.14 As shown in FIG. 1A , computing device 1400 includes a processor 1410 coupled to a memory 1420. The operation of computing device 1400 is controlled by processor 1410. Although computing device 1400 is shown as having only one processor 1410, it should be understood that processor 1410 may represent one or more central processing units, multi-core processors, microprocessors, microcontrollers, digital signal processors, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), graphics processing units (GPUs), etc. in computing device 1400. Computing device 1400 may be implemented as an independent subsystem, a board added to a computing device, and / or a virtual machine.

[0084] The memory 1420 may be used to store software executed by the computing device 1400 and / or one or more data structures used during operation of the computing device 1400. The memory 1420 may include one or more types of machine-readable media. Some common forms of machine-readable media may include floppy disks, diskettes, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, any other optical media, punch cards, paper tapes, any other physical media with patterns of holes, RAM, PROMs, EPROMs, FLASH-EPROMs, any other memory chips or cartridges, and / or any other media suitable for reading by a processor or computer.

[0085] The processor 1410 and / or the memory 1420 may be arranged in any suitable physical arrangement. In some embodiments, the processor 1410 and / or the memory 1420 may be implemented on the same board, in the same package (e.g., a system in a package), on the same chip (e.g., a system on a chip), etc. In some embodiments, the processor 1410 and / or the memory 1420 may include distributed, virtualized, and / or containerized computing resources. Consistent with such embodiments, the processor 1410 and / or the memory 1420 may be located in one or more data centers and / or cloud computing facilities.

[0086] In some instances, memory 1420 may include a non-transitory, tangible, machine-readable medium that includes executable code that, when executed by one or more processors (e.g., processor 1410), causes the one or more processors to perform methods described in further detail herein. For example, as shown, memory 1420 includes instructions for a convolutional neural network module 1430 that may be used to implement and / or simulate systems and models, and / or implement any of the methods described further herein. Convolutional neural network module 1430 may receive input 1440, such as an input data array for convolution, via data interface 1415, and generate output 1450, which may be the result of the convolution.

[0087] The data interface 1415 may include a communication interface, a user interface (e.g., a voice input interface, a graphical user interface, etc.). For example, the computing device 1400 may receive input 1440 (e.g., an input data array) from a networked database via a communication interface. Alternatively, the computing device 1400 may receive input 1440 from a user via a user interface, such as an input data array representing an input question.

[0088] In some embodiments, the convolutional neural network module 1430 is configured to calculate the convolution of the input data array and the filter matrix. The convolutional neural network model 1430 may also include a convolution submodule 1431 and a convolution mode submodule 1432. The convolution mode submodule 1432 may determine the current convolution stride and convolution mode, and provide a convolution mode for the input mapping circuit (e.g., Figure 6 510 in ) generates a control signal (e.g., in Figure 6 515 in FIG. 14 ). The convolution submodule 1431 may perform convolution of the input data item with the MAC unit (eg, Figure 4 Mapping and / or multiplication of weights in weight registers in 421 to 424).

[0089] Some examples of computing devices, such as computing device 1400, may include a non-transitory, tangible, machine-readable medium that includes executable code that, when executed by one or more processors (e.g., processor 1410), may cause the one or more processors to perform the processes of the method. Some common forms of machine-readable media that may include the process of the method are, for example, floppy disks, diskettes, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, any other optical media, punch cards, paper tapes, any other physical media with patterns of holes, RAM, PROMs, EPROMs, FLASH-EPROMs, any other memory chips or cartridges, and / or any other media that a processor or computer is suitable for reading.

[0090] The computing device 1400 may include a circuit for convolving input data and a weight matrix. The circuit includes an input register configured to store an input vector of non-repeating data items from an input data array; an input mapping circuit configured to receive a first data item from the input register and selectively send the first data item to a first output based on a control signal indicating a stride of the convolution. The first output is connected to a first weight register at a first computing unit, which performs a multiplication of the first data item with a first weight corresponding to the first weight register for convolution.

[0091] The computing device 1400 may be included in a system for convolving input data and a weight matrix. The system includes a memory storing a plurality of instructions, and one or more hardware processors executing the plurality of instructions to perform operations. The operations may include Fig.13 The method 1300 shown, for example: obtaining an input vector of non-repeating data items from an input data array at an input register; receiving a first data item of the input vector from the input register at a first pin of an input mapping circuit; selectively transmitting the first data item to a first output connected to a first weight register at a first computing unit within the input mapping circuit based on a control signal indicating a stride of the convolution; and performing a multiplication of the first data item with a first weight corresponding to the first weight register to perform a convolution.

[0092] An embodiment of the present invention provides a circuit for convolving input data and a weight matrix, comprising: an input register configured to store an input vector of non-repeating data items from an input data array; an input mapping circuit configured to receive a first data item from the input register and selectively send the first data item to a first output based on a control signal indicating the stride of the convolution, wherein the first output is connected to a first weight register at a first computing unit, and the first computing unit performs multiplication of the first data item with a first weight corresponding to the first weight register for the convolution.

[0093] In some embodiments, the input vector of non-repeating data items is obtained by unrolling the non-repeating data items from the input data array based on the stride of the convolution.

[0094] In some embodiments, the input register is configured to: output at least the first data item of the input vector to the first output in a current iteration; shift the input vector left by a number of units; and output at least a second data item of the shifted input vector to the input mapping circuit in a next iteration.

[0095] In some embodiments, the circuit further comprises: an input multiplexer that selects a group of data items from the input vector in the input register at a current iteration to output to the input mapping circuit.

[0096] In some embodiments, the input mapping circuit implements a matrix structure that selectively maps a set of inputs to a set of outputs, and wherein the matrix structure is selected based on one or more control signals indicating the stride of the convolution in a current iteration and an angular mode or non-angular mode of the convolution.

[0097] In some embodiments, the matrix structure is in the form of a superposition of a first matrix structure corresponding to the first step and angle pattern and a second matrix structure corresponding to the second step and angle pattern.

[0098] In some embodiments, the matrix structure is in the form of a superposition of a first matrix structure corresponding to the first step and non-angular pattern and a second matrix structure corresponding to the second step and non-angular pattern.

[0099] In some embodiments, the matrix structure is implemented by a plurality of multiplexers, and each multiplexer corresponds to a row of the matrix structure.

[0100] Another embodiment of the present invention provides a method for convolving input data and a weight matrix, comprising: obtaining, at an input register, an input vector of non-repeating data items from an input data array; receiving a first data item of the input vector from the input register at a first input of an input mapping circuit; selectively transmitting the first data item to a first output of a first weight register connected to a first computing unit within the input mapping circuit based on a control signal indicating a stride of the convolution; and performing multiplication of the first data item with a first weight corresponding to the first weight register for the convolution.

[0101] In some embodiments, obtaining the input vector comprises: unfolding the non-repeating data items from the input data array based on the stride of the convolution.

[0102] In some embodiments, the method further includes: in a current iteration, outputting at least the first data item of the input vector to the first output by the input register; shifting the input vector left by a number of units; and in a next iteration, outputting at least a second data item of the shifted input vector to the input mapping circuit by the input register.

[0103] In some embodiments, the method further comprises: selecting an input multiplexer that selects a set of data items from the input vector in the input register to output to the input mapping circuit at a current iteration.

[0104] In some embodiments, the method further includes: selecting a matrix structure based on one or more control signals indicating the stride of the convolution at the current iteration and one or more control signals indicating the angular mode or non-angular mode of the convolution, wherein the matrix structure is implemented by the input mapping circuit, and the input transformation circuit selectively maps a set of inputs to a set of outputs.

[0105] In some embodiments, the matrix structure is in the form of a superposition of a first matrix structure corresponding to the first step and angle pattern and a second matrix structure corresponding to the second step and angle pattern.

[0106] In some embodiments, the matrix structure is in the form of a superposition of a first matrix structure corresponding to the first step and non-angular pattern and a second matrix structure corresponding to the second step and non-angular pattern.

[0107] In some embodiments, the matrix structure is implemented by a plurality of multiplexers, and each multiplexer corresponds to a row of the matrix structure.

[0108] Another embodiment of the present invention provides a system for convolving input data and a weight matrix, comprising: a memory storing a plurality of instructions; one or more hardware processors executing the plurality of instructions to perform operations, the operations comprising: obtaining an input vector of non-repeating data items from an input data array at an input register; receiving a first data item of the input vector from the input register at a first pin of the input mapping circuit; selectively transmitting the first data item to a first output of a first weight register connected to a first computing unit within the input mapping circuit based on a control signal indicating the stride of the convolution; and performing multiplication of the first data item with a first weight corresponding to the first weight register for the convolution.

[0109] In some embodiments, the input vector of non-repeating data items is obtained by unrolling the non-repeating data items from the input data array based on the stride of the convolution.

[0110] In some embodiments, the operation further includes: selecting a matrix structure based on one or more control signals indicating the stride of the convolution and an angular mode or a non-angular mode of the convolution at a current iteration, wherein the matrix structure is implemented by the input mapping circuit, and the input mapping circuit selectively maps a set of inputs to a set of outputs.

[0111] In some embodiments, the matrix structure is in the form of a superposition of a first matrix structure corresponding to the first step and angle pattern and a second matrix structure corresponding to the second step and angle pattern.

[0112] The foregoing summarizes the features of several embodiments so that those skilled in the art can better understand aspects of the present disclosure. Those skilled in the art should understand that they can easily use the present disclosure as a basis for designing or modifying other processes and structures to achieve the same purposes and / or achieve the same advantages as the embodiments introduced herein. Those skilled in the art should also recognize that such equivalent constructions do not depart from the spirit and scope of the present disclosure, and that they can be subjected to various changes, substitutions and modifications without departing from the spirit and scope of the present disclosure.

Claims

1. A circuit for convolving input data and a weight matrix, comprising: an input register configured to store an input vector of non-repeating data items from an input data array; an input mapping circuit configured to receive a first data item from the input register and selectively send the first data item to a first output based on a control signal indicating a stride of the convolution, Wherein, the first output is connected to a first weight register at a first computing unit, and the first computing unit performs multiplication of the first data item with a first weight corresponding to the first weight register for the convolution.

2. The circuit according to claim 1, wherein An input vector of the non-repeated data items is obtained by unrolling the non-repeated data items from the input data array based on the stride of the convolution.

3. The circuit according to claim 1, wherein: The input registers are configured as: outputting at least the first data item of the input vector to the first output in a current iteration; Shift the input vector left by a certain number of units; as well as In a next iteration, at least a second data item of the shifted input vector is output to the input mapping circuit.

4. The circuit according to claim 1, further comprising: An input multiplexer selects a group of data items from the input vector in the input register to output to the input mapping circuit at a current iteration.

5. The circuit according to claim 1, wherein: The input mapping circuit implements a matrix structure that selectively maps a set of inputs to a set of outputs, and Therein, the matrix structure is selected based on one or more control signals indicating the stride of the convolution in a current iteration and an angular mode or a non-angular mode of the convolution.

6. A method for convolving input data and a weight matrix, comprising: At the input register, an input vector of non-repeating data items is obtained from the input data array; receiving a first data item of the input vector from the input register at a first input of an input mapping circuit; selectively transmitting the first data item to a first output of a first weight register connected to a first computation unit within the input mapping circuit based on a control signal indicating a stride of the convolution; as well as Multiplication of the first data item with a first weight corresponding to the first weight register is performed for the convolution.

7. The method according to claim 6, wherein: Obtaining the input vector comprises: The non-repeating data items are unrolled from the input data array based on the stride of the convolution.

8. The method according to claim 6, further comprising: In a current iteration, outputting at least the first data item of the input vector to the first output by the input register; Shift the input vector left by a number of units; as well as In a next iteration, at least a second data item of the shifted input vector is output by the input register to the input mapping circuit.

9. A system for convolving input data and a weight matrix, comprising: A memory storing a plurality of instructions; One or more hardware processors execute the plurality of instructions to perform operations, the operations comprising: At the input register, an input vector of non-repeating data items is obtained from the input data array; receiving a first data item of the input vector from the input register at a first pin of the input mapping circuit; selectively transmitting the first data item to a first output of a first weight register connected to a first computation unit within the input mapping circuit based on a control signal indicating the stride of the convolution; and Multiplication of the first data item with a first weight corresponding to the first weight register is performed for the convolution.

10. The system according to claim 9, wherein: An input vector of the non-repeated data items is obtained by unrolling the non-repeated data items from the input data array based on the stride of the convolution.

Images