Neural processing filter
By defining filters in the neural processing unit as a combination of multiple sub-filters, the problem of filter size limitation in the prior art is solved, achieving more efficient processing power and resource utilization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ARM LTD
- Filing Date
- 2021-03-01
- Publication Date
- 2026-07-24
Smart Images
Figure CN113469347B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to a computer-implemented method, a non-transitory computer-readable storage medium, and a system for providing filters in a neural processing unit. Background Technology
[0002] Neural processing systems may employ neural processing units (NPUs). NPUs (also known as neural processors, neural network accelerators, and AI accelerators) are used to accelerate machine learning algorithms. By including specialized electronic circuitry designed for one or more specified neural network architectures, NPUs process input data, such as image data, more efficiently than general-purpose processors.
[0003] A specific type of neural network is called a Convolutional Neural Network (CNN), and it is commonly used for image classification. A CNN consists of one or more convolutional layers, each of which takes an input feature map and convolves it with a filter (also called a kernel) to produce an output feature map. The purpose of convolution is to extract high-level features from the input feature map, such as edges in an image. The filters in a convolutional layer consist of a set of weights that are multiplied by the values in the input feature map during the convolution operation to produce the values in the output feature map. CNNs also contain other layer types, such as input, output, pooling, and fully connected layers.
[0004] Other types of neural networks also employ filters. Recurrent Neural Networks (RNNs), used to model time-varying behavior, consist of a network of neuron-like nodes arranged in successive layers. Input nodes receive data from outside the network, output nodes produce results, and hidden nodes modify the data between inputs and outputs. Each node in a layer is connected via unidirectional connections to every other node in the next successive layer. Each node has time-varying real-valued activations. Each connection has modifiable real-valued weights. These weights in an RNN can also be represented by filters.
[0005] Therefore, filters are a key element of neural networks and typically consume a lot of processing power. Summary of the Invention
[0006] This disclosure relates to a computer-implemented method for providing filters in a neural processing unit. According to a first aspect of this disclosure, the method includes:
[0007] - Receive the input corresponding to the target dimension of the filter;
[0008] - Receives input corresponding to the sub-filter dimension of each of a plurality of sub-filters that can be implemented in the neural processing unit; and
[0009] - Define a filter as a combination of multiple sub-filters, the combination having a dimension equal to the target dimension, and wherein at least two of the sub-filters in the combination have different sub-filter dimensions.
[0010] According to a second aspect of this disclosure, the sub-filter dimension of each of the plurality of sub-filters includes an X-dimensional parameter representing the number of columns of the sub-filter and a Y-dimensional parameter representing the number of rows of the sub-filter, and wherein the size of each sub-filter is defined by the product of the X-dimensional parameter and the corresponding Y-dimensional parameter; and wherein defining a filter as a combination of the plurality of sub-filters includes successively including the sub-filters in the combination until the target dimension of the filter is filled by the combination with no remaining space, each included sub-filter having the maximum possible sub-filter size suitable for the remaining space.
[0011] According to a third aspect of this disclosure, a non-transitory computer-readable storage medium is provided.
[0012] According to a fourth aspect of this disclosure, a system is provided.
[0013] The functions disclosed regarding the computer-implemented method can also be implemented in the non-transitory computer-readable storage medium and the system.
[0014] Further aspects, features, and advantages will become apparent from the following description of preferred embodiments given by way of example only, with reference to the accompanying drawings. Attached Figure Description
[0015] Figure 1 illustrates the filter F in the convolutional layer CL1. 1..k Existing techniques such as CNNs apply the input feature map IFM1 to generate the output feature map OFM1.
[0016] Figure 2 shows a prior art neural processing unit in which CNNs can be implemented.
[0017] Figures 3A-3E Some existing techniques for providing a 10×6 filter in an NPU that is only capable of providing 4×4, 4×1, 1×4 and 1×1 filters are shown.
[0018] Figures 4A-4H An embodiment of a method for providing a filter F in a neural processing unit according to some aspects of this disclosure is shown.
[0019] Figure 5 A flowchart is shown of a method for providing a filter F in a neural processing unit according to some aspects of this disclosure.
[0020] Figures 6A-6D An implementation of an extended filter according to some aspects of this disclosure is shown.
[0021] Figure 7 A flowchart illustrating the use of filters in a CNN according to some aspects of this disclosure is shown.
[0022] Figure 8 The system SY, which includes the processor PROC and the memory MEM, is shown. Detailed Implementation
[0023] Schemes of this application are provided with reference to the following description and accompanying drawings. In this description, for illustrative purposes, numerous specific details of certain examples are set forth. References to “example,” “scheme,” or similar language in the specification mean that a feature, structure, or characteristic described in connection with that example is included at least in that example. It should also be understood that a feature described with respect to one example or embodiment may also be used in another example or embodiment, and for the sake of brevity, not all features need to be repeated.
[0024] This disclosure relates to an implementation of filters in the context of convolutional layers in a CNN. However, it should be understood that filters can be used in other types of layers and other types of neural networks, such as RNNs.
[0025] Additionally, Figure 1 shows the filter F in the convolutional layer CL1. 1..kA prior art CNN is applied to an input feature map IFM1 to generate an output feature map OFM1. The CNN of Figure 1 can be used, for example, to classify an image provided to the CNN as the input feature map IFM1. The example CNN of Figure 1 includes two convolutional layers CL1 and CL2. Each convolutional layer includes filters F1 and F2, which are convolved with the input feature map to generate the output feature map. The input feature map is provided in the form of an array and can be described as having certain dimensions; specifically, one or more of the following: a column dimension equal to the number of columns in the input feature map, a row dimension equal to the number of rows in the input feature map, and a depth dimension. Using the example where the input feature map IFM1 is an image, the column and row dimensions of the array of input feature map IFM1 can identify locations in the image, such as pixels, and the depth dimension can correspond to a specific color channel (such as red, green, and blue) in the image represented by the input feature map IFM1. In this example, the values of the array elements correspond to the intensity of each of the red, green, and blue channels of each pixel represented by the image in the input feature map IFM1. Although an image is used as an example of the input feature map IFM1 in Figure 1, it should be noted that CNNs can be used to model other types of data, including text and speech, and also for classification or post-processing. Like the input feature map IFM1, each of the filters F1 and F2 in Figure 1 can also be described as an array with certain dimensions: again, these dimensions can include one or more of the following: a column dimension X equal to the number of columns in the filter, a row dimension Y equal to the number of rows in the filter, and a depth dimension Z. The example filter F1 in Figure 1 has X columns and Y rows and a depth of Z = 5, as indicated by the depth of 5 in its final output feature map OFM1. However, for simplicity, only the first layer of filter F1 with Z = 1 is shown.
[0026] Referring again to Figure 1, the convolution step in convolutional layer CL1 involves stepping filter F1 over the input feature map IFM1, and at each step: multiplying each element or "weight" in filter F1 by its corresponding element in the input feature map IFM1, and summing the results over all X and Y positions in filter F1. The summation of all X and Y positions in the filter provides the data value of the output feature map OFM1 at that step. The stride is determined by the filter "step size". After stepping the filter by the stride size, the summation is again calculated for all X and Y positions in the filter at the new position, and this provides the data value of the output feature map OFM1 at that step. This convolution operation is performed in the same way for each step in the depth dimension Z of filter F1, and therefore the output feature map OFM1 has a depth dimension equal to the depth dimension Z of filter F1.
[0027] Referring further to convolutional layer CL1 in Figure 1, the output feature map OFM1 is then subjected to pooling or "subsampling" operations, where the values of group P1 of elements in the output feature map OFM1 are reduced to individual values in the output array. This pooling or subsampling operation may, for example, involve replacing the individual values in group P1 with their average or their maximum value. Pooling operations are typically used to reduce the dimension of the array input to the next convolutional layer (CL2 in this case). The output of convolutional layer CL1 is then used as the input feature map for convolutional layer CL2.
[0028] Similar operations to those described in the reference convolutional layer CL1 are also performed in convolutional layer CL2. Specifically, convolution is performed by filter F2, and pooling is performed on the result of the convolution using group P2.
[0029] The output of convolutional layer CL2 in Figure 1 is then provided as input to the classifier layer. The classifier layer may include one or more fully connected layers, and it takes the output of convolutional layer CL2 and provides an N-dimensional vector as output, where N is the number of categories that the program must select to classify the input image.
[0030] During training, the weights of filters F1 and F2 in Figure 1 are adjusted to optimize their values using a mechanism called backpropagation. The filter weights are adjusted so that the CNN can classify various different images belonging to the same category, where the probability of them having the same category is high. Once this is done, the filters with optimized weights can then be convolved with the news images to determine which category each image belongs to.
[0031] As can be understood from the above description of CNNs, convolution places a heavy processing burden on the processor. In fact, a paper titled "A Scalable Multi-TeraOPS DeepLearning Processor Core for AI Training and Inference" presented by B. Fleischer et al. at the 2018 VLSI Symposium reported that convolutions account for 70% to 90% of the processing in ordinary neural networks. Therefore, any improvement in the efficiency of the convolution process will have a significant impact on CNNs; and similarly, on other neural networks that use filters.
[0032] Figure 2 illustrates a prior art neural processing unit in which a CNN can be implemented. Referring to Figure 2, in this example, sixteen computation engines can be used to perform up to sixteen simultaneous convolutions. The unit—interface, DMA engine, control unit, and synchronization unit—manages the data flow to and from the sixteen computation engines. Each computation engine contains a MAC engine that performs the actual "multiplication and accumulation" portion of each convolution. Each computation engine also includes memory in the form of static RAM units (SRAM). During operation, the input feature map is stored in the SRAM and routed to the MAC engine in one of the computation engines via the DMA unit, control unit, and synchronization unit. The synchronization unit also determines whether one or both of the input feature map and the filter need to be subdivided into smaller and therefore more tractable blocks based on the size of the input feature map and the filter. The unit SRAM also stores the filter weights of the filters used in the MAC engine, and these are received by the MAC engine via a weight decoder. Therefore, the MAC convolution engine convolves the input feature map with the filter, and the resulting output feature map is passed from the MAC engine to the SRAM via a programmable layer engine.
[0033] Depending on the nature of the network, the dimensions of filters used in convolutional layers can vary greatly. However, known neural processing units, such as those shown in Figure 2, are typically only able to implement a limited number of filter sizes naturally. For example, a particular neural processing unit might only provide filters with the following dimensions (expressed as column number × row number): 4x4, 4x1, 1x4, or 1x1. The filter size is fixed in the hardware by the integrated circuit designer when fabricating the neural processing unit. When implementing a neural network on a neural processing unit, the programmer should consider the size and dimension of the available filters for that neural network, and how to synthesize and modify the neural network filters to fit within the filter size supported by the neural processing unit. If the desired dimension of the filter (i.e., the target filter dimension) exceeds the dimension available from any filter in the neural processing unit, the programmer instructs the neural processing unit to use multiple identical “sub-filters” to provide the desired filter dimension. If only sub-filters of the same size are selected in this process, the resulting filter typically has a size exceeding the desired filter dimension. Zero values, or “weight padding,” are typically used for filter array elements located outside the desired filter dimension. Filters are said to be “padded,” and this leads to low processing efficiency.
[0034] To further illustrate this point, Figures 3A-3E Some existing techniques for providing a 10×6 filter in an NPU that can only provide 4×4, 4×1, 1×4, or 1×1 filters are shown. Figure 3A In the context, the desired or “target” filter has a target X-dimensional parameter X representing the number of columns of the target filter. Tand the target Y-dimensional parameter Y representing the number of rows of the target filter T The size of the target filter is determined by X-dimensional parameters X. T and Y-dimensional parameters Y T The product definition is used. Therefore, the target filter shown has dimensions of 10×6 and 60. Figures 3A-3E In the example, a specific neural processing unit only provides filters with the following dimensions (referred to as "sub-filters"): 4×4, 4×1, 1×4, or 1×1. Using existing technologies, users of neural processing units can provide, for example... Figure 3B The filter shown is an example using 60 sub-filters, each with a dimension of 1×1. This would provide exactly the desired target filter dimension, but would be quite slow due to the inefficiency of processing only a single 1×1 filter at a time when the hardware supports up to 4×4 filter elements, and the inefficiency of coordinating the data movement involved in processing the results from 60 individual sub-filters. Alternatively, the user can provide the target filter using six sub-filters, each with a dimension of 4×4. Since only six sub-filters are used, this involves less coordination of data movement. However, the resulting filter is larger than the target dimension indicated by shading "filling" at the bottom and right edges of the filter. Typically, zero values are inserted into these shading locations in the filter, as described above, and the computational sum of the filled region is zero, i.e., it is a waste of computation. Figure 3D and Figure 3E The solution also uses multiple identical sub-filters and includes filter padding. Filter padding can be undesirable because it results in outcomes that do not affect the elements of the output feature map and thus wastes processing resources to varying degrees, depending on how much padding is generated in each case.
[0035] According to this disclosure, a computer-implemented method for providing a filter F in a neural processing unit includes:
[0036] - Target dimension X of receiver and filter T , Y T Corresponding input;
[0037] - Receiver and multiple sub-filters SF that can be implemented in the neural processing unit 1..n The sub-filter dimension X of each of them 1..n ', Y 1..n 'Corresponding input;
[0038] - Define filter F as multiple sub-filters SF 1..n A combination of elements that has the same dimensions as the target dimension X. T ,Y T The dimension of X, and wherein at least two of the sub-filters in the combination have a sub-filter dimension X.1..n ', Y 1..n They are not equal.
[0039] As described in more detail below, this method can be used to provide filters in neural processing units such as those described with reference to Figure 2. This method can be used to provide filters for neural network layers (such as convolutional layers). This method can also be used to provide filters in other types of neural networks (such as RNNs). Advantageously, the disclosed filter implementation allows for more efficient use of processing power in the neural processing unit because its sub-filters are equivalent in size to the target filter. Therefore, filter padding and unnecessary processing of the input feature map by padding elements can be avoided. Furthermore, the need to cache the feature map content in the regions corresponding to the weight padding is eliminated.
[0040] In one example implementation, the method can be executed by a processor that controls the operation of multiple parallel processors within the neural processing unit. Referring to Figure 2, the method can be executed substantially by a synchronization unit. Alternatively, the method can be executed by another processor within the neural processing unit, or actually elsewhere within the neural processing unit of Figure 2. Other aspects of the method can also be executed elsewhere within the neural processing unit of Figure 2. Alternatively, the method can be executed by a general-purpose processor, wherein the method defines sub-filters of filters that are subsequently implemented in possibly separate neural processing units.
[0041] Figures 4A-4H An embodiment of a method for providing a filter F in a neural processing unit according to some aspects of this disclosure is shown. Figure 4A In the diagram, the desired or target filter is represented as a target X-dimensional parameter X with the number of columns representing the target filter. T and the target Y-dimensional parameter Y representing the number of rows of the target filter T The target filter shown has dimensions of 10×6 and 60. Figures 4A-4H In the example, a specific neural processing unit only provides sub-filters with the following dimensions X 1..n '×Y 1..n 'of n Several filters (called "sub-filters"): 4×4, 4×1, 1×4, 2×2, and 1×1, such as Figure 4H As shown. Therefore, the dimension of each sub-filter includes an X-dimensional parameter X representing the number of columns of the sub-filter. 1..n ' and the Y-dimensional parameter Y representing the number of rows of the sub-filter 1..n The size of each sub-filter can also be determined by X-dimensional parameters X. 1..n 'and the corresponding Y-dimensional parameter Y 1..n The definition of the product of '.
[0042] refer to Figure 4BProvide those dimensions X that are equivalent to the target filter. T , Y T One technique for combining filters of dimension F includes a single 4×4 sub-filter and forty-four sub-filters, each with a dimension of 1×1. It should be noted that the sub-filters in the combination have a dimension X of 1×1. 1..n ', Y 1..n 'Not equal.' This can help to provide exactly the desired target filter dimension, and in addition, it reduces the processing overhead of combining the results of individual convolutions through sub-filters compared to using 60 single sub-filters of size 1×1. Figures 4C to 4G This shows those dimensions X used to provide a filter equivalent to the target filter. T , Y T And similarly subject to the following constraint, the alternative technique for filter F: at least two of the sub-filters in the combination have a sub-filter dimension X. 1..n ', Y 1..n They are not equal.
[0043] Figure 5 A flowchart illustrating a method for providing a filter F in a neural processing unit according to some aspects of this disclosure is shown. (Reference) Figure 5 The method includes receiving and filtering the target dimension X. T , Y T The corresponding input steps. For example, the target dimension can be received from memory. The memory may be within the neural processing unit. Optionally, the target dimension can be received in the form of user input. For example, these can be input by a programmer during the NPU setup phase. The method also includes the step of receiving multiple sub-filters SF that are implementable in the neural processing unit. 1..n The sub-filter dimension X of each of them 1..n ', Y 1..n The corresponding input. The sub-filter dimension X can be received from memory. 1..n ', Y 1..n For example, they can be stored in memory that stores NPU configuration data and forms part of the NPU. Subsequently, filter F is defined as multiple sub-filters SF. 1..n A combination of elements that has the same dimensions as the target dimension X. T , Y T The dimension of X, and wherein at least two of the sub-filters in the combination have a sub-filter dimension X. 1..n ', Y 1..n 'Not equal.' In some implementations, this step can be performed iteratively by including successive filters in the combination until a complete filter is defined, such as... Figure 5 The loop indicated in the text.
[0044] It can be defined in various combinations. Figure 5 The target filter dimension X in T , Y T The sub-filters. The general problem of rectangular subdivision is NP-complete and therefore computationally challenging. In some implementations, lookup tables can be used to provide a predetermined selection of sub-filters. The predetermined selection may have been optimized based on the performance characteristics of the filters (such as the total time spent generating the output feature map from the input feature map). This approach may be practical for commonly used target filter sizes. However, in general, it may be necessary to compute the filter dimensions for a large number of different filter sizes, sometimes in real-time. Here, a greedy approach can be used to provide a solution in a finite amount of time. In this approach, the target filter dimension is provided by iteratively filling the dimension of the target filter with sub-filters, selecting the largest possible sub-filter size that fits the remaining space at each iteration. This greedy approach for the rectangular subdivision problem has a polynomial time complexity, which allows us to address the computational intractability of the general rectangular subdivision problem. This also results in the target filter being defined by a smaller number of sub-filters, thus helping to reduce the processing overhead of determining the output feature map from the individual convolution results of multiple sub-filters.
[0045] In this method, multiple sub-filters SF 1..n The sub-filter dimension X of each of them 1..n ', Y 1..n 'Including X-dimensional parameters X representing the number of columns of the sub-filters' 1..n ' and the Y-dimensional parameter Y representing the number of rows of the sub-filter 1..n The size of each sub-filter is determined by X-dimensional parameters X. 1..n 'and the corresponding Y-dimensional parameter Y 1..n The product definition of '. Here, a filter is defined as multiple sub-filters SF. 1..n The combination includes sub-filters SF 1..n Successively included in the combination up to the target dimension X of filter F. T ,Y T Until the combined sub-filters are filled with no remaining space, each included sub-filter has the largest possible sub-filter size that fits the remaining space.
[0046] In the above method, which includes subfilters with the largest possible subfilter size suitable for the remaining space, if more than one subfilter satisfies the criterion, the subfilter with the dimension closest to a square shape can be optionally selected. This helps to minimize the total number of subfilters used in filter F and thus reduces the processing overhead of combining the results of individual convolutions by the subfilters. For example, a 4×1 subfilter and a 2×2 subfilter have the same subfilter size, and if both subfilters satisfy the criterion of the largest subfilter size suitable for the remaining space, the 2×2 subfilter is selected because its dimension is square. In another example, a 4×4 subfilter, a 2×8 subfilter, and a 1×16 subfilter all have the same size, and if all three subfilters satisfy the criterion of the largest subfilter size suitable for the remaining space, the 4×4 subfilter is selected because its dimension is square. This can be summarized as follows: Define a filter as multiple subfilters SF. 1..n Combinations may include sub-filters SF 1..n Successively included in the combination up to the target dimension X of filter F. T , Y T Until the combined sub-filters are filled with no remaining space, each included sub-filter has the largest possible sub-filter size to fit the remaining space, and if multiple sub-filters with the same sub-filter size fit the remaining space, then a sub-filter with the largest possible sub-filter size is selected from among the multiple sub-filters with the same sub-filter size. The sub-filter with the lowest value.
[0047] In some implementations, the dimensions of the target filter may optionally be filled in row-major order. In these implementations, the sub-filters SF... 1..n Successively included in the combination up to the target dimension X of filter F. T , Y T The steps until the filter is filled with combined data and there is no remaining space include filling the target dimension X of the filter in row-major order by performing the following operations on each of the multiple sets of rows of the filter F: T , Y T :
[0048] - Select an initial sub-filter SFo1 for this group of rows, which has the largest possible sub-filter size suitable for the remaining space;
[0049] - Define a set of rows within the sub-filter dimension as having the same Y-dimensional parameters Y as the initial sub-filter SFo1. 1..n A group of lines;
[0050] - Successively including sub-filters SF 1..nThe sub-filters are filled until there is no space left in the row, and each included sub-filter has the maximum possible sub-filter size to fit the remaining space in the row.
[0051] This reference Figure 4C This will be explained. Here, the 10×6 dimension of the example target filter is filled with an initial sub-filter SFo1 with a dimension of 4×4; this is from a 10×6 array. Figure 4H The largest sub-filter among those available sub-filters. Figures 4A-4H Sub-filters of the same size with dimensions 8×2 cannot be used, but if available, 4×4 sub-filters would still be preferred because, as mentioned above, their dimensions are closer to a square. Therefore, when selecting an initial sub-filter SFo1 for this set of rows, which has the largest possible sub-filter size suitable for the remaining space, this may optionally include: if multiple sub-filters of the same size fit the remaining space, then selecting from the multiple sub-filters of the same size... The sub-filter with the lowest value. Figure 4C The sub-filter SFo1 in the set has 4 rows, and the remaining space in the 4 rows of that set is then filled before filling the next set of rows. When filling the remaining space in the 4 rows of that set, it is also preferable to have a sub-filter with the largest possible sub-filter size that fits the remaining space in the rows of that set. Another optional selection criterion is that the included sub-filters have the same Y-dimensional parameter Y as the initial sub-filter SFo1. 1..n Therefore, it successively includes the sub-filter SF. 1..n The process continues until no space remains in the set of rows, with each included sub-filter having the largest possible sub-filter size to fit the remaining space in the set of rows. Optionally, if multiple sub-filters with the same sub-filter size fit the remaining space, a sub-filter with the same Y-dimensional parameter Y as the initial sub-filter SFo1 may be included. 1..n The sub-filter is obtained by maintaining the same Y-dimensional parameters Y as the initial sub-filter SFo1. 1..nThis avoids row splitting, thus simplifying data selection and movement. The 4×4 sub-filter is the largest sub-filter that fits the remaining space in the 4 rows of the group, and therefore a second 4×4 sub-filter is included in the combination. Furthermore, this sub-filter advantageously has a Y-dimensional parameter of 4, which is the same as the Y-dimensional parameter of the initial sub-filter SFo1. The remaining space in the 4 rows of the group has a dimension of 2×4, and a 1×4 sub-filter is included here, firstly because it is the largest sub-filter that fits the remaining space in the 4 rows of the group, and secondly because it has a Y-dimensional parameter of 4, which is the same as the Y-dimensional parameter of the initial sub-filter SFo1. The 1×4 sub-filter has the same size as the 2×2 sub-filter, and therefore, it is possible that a 2×2 sub-filter has already been included, such as... Figure 4G As shown, however, this is less ideal than 1×4 because it results in row splitting. Finally, fill with another 1×4 sub-filter. Figure 4C The remaining space in the rows (or optionally, and less ideally, including the two steps of the 2×2 sub-filter, such as...) Figure 4G (As shown). Next, Figure 4C The remaining space in the target filter has a dimension of 10×2. The largest sub-filter suitable for this space is a 4×1 sub-filter. Figure 4C ) or 2×2 sub-filter ( Figure 4G A 4×1 sub-filter is preferred because it has the same X-dimensional parameter as the initial sub-filter SFo1; specifically 4, and this helps avoid column splitting, thus simplifying data selection and movement. Therefore, an initial sub-filter SFo1 is selected for this group of rows, which has the largest possible sub-filter size suitable for the remaining space, optionally including: if multiple sub-filters with the same sub-filter size fit the remaining space, then including those with the same X-dimensional parameter X as the initial sub-filter SFo1 of the previous group of rows. 1..n The sub-filter is used as an alternative to the following operation: if multiple sub-filters with the same sub-filter size fit the remaining space, then select one from the multiple sub-filters with the same sub-filter size. The sub-filter with the lowest value. Now fill the sub-filters used for this group of rows with additional filters subject to the following constraints ( Figure 4C The middle row is 1 line. Figure 4G The remaining portion of the set of rows defined by the initial sub-filter (2 rows in total): at each step, the maximum filter size is selected. When filling the remaining space in the set of rows, at each step, a sub-filter with the maximum possible sub-filter size suitable for the remaining space in the set of rows is also preferred. Therefore, in Figure 4CThe 4×1 and 1×1 sub-filters have the same Y-dimensional parameter 1 as the initial sub-filter SFo1 that defines the first row of this group. Two 1×1 sub-filters are used at the end of this row because these are the largest sub-filters suitable for this row, and 2×1 sub-filters are not available. Figure 4C In the same way, the last group's row 1 is filled. This fills the dimensions of the target filter into each group of rows, with each group starting with the largest sub-filter; that is, determined by the product of the sub-filter's row size and column size, this largest sub-filter fits the remaining space, and fills the space in each group of rows defined by the initial filter in each row with the most possible sub-filters. This is achieved by selecting the initial sub-filter SFo1 and subsequently including sub-filters SF... 1..n The implementation also benefits from the ability to provide a solution within a limited amount of time by selecting the largest possible sub-filter size that fits the remaining space when filling the set of rows.
[0052] In some implementations, instead of filling the dimensions of the target filter in row-major order, the dimensions can be filled in column-major order. In these implementations, the sub-filters SF... 1..n Successively included in the combination up to the target dimension X of filter F. T , Y T The steps until the filter is filled with multiple sets of columns until no space remains include filling the target dimension X of the filter in column-major order by performing the following operations on each of the multiple sets of columns of the filter F. T , Y T :
[0053] - Select an initial sub-filter SFo1 for this group of columns, which has the largest possible sub-filter size suitable for the remaining space;
[0054] - Define a set of columns within the sub-filter dimension as having the same X-dimensional parameters X as the initial sub-filter SFo1. 1..n A set of columns;
[0055] - Successively including sub-filters SF 1..n The group of filters is filled until there is no space left in the group of filters, and each included sub-filter has the largest possible sub-filter size that fits the remaining space in the group of filters.
[0056] Similar to row priority selection, this is achieved by selecting the initial sub-filter SFo1 and subsequently including sub-filters SFo1. 1..nWhen filling the column group, the largest possible sub-filter size suitable for the remaining space is selected. This implementation also benefits from the ability to provide a solution within a limited timeframe. Similar to row-major selection, when selecting an initial sub-filter SFo1 for the column group, which has the largest possible sub-filter size suitable for the remaining space, it may optionally include: if multiple sub-filters with the same sub-filter size are suitable for the remaining space, then selecting from the multiple sub-filters with the same sub-filter size... The sub-filter with the lowest value. Another optional selection criterion is that the included sub-filters have the same X-dimensional parameters X as the initial sub-filter SFo1. 1..n Therefore, it successively includes the sub-filter SF. 1..n The group of filters is filled until there is no remaining space in the group. Each included sub-filter has the largest possible sub-filter size to fit the remaining space in the group. Optionally, if multiple sub-filters with the same sub-filter size fit the remaining space, then sub-filters with the same X-dimensional parameter X as the initial sub-filter SFo1 are included. 1..n The sub-filters. Another optional criterion is that, when selecting an initial sub-filter SFo1 for the column group, this initial sub-filter SFo1 may optionally include: if multiple sub-filters with the same sub-filter size fit the remaining space, then include those with the same Y-dimensional parameter Y as the initial sub-filter SFo1 of the previous group's columns. 1..n The sub-filters, as an alternative to the following operation: if multiple sub-filters with the same sub-filter size fit the remaining space, then select from the multiple sub-filters with the same sub-filter size. The sub-filter with the lowest value.
[0057] In some implementations, the defined filter includes a filter with a maximum X-dimensional parameter X. 1..n 'or the maximum Y-dimensional parameter Y' 1..n 'or the largest size sub-filter.' In these implementations, the target dimension X of the target filter is... T , Y T Including the target X-dimensional parameter X representing the number of columns of filter F T And the target Y-dimensional parameter Y represents the number of rows of filter F. T ; and multiple sub-filters SF 1..n The sub-filter dimension X of each of them 1..n ', Y 1..n 'Including the corresponding X-dimensional parameter X representing the column number of the sub-filter' 1..n ' and the corresponding Y-dimensional parameter Y representing the number of columns of the sub-filter 1..n Furthermore, the combination includes at least one sub-filter SF.1..n The at least one sub-filter has:
[0058] - Maximum X-dimensional parameter X 1..n ',or
[0059] - Maximum Y-dimensional parameter Y 1..n ',or
[0060] - Maximum size, which is determined by the X-dimensional parameter X. 1..n 'and Y-dimensional parameter Y 1..n The definition of the product of ';
[0061] The at least one sub-filter is suitable for the target dimension X of filter F. T Y T .
[0062] In terms of column or row or size, using the largest sub-filter provides better use of the available sub-filters.
[0063] In some implementations, the target filter may be referred to as a "large" target filter, which is defined herein as a target filter whose dimension is too large to be provided by any single sub-filter. A large target filter can be defined as a filter where the target dimension X, Y includes the target X-dimensional parameters X. T And the target Y-dimensional parameter Y T Furthermore, multiple sub-filters SF that can be implemented in the neural processing unit. 1..n The sub-filter dimension X of each of them 1..n ', Y 1..n 'Including the corresponding X-dimensional parameter X representing the column number of the sub-filter' 1..n ' and the corresponding Y-dimensional parameter Y representing the number of columns of the sub-filter 1..n '; and among them:
[0064] - Target X-dimensional parameters X T Equal to or exceeding the maximum realizable X-dimensional parameter X' 最大 The maximum achievable X-dimensional parameter is the X-dimensional parameter X that can be realized in the neural processing unit. 1..n The maximum value of '; and / or
[0065] - Target Y-dimensional parameters Y T Equal to or exceeding the maximum realizable Y-dimensional parameter Y' 最大 The maximum achievable Y-dimensional parameter is the Y-dimensional parameter Y that can be realized in the neural processing unit. 1..n The maximum value of '.
[0066] In some implementations, a minimum number of sub-filters are used to provide the filter. An optimization function can be used to determine the target dimension X, which sums up to be equivalent to the filter F. T Y T The minimum number of sub-filters. This optimization can be performed before the filter needs to be implemented and, for example, stored in a lookup table to save time during its implementation. In these implementations, the target dimension X is equivalent to the filter F. T , Y T Multiple sub-filters SF 1..n The combination of the target dimension X, which is equivalent to the filter F, T , Y T The minimum number of sub-filters corresponds to this.
[0067] Dilated filters can also be implemented using the above principles. During the convolution step, the dilated filter sparsely samples the input feature map. A dilated filter can be implemented by expanding the filter and inserting zero values according to the dilation rate. Additionally, Figures 6A-6D An implementation of an extended filter according to some aspects of this disclosure is shown. Figure 6A The image shows a 5×5 filter. Figure 6B In, it is shown Figure 6A An expanded version of the filter, where the expansion rate is 1. A 5×5 filter is expanded to a 9×9 filter. Zero weights are used in the shaded regions of the filter. Therefore, Figure 6B The target filter has a dimension of 9×9. Figure 6D It shows the use of Figure 6C The example shown illustrates a possible combination of sub-filters to provide an expanded target filter, subject to the following constraints: the combination of sub-filters has a dimension equivalent to the target filter dimension, and at least two of the sub-filters in the combination have a sub-filter dimension X. 1..n ', Y 1..n They are not equal. Therefore, here, filter F includes an extended filter; and consists of multiple sub-filters SF. 1..n The combination provides the target dimension X T Y T Corresponding to the dimension of the extended filter.
[0068] Once the filters have been defined, they can be used in neural networks. In one implementation, filters are used in CNNs to convolve the input feature maps. Additionally, Figure 7 A flowchart illustrating the use of filters in a CNN according to some aspects of this disclosure is shown. (Continued) Figure 5 The method, Figure 7 The methods also include:
[0069] - Receive input feature map (IFM); and
[0070] - Convolve the input feature map IFM with a defined filter F to provide the output feature map OFM.
[0071] Convolution can be performed with any appropriate stride; that is, it can be performed with one, two or more strides.
[0072] More specifically, when using the sub-filters of this disclosure, and as... Figure 7 As shown, the step of convolving the input feature map IFM with the defined filter F includes: convolving the input feature map IFM with multiple sub-filters SF at their respective positions in the combination. 1..n Each convolution in the model provides multiple corresponding partial feature maps, and the partial feature maps are summed to provide an output feature map (OFM).
[0073] As part of the convolution, the input feature map IFM can be padded to provide an output feature map with the same dimensions as the input feature map. In other words, it may have "the same padding". Therefore, the step of convolving the input feature map IFM with a defined filter F may include: padding the input feature map IFM such that the output feature map OFM has the same dimensions as the input feature map IFM.
[0074] Optionally, the input feature map IFM may not include any padding; that is, it may have “effective padding”.
[0075] The above-described embodiments of the method may be provided on a non-transitory computer-readable storage medium including a computer-readable instruction set stored thereon, which, when executed by at least one processor, causes at least one processor to perform the method.
[0076] The above-described embodiments of the method can also be implemented as a computer program product. The computer program product can be provided by dedicated hardware or hardware capable of running software in conjunction with appropriate software. When provided by a processor, these functions can be provided by a single dedicated processor, a single shared processor, or several separate processors that can share resources. Furthermore, the explicit use of the terms "processor" or "controller" should not be construed as specifically referring to hardware capable of running software, but may implicitly include, but is not limited to, digital signal processor (DSP) hardware, read-only memory (ROM) for storing hardware, random access memory (RAM), non-volatile storage devices, etc. Additionally, embodiments can take the form of a computer program product accessible from a computer-usable storage medium or a computer-readable storage medium, which provides program code for use by or in conjunction with a computer or any instruction execution system. For the purposes of this description, a computer-usable storage medium or a computer-readable storage medium can be any device that may include, store, communicate, propagate, or transmit programs for use by or in conjunction with an instruction execution system, device, or apparatus. The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system or device, or a propagation medium. Examples of computer-readable media include semiconductor or solid-state memory, magnetic tape, removable computer disks, random access memory (RAM), read-only memory (ROM), hard disks, and optical discs. Current examples of optical discs include compressed disc read-only memory (CD-ROM), optical disc read / write (CD-R / W), and Blu-ray. TM and DVD.
[0077] In some implementations, a system for performing the method is provided. Additionally, Figure 8 A system SY comprising a processor PROC and a memory MEM is illustrated. System SY includes at least one processor PROC and at least one memory MEM. The memory includes instructions that, when executed on at least one processor, cause the processor to provide a filter F for a neural processing unit by:
[0078] - Target dimension X of receiver and filter F T Y T Corresponding input;
[0079] - Receives multiple sub-filters SF that can be implemented in the neural processing unit. 1..n The sub-filter dimension X of each of them 1..n ', Y 1..n 'Corresponding input;
[0080] - Define the filter as multiple sub-filters (SF) 1..n A combination of elements that has the same dimensions as the target dimension X.T Y T The dimension of X, and wherein at least two of the sub-filters in the combination have a sub-filter dimension X. 1..n ', Y 1..n They are not equal.
[0081] The system may be, for example, a processor or a neural processing unit. A neural processing unit may correspond to the elements of the neural processing unit shown in Figure 2. In some embodiments, the system's processor may also be configured as:
[0082] - Receive input feature map (IFM); and
[0083] - The output feature map OFM is generated from the input feature map IFM by convolving it with the defined filter F.
[0084] The above embodiments should be understood as illustrative examples of this disclosure. Other embodiments are contemplated. For example, the embodiments described with respect to the method may also be implemented in a computer program product, a computer-readable storage medium, or a system. It should be understood that the features described with respect to any embodiment may be used alone or in combination with other features described, and may also be used in combination with one or more features of another embodiment or a combination of other embodiments. Furthermore, equivalents and modifications not described above may be employed without departing from the scope of this disclosure as defined in the appended claims.
Claims
1. A computer-implemented method, the method comprising: The target dimension (X) of the filter (F) of the receiving and neural processing unit T Y T The corresponding input; Receives multiple sub-filters (SFs) that can be implemented in the neural processing unit. 1..n The sub-filter dimension (X) of each of them 1..n ', Y 1..n The corresponding input; The filter (F) is defined as the plurality of sub-filters (SF). 1..n A combination of elements having a dimension equivalent to the target dimension (X). T Y T The dimension of the sub-filters in the combination, and the dimension of the sub-filters of at least two of the sub-filters in the combination (X). 1..n ', Y 1..n They are not equal; Receive input feature map (IFM); The input feature map (IFM) is convolved with the defined filter (F) to provide an output feature map (OFM), where The steps of convolving the input feature map IFM with the defined filter F include: convolving the input feature map IFM with each of the multiple sub-filters SF1..n in the combination at their respective positions to provide multiple corresponding partial feature maps, and summing the partial feature maps to provide the output feature map OFM.
2. The computer-implemented method according to claim 1, wherein the plurality of sub-filters (SF) 1..n The sub-filter dimension (X) of each of them 1..n ', Y 1..n ') includes an X-dimensional parameter (X) representing the number of columns of the sub-filter. 1..n ') and the Y-dimensional parameter (Y) representing the number of rows of the sub-filter 1..n '), and the size of each sub-filter is determined by the X-dimensional parameter (X). 1..n ') and the corresponding Y-dimensional parameters (Y) 1..n The definition of the product of '); and The filter is defined as the plurality of sub-filters (SF). 1..n The combination includes sub-filters (SF) 1..n The target dimension (X) is successively included in the combination up to the filter (F). T Y T The sub-filters are filled with the combination until no space remains, and each included sub-filter has the largest possible sub-filter size suitable for the remaining space.
3. The computer-implemented method according to claim 2, wherein the filter is defined as the plurality of sub-filters (SFs). 1..n The combination includes sub-filters (SF) 1..n The target dimension (X) is successively included in the combination up to the filter (F). T Y T Until the combination is filled with no remaining space, each included sub-filter has the maximum possible sub-filter size suitable for the remaining space, and wherein if multiple sub-filters with the same sub-filter size fit the remaining space, then a sub-filter with the same sub-filter size is selected from the multiple sub-filters having the same sub-filter size. The sub-filter with the lowest value.
4. The computer-implemented method according to claim 2, Among them, the sub-filter (SF) 1..n The target dimension (X) is successively included in the combination up to the filter (F). T Y T The target dimension (X) of the filter is filled with the combination until no space remains, including filling the filter (F) in row-major order by performing the following operation for each of the multiple sets of rows of the filter (F). T Y T ): Select an initial sub-filter (SFo1) for the group row, the initial sub-filter (SFo1) having the largest possible sub-filter size suitable for the remaining space; Define a set of rows within the sub-filter dimension as having the same Y-dimensional parameters (Y) as the initial sub-filter (SFo1). 1..n A group of rows; Successively including sub-filters (SF) 1..n The group rows are filled until there is no remaining space in the group rows, and each included sub-filter has the maximum possible sub-filter size that fits the remaining space in the group rows.
5. The computer-implemented method according to claim 2, Among them, the sub-filter (SF) 1..n The target dimension (X) is successively included in the combination up to the filter (F). T Y T The target dimension (X) of the filter is filled with the combination until no space remains, including filling the filter (F) in column-major order by performing the following operations for each of the multiple sets of columns of the filter (F). T Y T ): Select an initial sub-filter (SFo1) for the group, the initial sub-filter (SFo1) having the largest possible sub-filter size suitable for the remaining space; Define a set of columns within the sub-filter dimension as having the same X-dimensional parameters (X) as the initial sub-filter (SFo1). 1..n A set of columns; Successively including sub-filters (SF) 1..n The sub-filters are filled until there is no remaining space in the sub-filters, and each included sub-filter has the maximum possible sub-filter size that fits the remaining space in the sub-filters.
6. The computer-implemented method according to claim 1, wherein the target dimension (X) T Y T The target X-dimensional parameter (X) represents the number of columns of the filter (F). T ) and the target Y-dimensional parameter (Y) representing the number of rows of the filter (F). T ); and wherein the plurality of sub-filters (SF 1..n The sub-filter dimension (X) of each of them 1..n ', Y 1..n ') includes the corresponding X-dimensional parameter (X) representing the number of columns of the sub-filter. 1..n ') and the corresponding Y-dimensional parameter (Y) representing the column number of the sub-filter. 1..n ');and The combination said combination includes at least one sub-filter (SF) 1..n The at least one sub-filter has: Maximum X-dimensional parameter (X 1..n '),or Maximum Y-dimensional parameter (Y 1..n '),or Maximum size, the size is determined by the X-dimensional parameter (X 1..n ') and the Y-dimensional parameter (Y) 1..n The definition of the product of '). The at least one sub-filter is adapted to the target dimension (X) of the filter (F). T Y T ).
7. The computer-implemented method of claim 1, wherein the target dimension (X) equivalent to the filter (F) T Y T The multiple sub-filters (SF) 1..n The combination of the above and the target dimension (X) that is equivalent to the filter (F) T Y T The minimum number of sub-filters corresponds to this.
8. A non-transitory computer-readable storage medium comprising a computer-readable instruction set stored thereon, the computer-readable instruction set causing the at least one processor, when executed by at least one processor, to perform the method according to claim 1.
9. A system (SY) for providing a filter in a neural processing unit, the system comprising: At least one processor (PROC); as well as At least one memory module (MEM); The memory includes instructions that, when executed on the at least one processor, cause the processor to perform the method according to claim 1.