Dimensionality transformation for efficient bottleneck processing

Abstract

Certain aspects of the present disclosure provide techniques for efficient bottleneck processing via dimensionality transformation, including receiving a tensor and processing the tensor in a bottleneck block in a neural network model, the processing including applying a spatial-to-depth tensor transformation, applying a depth-wise convolution, and applying a depth-to-spatial tensor transformation.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS)

[0001] This application claims the benefit of and priority to U.S. patent application Ser. No. 17 / 651,549, filed Feb. 17, 2022, the entire contents of which are incorporated herein by reference.

[0002] introduction Aspects of the present disclosure relate to machine learning, and more specifically, to improving the efficiency of model processing.

[0003]

[0003] Artificial neural networks can be used to generate inferences for a wide variety of domains with substantial success. A large number of different neural network architectures have been proposed for various tasks, such as recurrent neural networks (RNNs), convolutional neural networks (CNNs), etc. In general, a neural network contains a number of neurons or nodes distributed across a series of layers, with weighted edges connecting the neurons between layers.

[0004]

[0004] Many neural network architectures use bottleneck blocks or layers at one or more points in the model. A bottleneck can generally be considered as a layer with relatively fewer neurons (e.g., the tensors processed are smaller and involve fewer elements) compared to the layers immediately before and after it. Such bottleneck blocks can be useful to reduce the number of features in the data at various points in the model, thereby reducing the required computational resources, while at the same time improving generalization, which helps prevent overfitting. However, existing bottleneck architectures still impose a very large computational overhead.

[0005]

[0005] Therefore, there is a need for an improved bottleneck architecture to improve the processing efficiency of machine learning models. Summary of the Invention

[0006]

[0006] Some aspects provide a method that includes receiving a first tensor and processing the first tensor in a bottleneck block within a neural network model, the processing including applying a space-to-depth tensor transformation, applying a depthwise convolution, and applying a depth-to-space tensor transformation.

[0007]

[0007] Another aspect provides a processing system configured to perform the aforementioned methods as well as methods described herein; a non-transitory computer readable medium comprising instructions which, when executed by one or more processors of the processing system, cause the processing system to perform the aforementioned methods as well as methods described herein; a computer program product embodied on the computer readable storage medium comprising code for performing the aforementioned methods as well as methods further described herein; and the processing system comprising means for performing the aforementioned methods as well as methods further described herein.

[0008] The following description and the annexed drawings set forth in detail certain illustrative features of the one or more aspects. [Brief description of the drawings]

[0009]

[0009] The accompanying drawings illustrate some aspects of one or more aspects and therefore should not be considered as limiting the scope of the present disclosure. [Figure 1]

[0010] 1 illustrates an exemplary spatial-to-depth dimensionality conversion technique. [Diagram 2]

[0011] 1 illustrates an exemplary spatial-to-depth dimensionality conversion technique. [Diagram 3]

[0012] 1 illustrates an example bottleneck block with dimensionality transformation to improve processing efficiency. [Figure 4]

[0013] 1 illustrates another example bottleneck block with dimensionality transformation to improve processing efficiency. [Diagram 5]

[0014] FIG. 13 is an exemplary flow diagram illustrating a method for processing data tensors in a bottleneck block using dimensionality transformation to improve efficiency. [Figure 6]

[0015] FIG. 11 is another example flow diagram illustrating a method for processing data tensors in a bottleneck block using dimensionality transformation to improve efficiency. [Figure 7]

[0016] FIG. 1 is an exemplary flow diagram illustrating a method for processing bottleneck blocks using dimensionality transformation to improve efficiency. [Figure 8]

[0017] 1 illustrates an exemplary processing system configured to carry out various aspects of the present disclosure.

[0010]

[0018] For ease of understanding, wherever possible, like reference numerals have been used to designate like elements common to the figures, and it is contemplated that elements and features of one embodiment may be beneficially incorporated in other embodiments without further recitation. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011]

[0019] Aspects of the present disclosure provide techniques for more efficiently processing data through dimensionality transformation at the bottleneck block of a neural network.

[0012]

[0020] Typically, early layers of a neural network operate on features with large spatial size (e.g., height and width of image or video input data) and a relatively small number of channels (also referred to as depth in some aspects), which requires a large number of multiply-accumulate (MAC) operations relative to the number of parameters in these layers. To improve efficiency, some CNN architectures employ significant spatial downsampling in early layers, which can cause significant loss of information and thus adversely affect model performance.

[0013]

[0021] In addition, many convolutional neural network architectures require larger receptive fields in early layers to improve accuracy. Therefore, due to the large spatial size of the inputs to these early layers, large kernel sizes with large receptive fields are often used to effectively aggregate global spatial information. However, such large kernel sizes can result in extremely large computational complexity and memory access latency, which adversely affects model performance.

[0014]

[0022] Aspects of the present disclosure provide a bottleneck block incorporating a dimensionality conversion technique that transfers operations in spatial dimensions to channel dimensions via spatial-depth operations, thereby improving the efficiency of data processing. This efficiency is achieved because subsequent depth-wise convolutions are more efficient despite the increase in channel dimensionality because fewer total operations are required when operating on a smaller spatial dimension.

[0015]

[0023] The bottleneck blocks described herein may further include depth-spatial operations to restore the desired dimensionality of the layer output. In some aspects, these spatial-depth operations (and associated depth-spatial operations) may be implemented using hardware or accelerators, allowing them to be performed efficiently with reduced latency.

[0016]

[0024] Such spatial-depth bottleneck blocks can provide substantially improved receptive fields (allowing kernels to effectively cover larger regions of the input tensor without increasing computational overhead) compared to traditional bottlenecks, because they aggregate spatial information with larger receptive fields by shifting computation from the spatial dimension to the channel dimension (e.g., using the same size kernels can be up to r times larger, where r is the spatial magnification factor used to configure the spatial-depth operation). This can enable neural networks to achieve or exceed the performance of comparable existing architectures while using fewer computational operations, thereby leading to lower latency in data processing. Notably, the overall model size increase resulting from applying such spatial-depth techniques throughout the depth-wise convolutional layers is negligible. Thus, model performance is improved while minimizing additional model complexity.

[0017]

[0025] In some aspects, the spatial expansion rate may be selected based on a desired spatial dimensionality of a depth unit layer of a depth unit bottleneck, or based on a desired maximum number of operations required to perform a depth unit convolution. For example, if the spatial dimensionality of a bottleneck convolution (or the number of operations required for a bottleneck convolution) exceeds a threshold, the spatial expansion rate may be defined (e.g., by an administrator or user) to ensure that the desired spatial dimensionality is met such that the number of operations required to perform a depth unit convolution is less than or equal to a desired maximum.

[0018]

[0026] Aspects of the present disclosure can be generally applied to improve the computational efficiency of processing data using bottleneck layers (or blocks) of neural networks, both during training (while training data is being passed through the model) and during runtime (when new data is being passed through to generate inferences).

[0019] Example Element-To-Depth Technique

[0027] 1 shows an exemplary element-wise spatial-depth dimensionality conversion technique. That is, the illustrated example includes a spatial-depth conversion 110 that reduces the spatial dimensionality of an input data tensor 105 by expanding the channel or depth dimensionality of the tensor. Thus, the input data is transformed into a "shape" while information is preserved. Advantageously, it is more efficient to perform a depth-wise convolution on this transformed tensor, since computations can be performed more efficiently for tensors with smaller spatial dimensions and larger channel dimensions compared to tensors with larger spatial dimensions and smaller channel dimensions.

[0020]

[0028] As shown, the input tensor 105 has spatial dimensionality (e.g., height and width) and channel dimensionality (e.g., depth). The tensor 105 shown has a single channel (e.g., channel dimensionality of 1) and spatial dimensionality of height and width of 4 for ease of illustration, although aspects of the present disclosure are readily applicable to tensors of any dimensionality.

[0021]

[0029] In the illustrated embodiment, the space-depth transform 110 reconstructs the input tensor 105 into the output tensor 115. Specifically, the space-depth transform 110 reduces the spatial dimensionality of the input tensor 105 (e.g., from 4×4 to 2×2 in the illustrated example) while increasing the channel dimensionality by a corresponding amount (e.g., from 1 to 4 in the illustrated example) to maintain the "volume" of the tensor (e.g., the number of elements in the tensor). That is, the volume of the output tensor 115 remains the same as the volume of the input tensor 105, since 4×4×1=2×2×4. In this manner, no data is lost in the space-depth transform 110 (e.g., all data elements in the input tensor 105 are included in the output tensor 115).

[0022]

[0030] The particular reconstruction or modification used to transform the input tensor 105 is defined based at least in part on a spatial expansion factor r, which is a configurable hyperparameter. In some aspects, the spatial expansion factor may be defined by a user or administrator to transform the input tensor into a tensor with a desired spatial dimensionality of the depthwise convolution. For example, assume that the desired spatial dimensionality of the bottleneck is M×M, but the input tensor has a larger spatial dimensionality N×N. In such an example, a user may select a spatial expansion factor that will transform the input tensor into an intermediate tensor with the desired spatial dimensionality M×M.

[0023]

[0031] In some embodiments, the spatial-to-depth transform 110 converts an input tensor of size C×H×W into a tensor of size

[0024]

number

[0025] where C is the number of channels (also called the depth of the tensor), H and W are the spatial dimensions of the tensor (e.g., height and width), and r is the spatial expansion factor. In the illustrated embodiment, C=1, H=W=4, and r=2.

[0026]

[0032] In some cases, the spatial-depth transform 110, as described herein, may be referred to as a pixel-to-depth transform or an element-to-depth transform because the tensor transform is performed on a pixel-by-pixel (or element-by-element) basis, where each element in the original input tensor 105 is processed independently and may not remain next to the same neighboring elements in the output tensor 115. To clarify the concept of how the elements are reconstructed, emphasis is placed on elements a, c, i, and k, which are placed in the first channel 115A of the output tensor 115. In the illustrated embodiment, elements a, c, i, and k in the input tensor 105 are placed in a first channel 115A of the output tensor 115, elements b, d, j, and l in the input tensor 105 are placed in a second channel 115B of the output tensor 115, elements e, g, m, and o in the input tensor 105 are placed in a third channel 115C of the output tensor 115, and elements f, h, n, and p in the input tensor 105 are placed in a fourth channel 115D of the output tensor 115.

[0027]

[0033] In various aspects, the particular rearrangement of elements (e.g., which elements in the input are assigned to which channels in the output) may vary based on the particular implementation and architecture of the model. In general, however, the space-to-depth transform will be fixed in that each position in the input tensor is associated with a corresponding fixed position in the output tensor (e.g., the element in the upper left corner of the input tensor 105 will always be located in the upper left corner of the first channel of the output tensor 115). Because the space-to-depth transform 110 is a fixed rearrangement of elements, it can be implemented efficiently via hardware or with hardware acceleration.

[0028]

[0034] Although the illustrated example involves a spatial-depth transform 110, a complementary depth-spatial transform can be used to effectively reverse the dimensionality transform. That is, a depth-spatial transform using the same spatial expansion factor r may be used to increase the spatial dimensionality of an input tensor while decreasing the channel dimensionality to preserve the volume of the tensor (e.g., transforming tensor 115 to tensor 105).

[0029] Exemplary Area-Depth Techniques

[0035] 2 illustrates an exemplary region-by-region space-to-depth dimensionality transformation technique. In contrast to the space-to-depth transformation 110 of FIG. 1 (which operates element-by-element), the space-to-depth transformation 210 reconstructs at the level of non-overlapping regions in the input tensor as indicated by shading in the illustrated embodiment. To clarify the concept of how the regions are rendered, a first region 207, including elements a, b, e, and f, is also highlighted. In some aspects, the space-to-depth transformation 210 may be referred to as a region-to-depth transformation.

[0030]

[0036] In the illustrated aspect, the space-to-depth transform 210 reconstructs the input tensor 205 into an output tensor 215 by reducing the spatial dimensionality of the input tensor 205 (e.g., from 4×4 to 2×2 in the illustrated example) while increasing the channel dimensionality by a corresponding amount (e.g., from 1 to 4 in the illustrated example) to preserve the volume of the tensor. As with the space-to-depth transform 110 shown in FIG. 1, no data is lost in the space-to-depth transform 210 (e.g., all data elements in the input tensor 205 are included in the output tensor 215).

[0031]

[0037] As noted above, the particular reconstruction or modification used to transform the input tensor 205 is defined based at least in part on a spatial expansion factor r, which may be a configurable hyperparameter. Similar to the space-to-depth transform 110 of FIG. 1, the space-to-depth transform 210 transforms an input tensor of size C×H×W into a space-to-depth transform of size

[0032]

number

[0033] where C is the number of channels (also called the depth of the tensor), H and W are the spatial dimensions of the tensor (e.g., height and width), and r is the spatial expansion factor. In the illustrated embodiment, C=1, H=W=4, and r=2.

[0034]

[0038] However, the space-to-depth transform 210 may be referred to as a region-to-depth transform since the tensor reconstruction is performed region by region. Specifically, in the illustrated embodiment, the first region 207 (including elements a, b, e, and f in the input tensor 205) is placed in a first channel 215A of the output tensor 215, the second region including elements c, d, g, and h in the input tensor 205 is placed in a second channel 215B of the output tensor 215, the third region including elements i, j, m, and n in the input tensor 205 is placed in a third channel 215C of the output tensor 215, and the fourth region including elements k, l, o, and p in the input tensor 205 is placed in a fourth channel 215D of the output tensor 215.

[0035]

[0039] In various aspects, the particular rearrangement of regions (e.g., which regions in the input are assigned to which channels in the output) may vary based on the particular implementation and architecture of the model. In general, however, the space-to-depth transformation will be fixed in that each region in the input tensor is associated with a corresponding fixed channel in the output tensor (e.g., the region in the upper left corner of the input tensor 205 will always be placed in the first channel of the output tensor 215). Because the space-to-depth transformation 210 is a fixed rearrangement of regions, it can be implemented efficiently via hardware or with hardware acceleration.

[0036]

[0040] Although the illustrated example involves a spatial-depth transform 210, a complementary depth-spatial transform can be used to effectively reverse the dimensionality transform. That is, a depth-spatial transform using the same spatial expansion factor r may be used to increase the spatial dimensionality of an input tensor while decreasing the channel dimensionality to preserve the volume of the tensor (e.g., transforming tensor 215 to tensor 205).

[0037]

[0041] In various aspects, the region-based space-to-depth transform 210 can be used in place of the element-based space-to-depth transform 110 shown in Figure 1. The space-to-depth transform 210 has the same expressiveness as the space-to-depth transform 110, but in some implementations may provide advantages such as improved spatial gating (a spatial version of channel gating).

[0038] Example Bottleneck Block Using Dimensionality Transformation Across Pointwise Convolution

[0042] 3 illustrates an example bottleneck block 300 with dimensionality transformation to improve processing efficiency. The bottleneck block 300 generally includes a depth unit convolution 330 along with a spatial-to-depth transformation 320 and a corresponding depth-to-spatial transformation 350 that are used before and after the bottleneck convolution, respectively. By using these dimensionality transformations, the bottleneck block 300 can operate more efficiently (e.g., with reduced latency and computational resources).

[0039]

[0043] In the bottleneck block 300, the number of channels in the input tensor 305 is first reduced using a first convolution 1×1 (point-wise) convolution 310, thereby reducing the memory and MAC count for subsequent convolutions.

[0040]

[0044] in particular,

[0041]

number

[0042] Use pointwise convolution kernels of size C in The input tensor 305 is convolved with a matrix of C in It has dimensionality x1 x1, and C inis the number of channels in the input tensor 305, H and W are the spatial dimensions of the input tensor 305, t is the channel expansion factor, and r is the spatial expansion factor, as described above. In at least one embodiment, the input tensor 305 is an activation tensor as an input to a layer of a neural network, and the kernels correspond to the weights of that layer.

[0043]

[0045] The illustrated example is for point-wise convolution 310

[0044]

number

[0045] pointwise kernels are used, but any number of pointwise kernels can be used. The illustrated bottleneck block 300 uses tC in kernels for pointwise convolution 310, just to reflect the traditional configuration of a conventional bottleneck block.

[0046]

number

[0047] pointwise kernels are used.

[0048] As shown, this first point-wise convolution 310 has a dimensionality

[0049]

number

[0050] That is, the intermediate tensor 315 has the same spatial dimensions but has an altered (e.g., reduced) channel depth.

[0051] The intermediate tensor 315 is then processed using a space-to-depth transform 320 (e.g., an element-based dimensionality transform or a region-based dimensionality transform) to generate another intermediate tensor 325 having increased depth (e.g., more channels) and reduced spatial dimensionality. In the illustrated example, the intermediate tensor 325 has dimensionality

[0052]

number

[0053] where the spatial dimensionality is reduced by r.

[0054]

[0048] The depth-wise convolution 330 can then be applied to the tensor 325, which has a reduced spatial size compared to the original input tensor 305, meaning that the depth-wise convolution 330 can be performed using fewer computational operations. Thus, the spatial depth operation 320 beneficially reduces latency and computational resources without reducing the information in the input data. In the illustrated bottleneck block 300, the depth-wise convolution 330 is performed using a time tC in The method is performed using depthwise convolution kernels, each having a dimensionality of 1 × 3 × 3. Although depthwise convolution kernels having a spatial size of 3 × 3 are illustrated, in other examples, the depthwise kernels may be of any spatial size.

[0055]

[0049] The illustrated example shows tC for depthwise convolution 330. in Although depth-wise kernels are used, other numbers of kernels can be used. In some aspects, the number of depth-wise kernels used for the depth-wise convolution 330 is selected to match the channel dimensionality of the preceding layer (e.g., of the tensor 325).

[0056]

[0050] Advantageously, due to the space-to-depth transformation 320, the depth-wise convolution 330 can effectively function as a dilation convolution (covering an increased receptive field in the original input tensor 305 compared to the conventional bottleneck), thereby improving model accuracy.

[0057] As shown, the depthwise convolution 330 is a dimensional convolution that matches the intermediate tensor 325.

[0058]

number

[0059] This results in another intermediate tensor 335 having the following structure:

[0060] After the depth-wise convolution 330, the bottleneck block 300 includes a second point-wise convolution 340, which each in r with dimensionality of ×1×1 2 C out This is performed using pointwise convolution kernels. The resulting intermediate tensor 345 has dimensionality

[0061]

number

[0062] The illustrated example has r 2 C out Although pointwise kernels are used, other numbers of kernels can be used. In some embodiments, the number of kernels used for the pointwise convolution 340 is determined so that the resulting tensor 345 is 2 C out The final output tensor 355 (after the depth-to-space transform 350 is performed) has C out The channel count is selected to ensure that the number of channels is equal to or greater than 1.

[0063]

[0053] A depth-to-space transform 350 is then used to transform the intermediate tensor 345 into a tensor of dimension C out In some embodiments, the bottleneck block 300 may be configured such that the dimensionality of the output tensor 355 matches the dimensionality of a conventional bottleneck block, allowing for efficient replacement of the bottleneck block 300 without reconstructing the rest of the network.

[0064]

[0054] In the bottleneck block 300, a second point-wise convolution 340 aggregates the channel information in the output features of the depth-wise convolution 330, and then a depth-spatial operation 350 is applied.

[0065] Advantageously, the bottleneck block 300 may use approximately the same number of parameters but require substantially fewer operations compared to a conventional bottleneck (without spatial-to-depth transform 320 or depth-to-spatial transform 350). For example, if the depthwise convolution 330 is performed using a 3×3 kernel, the conventional bottleneck block requires:

[0066]

number

[0067] parameters are required, tC in HW * (C in +9+C out ) MACs. In contrast, a more efficient bottleneck block 300 can be implemented using

[0068]

number

[0069] It requires parameters,

[0070]

number

[0071] This reduced number of MAC operations can reduce the latency and computational resources required to process data using bottleneck block 300.

[0072]

[0056] In some aspects, the bottleneck block 300 is selectively used based on the characteristics of the device performing the convolution. For example, if the computational resources (which may include processing power, memory, energy availability (e.g., whether the device uses battery power), etc.) meet a certain threshold, the system may use a conventional bottleneck. In contrast, if the processing device is a low-power device (e.g., a mobile device or other battery-powered device) with reduced computational resources or energy availability that does not meet the threshold, the system may replace the conventional bottleneck with an efficient bottleneck block 300 that uses spatial-depth dimensionality transformation and depth-spatial dimensionality transformation to improve computational efficiency. This may allow a less capable device to achieve similar accuracy with reduced computational complexity, while a more capable device can retain the highest accuracy of the model.

[0073] In some aspects, when the bottleneck block 300 is used, it can replace all of the bottlenecks in a given neural network architecture. However, in other aspects, the bottleneck block 300 can be selectively used to replace only a subset of existing (traditional) bottlenecks. For example, the bottleneck block 300 may be used to replace only the last traditional bottleneck in the architecture, the penultimate bottleneck, all bottlenecks in the latter half of the architecture, etc. This can improve the accuracy of the assembled model. Thus, the amount of traditional bottleneck blocks replaced by an efficient bottleneck block such as 300 can be a hyperparameter in various examples.

[0074] An exemplary bottleneck block using dimensionality transformation across depthwise convolutions

[0058] Figure 4 illustrates an example bottleneck block 400 with dimensionality transformation to improve processing efficiency. The bottleneck block 400 generally includes a depth unit convolution 430 along with a spatial-depth transformation 420 and a corresponding depth-to-spatial transformation 440 that are used before and after the bottleneck convolution, respectively. By using these dimensionality transformations, the bottleneck block 400 can operate more efficiently (e.g., with reduced latency and computational resources).

[0075] 3 used a modified first pointwise convolution to reduce the channel depth of the input, the bottleneck block 400 can use the same pointwise convolution as a conventional bottleneck. In the bottleneck block 400, the spatial-depth and depth-spatial operations are applied across (immediately before and after) a depthwise separable convolution layer, as discussed in more detail below.

[0076] In the bottleneck block 400, the input tensor 405 is first processed using a first 1×1 (point-wise) convolution 410. Specifically, t in We use pointwise convolution kernels of size C in The input tensor 405 is convolved with C in ×1×1 dimensionality, C in is the number of channels in the input tensor 405, and as described above, H and W are the spatial dimensions of the input tensor 405, t is the channel expansion factor, and r is the spatial expansion factor.

[0077]

[0061] The illustrated example shows tC for point-wise convolution 410. in Although we use pointwise kernels, other numbers of pointwise kernels can be used. The illustrated bottleneck block 400 uses tC for the pointwise convolution 410 to reflect the traditional construction of a conventional bottleneck block. in pointwise kernels are used.

[0078] As shown, this first point-wise convolution is a convolution of dimension tC in t, yielding an intermediate tensor 415 with xHxW, i.e., the intermediate tensor 415 has the same spatial dimensions, but with the channel depth modified based only on the channel broadening factor t.

[0079] In the bottleneck block 400, the intermediate tensor 415 is then processed using a space-to-depth transform 420 (e.g., an element-based dimensionality transform or a region-based dimensionality transform) to generate an intermediate tensor 425 having increased depth (e.g., more channels) and reduced spatial dimensionality. In the illustrated embodiment, the intermediate tensor 425 has a dimensionality of

[0080]

number

[0081] has.

[0082]

[0064] A depthwise convolution 430 may then be applied to the tensor 425, which has a reduced spatial size compared to the original input tensor 405. Thus, the depthwise convolution 430 may be performed using fewer computational operations, which beneficially reduces latency and computational resources. In the illustrated bottleneck block 400, the depthwise convolution 430 is 2 tC in It is performed using depthwise convolution kernels (whereas the conventional block takes tC in 3 kernels are used), in this example each kernel has a dimensionality of 1 × 3 × 3. Although a depthwise convolution kernel having a spatial size of 3 × 3 is illustrated, in other examples the depthwise kernel may be of any spatial size.

[0083]

[0065] The illustrated example shows r 2 tC in Although depthwise kernels are used, other numbers of kernels can be used. In some aspects, the number of depthwise kernels used for the depthwise convolution 430 is selected to match the channel dimensionality of the preceding layer (e.g., of the tensor 425).

[0084]

[0066] Advantageously, due to the space-to-depth transformation 420, the depth-wise convolution 430 can effectively function as a dilation convolution (covering an increased receptive field in the original input tensor 405 compared to the conventional bottleneck), thereby improving model accuracy.

[0085] As shown, the depthwise convolution 430 has a dimensionality that matches the intermediate tensor 425.

[0086]

number

[0087] This results in an intermediate tensor 435 having the following structure:

[0088] Following the depth-wise convolution 430, a depth-to-space transformation 440 is then used to convert the intermediate tensor 435 to a tensor of dimensionality tC in The tensor vector can be transformed into an intermediate tensor 445 having dimensions H×W×H.

[0089]

[0069] The bottleneck block 400 then includes a second point-wise convolution 450, which in C with dimensionality x1 x1 out This is performed using pointwise convolution tensors. The resulting output tensor 455 has dimensionality C out ×H×W. In some embodiments, the bottleneck block 300 is configured such that the dimensionality of the output tensor 355 matches the dimensionality of a conventional bottleneck block, allowing for efficient replacement of the bottleneck block 300 without reconstructing the rest of the network.

[0090] The illustrated example uses C for the second point-wise convolution 450. out Although pointwise kernels are used, other numbers of kernels can be used. In some embodiments, the number of kernels used for the pointwise convolution 450 is determined based on the number of kernels that the resulting output tensor 455 has. out The channel count is selected to ensure that the number of channels is equal to or greater than 1.

[0091] Advantageously, the bottleneck block 400 uses exactly the same number of operations (e.g., MACs) while using slightly more parameters compared to a conventional bottleneck (without spatial-to-depth transform 420 or depth-to-spatial transform 440). For example, when the depthwise convolution 430 is performed using a 3×3 kernel, the conventional bottleneck block:

[0092]

number

[0093] parameters are required, tC in HW * (C in +9+C out ) MACs. In contrast, the bottleneck block 400 requires the same number of operations and

[0094]

number

[0095] parameters are required.

[0096]

[0072] In some aspects, similar to the bottleneck block 300 shown in Figure 3, the bottleneck block 400 may be selectively used based on the characteristics of the device performing the convolution (e.g., based on the computational resources of the device that will process the bottlenecks). Similarly, the bottleneck block 400 may be used to selectively replace only a subset of the traditional bottlenecks in the architecture, such as only the last traditional bottleneck in the architecture, the penultimate bottleneck, all bottlenecks in the latter half of the architecture, etc. This can improve the accuracy of the assembled model.

[0097] Exemplary Method for Processing Data Using Dimensionality Transformation Across Point-Wise Convolutions 5 is an exemplary flow diagram illustrating a method 500 for processing data tensors in a bottleneck block using dimensionality transformation to improve processing efficiency. In one aspect, the method 500 corresponds to the bottleneck block 300 described above with reference to FIG.

[0098] In some aspects, as discussed above, method 500 may be selectively used to replace one or more conventional bottlenecks in an existing neural network architecture. For example, method 500 may be used by a low-power device only in selected portions of the model, such as in later portions of the model.

[0099]

[0075] The method 500 begins with receiving an input tensor at block 505. In some cases, the input tensor may be an original input to the model, and in other cases, the input tensor may be an output tensor from a previous layer in the model. In general, the input tensor has a shape defined by its spatial dimensionality (e.g., height and width in the context of certain types of input data, such as image and video input data) and its channel dimensionality (also referred to as depth).

[0100] At block 510, a point-wise convolution is performed on the input tensor based at least in part on the spatial expansion factor (also referred to in some aspects as the spatial coefficient) to generate an intermediate tensor. For example, as discussed above with reference to the point-wise convolution 310 of FIG. 3, the processing system may use the point-wise convolution to reduce the number of channels in the input tensor based in part on the spatial expansion factor.

[0101]

[0077] The method 500 then proceeds to block 515, where a space-to-depth transformation is applied to the intermediate tensor (produced using the point-wise convolution in block 510) to reduce the spatial dimensionality and increase the channel dimensionality of the intermediate tensor while preserving the volume of the intermediate tensor. This space-to-depth transformation may include any suitable dimensionality transformation, such as the element-wise transformation discussed with reference to Figure 1, the area-wise transformation discussed with respect to Figure 2, etc.

[0102]

[0078] In block 520, a depth-wise convolution is performed on the transformed intermediate tensor to generate a second intermediate tensor.

[0103]

[0079] The method 500 then proceeds to block 525, where a point-wise convolution is performed on the second intermediate tensor to aggregate channel information of the second intermediate tensor based at least in part on the spatial expansion ratio. For example, as discussed above with reference to Figure 3, the point-wise convolution may be used to change the number of channels in the second intermediate tensor based in part on the spatial expansion ratio. This results in a third intermediate tensor.

[0104]

[0080] In block 530, a depth-space transformation may be applied to the third intermediate tensor (generated using the point-wise convolution in block 525) to increase the spatial dimensionality of the tensor and reduce the channel dimensionality while maintaining the volume of the tensor. As discussed above, this depth-space transformation may generally include any suitable dimensionality transformation, such as the element-wise transformation discussed with reference to Figure 1, the area-wise transformation discussed with respect to Figure 2, etc. In some aspects, the depth-space transformation and the space-depth transformation are selected to be complementary (e.g., if an element-wise space-depth transformation is used, an element-wise depth-space transformation should also be used).

[0105]

[0081] As indicated in block 535, the resulting tensor may then be returned as output from the bottleneck layer of the model (e.g., as input to the next layer).

[0106] Exemplary Method for Processing Data Using Dimensionality Transformation Across Depth-wise Convolutions 6 is an exemplary flow diagram illustrating a method 600 for processing data tensors in a bottleneck block using dimensionality transformation to improve processing efficiency. In one aspect, the method 600 corresponds to the bottleneck block 400 described above with reference to FIG.

[0107] In some aspects, as discussed above, method 600 may be selectively used to replace one or more conventional bottlenecks in an existing neural network architecture. For example, method 600 may be used by a low-power device only in selected portions of the model, such as in later portions of the model.

[0108]

[0084] The method 600 begins at block 605, where an input tensor is received. As discussed above, this tensor is input to the bottleneck layer, but may not be the original input to the model. That is, the input tensor may be an output tensor from a previous layer in the model. The input tensor has existing dimensionality, including spatial dimensionality (also referred to as height and width) and channel dimensionality (also referred to as depth).

[0109]

[0085] At block 610, a point-wise convolution is performed on the input tensor to generate an intermediate tensor. This can be a conventional point-wise convolution, as discussed above with reference to the point-wise convolution 410 of Figure 4. That is, because the method 400 uses a dimensionality transformation immediately before and after the depth-wise convolution, an existing conventional point-wise convolution can be used.

[0110]

[0086] The method 600 then proceeds to block 615, where a space-to-depth transformation is applied to the intermediate tensor (produced using the point-wise convolution in block 610) to reduce the spatial dimensionality and increase the channel dimensionality of the intermediate tensor while preserving the volume of the intermediate tensor. This space-to-depth transformation may include any suitable dimensionality transformation, such as the element-wise transformation discussed with reference to Figure 1, the area-wise transformation discussed with respect to Figure 2, etc.

[0111]

[0087] At block 620, a bottleneck depth-wise convolution is performed on the transformed intermediate tensor based at least in part on the spatial magnification ratio to generate a second intermediate tensor. For example, as discussed above with reference to the depth-wise convolution 430 of Figure 4, the depth-wise convolution may be performed using multiple kernels defined based at least in part on the spatial magnification ratio. This results in a second intermediate tensor.

[0112]

[0088] The method 600 then proceeds to block 625, where a depth-to-space transformation is applied to the second intermediate tensor (produced using the depth-wise convolution in block 620) to increase the spatial dimensionality of the tensor and reduce the channel dimensionality while preserving the volume of the tensor. As discussed above, this depth-to-space transformation may generally include any suitable dimensionality transformation, such as the element-wise transformation discussed with reference to FIG. 1, the region-wise transformation discussed with respect to FIG.

[0113] At block 630, a point-wise convolution is performed on the transformed second tensor. This can be a conventional point-wise convolution, as discussed above with reference to point-wise convolution 450 of Figure 4. That is, because method 400 uses a dimensionality transformation immediately before and after the depth-wise convolution, an existing conventional point-wise convolution can be used.

[0114]

[0090] As indicated at block 635, the resulting tensors may then be returned as output from the bottleneck layer of the model (e.g., to a subsequent layer of the model).

[0115] Exemplary Method for Processing Bottleneck Layers Using Dimensionality Transformation FIG. 7 is an example flow diagram illustrating a method 700 for processing bottleneck blocks using dimensionality transformation to improve efficiency.

[0116]

[0092] The method 700 begins at block 705, where a first tensor is received.

[0117]

[0093] At block 710, the first tensor is processed in a bottleneck block within the neural network model, where the processing includes applying a spatial-to-depth tensor transformation, applying a depth-wise convolution, and applying a depth-to-spatial tensor transformation.

[0118]

[0094] In some aspects, the space-to-depth tensor transformation is configured to reduce the spatial dimensionality of the input tensor to the space-to-depth tensor transformation by a spatial factor r and to increase the channel dimensionality of the input tensor to the space-to-depth tensor transformation by a spatial factor r, such that the volume of the output vector from the space-to-depth tensor transformation matches the volume of the input tensor to the space-to-depth tensor transformation.

[0119]

[0095] In some aspects, the depth-to-space tensor transformation is configured to increase the spatial dimensionality of the input tensor to the depth-to-space tensor transformation by a spatial factor r and to decrease the channel dimensionality of the input tensor to the depth-to-space tensor transformation by a spatial factor r, such that the volume of the output vector from the depth-to-space tensor transformation matches the volume of the input tensor to the depth-to-space tensor transformation.

[0120]

[0096] In some aspects, the spatial-to-depth tensor transformation immediately follows the first point-wise convolution of the bottleneck block and immediately follows the depth-wise convolution.

[0121]

[0097] In some aspects, the depth-to-space tensor transformation immediately follows the second point-wise convolution in the bottleneck block.

[0122] In some aspects, the first point-wise convolution comprises:

[0123]

number

[0124] kernels, where t is the channel expansion factor, r is the spatial expansion factor, and C in is the number of channels in the first tensor.

[0125] In some aspects, the output of the spatial-to-depth tensor transform is dimensional

[0126]

number

[0127] where t is the channel expansion factor, r is the spatial expansion factor, and C in is the number of channels in the first tensor, and H and W are the spatial dimensions of the first tensor.

[0128]

[0100] In some aspects, the depth-to-space tensor transformation immediately precedes the second point-wise convolution in the bottleneck block.

[0129] In some aspects, the output of the spatial-to-depth tensor transform is dimensional

[0130]

number

[0131] where t is the channel expansion factor, r is the spatial expansion factor, and C in is the number of channels in the first tensor, and H and W are the spatial dimensions of the first tensor.

[0132]

[0102] In some aspects, the method 700 further includes determining that the input tensor to the neural network model exceeds a spatial dimensionality threshold and determining a spatial coefficient r based on a desired spatial dimensionality of the depth-wise convolution of the bottleneck block.

[0133]

[0103] In some aspects, the method 700 further includes determining the computational power characteristics of a device that processes a bottleneck block in the neural network model and replacing a segment of the neural network model with the bottleneck block based on the determined characteristics.

[0134]

[0104] In some aspects, the neural network model includes multiple bottleneck blocks, and among the multiple bottleneck blocks, only the bottleneck blocks that are later in the neural network model include a spatial-to-depth tensor transformation (or a depth-to-spatial tensor transformation).

[0135]

[0105] The method 700 then proceeds to block 715, where subsequent blocks in the neural network model are optionally processed, i.e., the processing system may process the data (e.g., tensors output from the bottleneck block) using blocks or layers subsequent to the bottleneck block or layer.

[0136] Exemplary Processing System for Efficient Bottleneck Processing Using Dimensionality Transformation - Patent application In some aspects, the techniques, methods, and workflows described with reference to FIGS. 1-7 may be implemented on one or more devices or systems.

[0137] FIG. 8 illustrates an example processing system 800 that may be configured to perform various method aspects described herein, including, for example, those described with reference to FIGS. 1-7.

[0138]

[0108] The processing system 800 includes a central processing unit (CPU) 802, which in some embodiments may be a multi-core CPU. Instructions executed in the CPU 802 may be loaded from a program memory associated with the CPU 802 or may be loaded from a memory partition 824, for example.

[0139]

[0109] The processing system 800 also includes additional processing components tailored to particular functions, such as a graphics processing unit (GPU) 804, a digital signal processor (DSP) 806, a neural processing unit (NPU) 808, a multimedia processing unit 810, and wireless connectivity components 812.

[0140]

[0110] An NPU, such as 808, is generally a specialized circuit configured to implement all the necessary control and computational logic to execute machine learning algorithms, such as algorithms for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), etc. An NPU may alternatively be referred to as a neural signal processor (NSP), a tensor processing unit (TPU), a neural network processor (NNP), an intelligence processing unit (IPU), a vision processing unit (VPU), or a graph processing unit.

[0141]

[0111] An NPU, such as 808, is configured to accelerate the execution of common machine learning tasks, such as image classification, machine translation, object detection, and various other predictive models. In some examples, multiple NPUs may be instantiated on a single chip, such as a system-on-chip (SoC), while in other examples, they may be part of a dedicated neural network accelerator.

[0142]

[0112] NPUs can be optimized for training or inference, or in some cases can be configured to balance performance between both. For NPUs capable of performing both training and inference, the two tasks can still generally be performed independently.

[0143]

[0113] NPUs designed to accelerate training are generally configured to accelerate the optimization of new models, which is a highly computationally intensive operation that involves inputting an existing dataset (often labeled or tagged), iterating over the dataset, and then adjusting model parameters, such as weights and biases, to improve model performance. In general, optimization based on mispredictions involves backpropagating through layers of the model to determine gradients to reduce prediction errors.

[0144]

[0114] NPUs designed to accelerate inference are generally configured to operate on a complete model. Thus, such NPUs may be configured to input new data and rapidly process the data through an already trained model to generate a model output (e.g., an inference).

[0145]

[0115] In one implementation, the NPU 808 is part of one or more of the CPU 802, the GPU 804, and / or the DSP 806.

[0146] In some embodiments, the wireless connectivity component 812 may include sub-components for, for example, third generation (3G) connectivity, fourth generation (4G) connectivity (e.g., 4G LTE), fifth generation connectivity (e.g., 5G or NR), Wi-Fi connectivity, Bluetooth connectivity, and other wireless data transmission standards. The wireless connectivity processing component 812 is further connected to one or more antennas 814.

[0147]

[0117] The processing system 800 may also include one or more sensor processing units 816 associated with any type of sensor, one or more image signal processors (ISPs) 818 associated with any type of image sensor, and / or a navigation processor 820, which may include satellite-based positioning system components (e.g., GPS or GLONASS), as well as inertial positioning system components.

[0148]

[0118] Processing system 800 may also include one or more input and / or output devices 822, such as a screen, a touch-sensitive surface (including a touch-sensitive display), physical buttons, a speaker, a microphone, etc.

[0149]

[0119] In some embodiments, one or more of the processors of processing system 800 may be based on the ARM or RISC-V instruction set.

[0150] Processing system 800 also includes memory 824, which represents one or more static and / or dynamic memories, such as dynamic random access memories, flash-based static memories, etc. In this example, memory 824 includes computer-executable components capable of being executed by one or more of the aforementioned processors of processing system 800.

[0151]

[0121] Specifically, in this example, memory 824 includes a spatial-depth component 824A, a point-wise convolution component 824B, an analysis component 824C, a training component 824D, a depth-spatial component 824E, a depth-wise convolution component 824F, an inference component 824G, and a set of model parameters 824H. These components may be configured according to one or more aspects described herein.

[0152]

[0122] For example, the spatial-to-depth component 824A and the depth-to-spatial component 824E may be configured to perform spatial-to-depth tensor transformations and depth-to-spatial tensor transformations, respectively, as part of one or more bottleneck blocks in a neural network.

[0153]

[0123] The point-wise convolution component 824B and the depth-wise convolution component 824F may be configured to perform point-wise convolution and depth-wise convolution, respectively, as part of one or more bottleneck blocks in the neural network.

[0154]

[0124] The analysis component 824C may be configured to determine a spatial expansion rate, determine whether to replace one or more conventional bottlenecks in the model with efficient bottleneck blocks (having spatial-depth conversion layers and depth-spatial conversion layers), and the like.

[0155] The training component 824D and the inference component 824G may generally be configured to train one or more models (e.g., to refine a set of model parameters 824H) and generate inferences using the models (e.g., using the trained model parameters 824H), respectively. For example, a bottleneck block such as that shown in Figures 3 and 4 may be trained by the training component 824D.

[0156]

[0126] The set of model parameters 824H may generally include parameters for one or more machine learning models (e.g., neural networks), including models that use the efficient bottlenecks described herein.

[0157]

[0127] The processing system 800 further comprises a spatial-to-depth circuit 826 and a depth-to-spatial circuit 828, which may be configured to perform spatial-to-depth and depth-to-spatial conversions, respectively, via hardware.

[0158]

[0128] Although shown as separate circuits in FIG. 8 for clarity, the spatial-depth circuit 826 and the depth-spatial circuit 828 may be implemented within other processing devices of the processing system 800, such as the CPU 802, GPU 804, DSP 806, NPU 808, etc.

[0159]

[0129] In general, the processing system 800 and / or its components may be configured to perform the methods described herein.

[0160]

[0130] Notably, in other aspects, aspects of the processing system 800 may be omitted, such as when the processing system 800 is a server computer, etc. For example, the multimedia components 810, the wireless connectivity 812, the sensors 816, the ISP 818, and / or the navigation components 820 may be omitted in other aspects. Furthermore, aspects of the processing system 800 may be distributed among multiple devices.

[0161]

[0131] The components shown, and other components not shown, may be configured to perform various aspects of the methods described herein.

[0162] Example clauses

[0132] Clause 1: A method comprising: receiving a first tensor; and processing the first tensor in a bottleneck block within a neural network model, the processing including applying a spatial-to-depth tensor transformation, applying a depth unit convolution, and applying a depth-to-spatial tensor transformation.

[0163]

[0133] Clause 2: The method described in Clause 1, wherein the spatial-depth tensor transformation is configured to reduce the spatial dimensionality of the input tensor to the spatial-depth tensor transformation by a spatial factor r and to increase the channel dimensionality of the input tensor to the spatial-depth tensor transformation by a spatial factor r, and wherein the volume of the output vector from the spatial-depth tensor transformation matches the volume of the input tensor to the spatial-depth tensor transformation.

[0164]

[0134] Clause 3: A method as described in clause 1 or 2, wherein the depth-to-space tensor transformation is configured to increase the spatial dimensionality of the input tensor to the depth-to-space tensor transformation by a spatial factor r and to decrease the channel dimensionality of the input tensor to the depth-to-space tensor transformation by a spatial factor r, and wherein the volume of the output vector from the depth-to-space tensor transformation matches the volume of the input tensor to the depth-to-space tensor transformation.

[0165]

[0135] Clause 4: A method according to any one of clauses 1 to 3, wherein the spatial-to-depth tensor transformation immediately follows the first point-wise convolution of the bottleneck block and immediately follows the depth-wise convolution.

[0166]

[0136] Clause 5: A method according to any one of clauses 1 to 4, wherein the depth-to-space tensor transformation immediately follows the second point-wise convolution in the bottleneck block.

[0167] Clause 6: The first point-wise convolution is

[0168]

number

[0169] kernels, where t is the channel expansion factor, r is the spatial expansion factor, and C in 6. The method of any one of clauses 1 to 5, wherein x is the number of channels in the first tensor.

[0170]

[0138] Clause 7: The output of the spatial-to-depth tensor transformation is dimensional

[0171]

number

[0172] where t is the channel expansion factor, r is the spatial expansion factor, and C in 7. The method of any one of clauses 1-6, wherein x is the number of channels in the first tensor, and H and W are spatial dimensions of the first tensor.

[0173]

[0139] Clause 8: A method according to any one of clauses 1 to 7, wherein the depth-to-space tensor transformation immediately precedes the second point-wise convolution in the bottleneck block.

[0174]

[0140] Clause 9: The output of the spatial-to-depth tensor transformation is the dimensionality

[0175]

number

[0176] where t is the channel expansion factor, r is the spatial expansion factor, and C in 9. The method of any one of clauses 1 to 8, wherein x is the number of channels in the first tensor, and H and W are spatial dimensions of the first tensor.

[0177]

[0141] Clause 10: A method according to any one of clauses 1 to 9, further comprising determining that an input tensor to the neural network model exceeds a spatial dimensionality threshold, and determining a spatial coefficient r based on a desired spatial dimensionality of the depth-wise convolution of the bottleneck block.

[0178]

[0142] Clause 11: A method described in any one of clauses 1 to 10, further comprising determining the computational capacity characteristics of a device that processes a bottleneck block in the neural network model, and replacing a segment of the neural network model with the bottleneck block based on the determined characteristics.

[0179]

[0143] Clause 12: A method described in any one of clauses 1 to 11, wherein the neural network model includes a plurality of bottleneck blocks, and among the plurality of bottleneck blocks, only the bottleneck block that is located at a later stage in the neural network model includes a spatial-to-depth tensor conversion layer.

[0180]

[0144] Clause 13: A system comprising a memory having computer-executable instructions and one or more processors configured to execute the computer-executable instructions to cause a processing system to perform a method according to any one of clauses 1 to 12.

[0181]

[0145] Clause 14: A system comprising means for carrying out the method according to any one of clauses 1 to 12.

[0182]

[0146] Clause 15: A non-transitory computer-readable medium comprising computer-executable instructions which, when executed by one or more processors of a processing system, cause the processing system to perform a method according to any one of clauses 1 to 12.

[0183]

[0147] Clause 16: A computer program product embodied on a computer-readable storage medium comprising code for performing the method according to any one of clauses 1 to 12.

[0184] Additional Considerations

[0148] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. The embodiments described herein are not intended to limit the scope, applicability, or aspects described in the claims. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. For example, changes may be made in the function and arrangement of the elements described without departing from the scope of the disclosure. Various embodiments may omit, substitute, or add various procedures or components as appropriate. For example, the methods described may be performed in an order different from that described, and various steps may be added, omitted, or combined. Also, features described with respect to some embodiments may be combined in some other embodiments. For example, an apparatus may be implemented or a method may be practiced using any number of the aspects described herein. Furthermore, the scope of the disclosure is intended to encompass apparatuses or methods that are implemented using other structures, functions, or structures and functions in addition to or other than the various aspects of the disclosure described herein. It should be understood that any aspect of the disclosure disclosed herein may be embodied by one or more elements of a claim.

[0185]

[0149] As used herein, the word "exemplary" means "serving as an example, instance, or illustration." Any aspect described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other aspects.

[0186]

[0150] As used herein, a phrase referring to "at least one of" a list of items refers to any combination of those items, including single members. By way of example, "at least one of a, b, or c" is intended to encompass a, b, c, ab, ac, bc, and abc, as well as any combination having multiple identical elements (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other permutation of a, b, and c).

[0187]

[0151] As used herein, the term "determining" encompasses a wide variety of actions. For example, "determining" may include calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, database, or another data structure), ascertaining, and the like. Also, "determining" may include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. Also, "determining" may include resolving, selecting, electing, establishing, and the like.

[0188]

[0152] As used herein, the term "connected to" may generally have the meaning in the context of sharing electronic signals and data between elements described herein in data communication between corresponding elements connected to each other. In some cases, elements may be directly connected to each other, for example, via one or more conductive traces, lines, or other conductive carriers that can carry signals and / or data between corresponding elements directly connected to each other. In other cases, elements may be indirectly connected to each other, for example, via one or more data buses or similar shared circuitry and / or integrated circuit elements for communicating signals and data between corresponding elements indirectly connected to each other.

[0189]

[0153] The methods disclosed herein include one or more steps or actions for achieving the method. The steps and / or actions of the methods may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order of specific steps and / or actions and / or the use of those steps and / or actions may be modified without departing from the scope of the claims. Furthermore, various operations of the methods described above may be performed by any suitable means capable of performing the corresponding functions. Those means may include various hardware and / or software component(s) including, but not limited to, circuits, application specific integrated circuits (ASICs), or processors, and / or various hardware and / or software module(s). In general, when operations are illustrated in figures, those operations may have corresponding equivalent means-plus-function components that are similarly numbered.

[0190]

[0154] The following claims are not intended to be limited to the embodiments set forth herein, but are to be accorded the full scope consistent with the language of the claims. Within the claims, reference to an element by the singular is not intended to mean "only one" unless expressly stated as such, but rather "one or more." Unless otherwise expressly stated, the term "several" refers to one or more. No element of a claim is to be construed under the provisions of 35 U.S.C. 112(f) unless the element is expressly recited using the phrase "means of" or, in the case of a method claim, unless the element is recited using the phrase "step of." All structural and functional equivalents of the elements of the various embodiments described throughout this disclosure that are known or later become known to those skilled in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Furthermore, nothing disclosed herein is intended to be made public, whether or not such disclosure is expressly recited in the claims.

Claims

1. 1. A computer-implemented method comprising: receiving a first tensor; processing the first tensor in a bottleneck block within a neural network model, applying a space-to-depth tensor transform configured to reconstruct an input tensor into a space-to-depth tensor transform; applying depthwise convolution; and processing, the processing including applying a depth-to-space tensor transform configured to reconstruct the input tensor into a depth-to-space tensor transform.

2. The space-to-depth tensor transformation is reducing the spatial dimensionality of the input tensor to the space-to-depth tensor transform by a spatial factor r; configured to increase the channel dimensionality of the input tensor to the space-to-depth tensor transform by the spatial factor r; The method of claim 1 , wherein the volume of the reconstructed input tensor that is the output tensor from the space-to-depth tensor transform corresponds to the volume of the input tensor to the space-to-depth tensor transform.

3. The depth-to-space tensor transformation is increasing the spatial dimensionality of the input tensor to the depth-to-spatial tensor transform by a spatial factor r; configured to reduce the channel dimensionality of the input tensor to the depth-to-spatial tensor transform by the spatial factor r; The method of claim 1 , wherein the volume of the reconstructed input tensor that is the output tensor from the depth-to-space tensor transform corresponds to the volume of the input tensor to the depth-to-space tensor transform.

4. The method of claim 1 , wherein the spatial-to-depth tensor transform immediately follows the first point-wise convolution of the bottleneck block and immediately follows the depth-wise convolution.

5. The method of claim 4 , wherein the depth-to-space tensor transform immediately follows a second point-wise convolution of the bottleneck block.

6. The first point-wise convolution [Equation 1] kernels, where t is the channel expansion factor, r is the spatial expansion factor, and C ｉｎ The method of claim 5 , wherein σ is the number of channels in the first tensor.

7. The output of the space-to-depth tensor transformation is the dimensionality [Equation 2] where t is the channel expansion factor, r is the spatial expansion factor, and C ｉｎ 6. The method of claim 5, wherein ∑ i = 1 ⁢ ... is the number of channels in the first tensor, and H and W are the spatial dimensions of the first tensor.

8. The method of claim 4 , wherein the depth-to-space tensor transform immediately precedes a second point-wise convolution of the bottleneck block.

9. The output of the space-to-depth tensor transformation is the dimensionality [Equation 3] where t is the channel expansion factor, r is the spatial expansion factor, and C ｉｎ 9. The method of claim 8, wherein ∑ i = 1 i ⁢ ... is the number of channels in the first tensor, and H and W are the spatial dimensions of the first tensor.

10. determining that an input tensor to the neural network model exceeds a spatial dimensionality threshold; The method of claim 1 , further comprising: determining a spatial factor r based on a desired spatial dimensionality of the depthwise convolution of the bottleneck block.

11. determining a computational capability characteristic of a device processing the bottleneck block in the neural network model; The method of claim 1 , further comprising: replacing a segment of the neural network model with the bottleneck block based on the determined characteristics.

12. 2. The method of claim 1, wherein the neural network model includes a plurality of bottleneck blocks, and only a bottleneck block of the plurality of bottleneck blocks that is later in the neural network model includes a spatial-to-depth tensor transformation layer.

13. 13. A non-transitory computer-readable medium containing computer-executable instructions that, when executed by one or more processors of a processing system, cause the processing system to perform the method of any one of claims 1 to 12.

14. 1. A processing system comprising: The non-transitory computer-readable medium of claim 13; one or more processors configured to execute the computer-executable instructions; A processing system comprising:

Images