Neural network architecture for implementing group convolutions

Abstract

A method, system, and apparatus are provided for processing an input image using a convolutional neural network (CNN). The CNN includes a sequence of layer blocks. Each of a first subset of layer blocks in the sequence performs an operation including receiving an input feature map for the layer block (706), generating an expanded feature map from the input feature map using group convolution (708), generating a reduced feature map from the expanded feature map (710), and generating an output feature map for the layer block from the reduced feature map. The input feature map is an h×w feature map with a c1 channel. The expanded feature map is an h×w feature map with a c2 channel, while the reduced feature map is an h×w feature map with a c1 channel, where c2 is greater than c1.

Description

[Technical Field]

[0001] background This specification generally relates to using integrated hardware circuits to perform group convolutions of convolutional neural networks. [Background technology]

[0002] A neural network is a machine learning model that uses one or more layers of nodes to generate an output, such as a classification, for a received input. Some neural networks include one or more hidden layers in addition to an output layer. Some neural networks may be convolutional neural networks configured for image processing or recurrent neural networks (RNNs) configured for speech and language processing. Different types of neural network architectures can be used to perform a variety of tasks related to classification or pattern recognition, prediction involving data modeling, and clustering of information.

[0003] A neural network layer can have a corresponding set of parameters or weights. The weights are used to process inputs (e.g., batches of inputs) through the neural network layer and generate the layer's corresponding outputs to compute neural network inference. A batch of inputs and a set of kernels can be represented as input and weight tensors, or multidimensional arrays. A hardware accelerator is a dedicated integrated circuit for implementing neural networks. The circuit contains memory with locations corresponding to the elements of the tensors that can be traversed or accessed using the circuit's control logic. Summary of the Invention [Problem to be solved by the invention]

[0004] overview This specification describes techniques for efficiently implementing group convolution on a hardware neural network accelerator. Group convolution convolves input feature maps by grouping them along the channel dimension of the input matrix, and each input group representing a group convolution is associated with a corresponding output group. In particular, based on these techniques, leveraging group convolution to process input images using a machine learning model, a convolutional neural network (CNN), implemented on an exemplary computing device such as a tablet or smartphone can achieve certain hardware and computing efficiencies. [Means for solving the problem]

[0005] For example, an input image is obtained for processing using a CNN. The CNN includes a sequence of layer blocks, each of which is configured to perform operations including: i) receiving an input feature map of the layer block; ii) generating an expanded feature map from the input feature map using group convolution; and iii) generating a reduced feature map from the expanded feature map. The input feature map of the layer block is an h×w feature map with a c1 channel. The expanded feature map is an h×w feature map with a c2 channel, while the reduced feature map is an h×w feature map with a c1 channel, where c2 is greater than c1. An output feature map of the layer block is generated from the reduced feature map.

[0006] One aspect of the subject matter described herein may be embodied in a method implemented by one or more computers. The method includes receiving an input image and processing the input image using a convolutional neural network. The convolutional neural network includes a sequence of layer blocks. Each of a first subset of layer blocks in the sequence includes receiving an input feature map for the layer block, the input feature map being an h×w feature map with a c1 channel. The first subset further includes generating an expanded feature map from the input feature map using group convolution, the expanded feature map being an h×w feature map with a c2 channel, where c2 is greater than c1. The first subset further includes generating a reduced feature map from the expanded feature map, the reduced feature map being an h×w feature map with a c1 channel. The first subset is further configured to perform an operation including generating an output feature map for the layer block from the reduced feature map.

[0007] These and other implementations may each optionally include one or more of the following features. For example, in some implementations, generating the augmented feature map includes generating an initial augmented feature map from the input feature map by applying a 1×1 convolution to the input feature map, where the initial augmented feature map is an h×w feature map with c2 channels. Generating the augmented feature map further includes generating the augmented feature map from the initial augmented feature map by applying a group convolution to the initial augmented feature map.

[0008] In some implementations, a 1x1 convolution has more output filters than input filters. In a group convolution, the total number of input and output filters can be the same. A sequence of layer blocks can include group convolution layer blocks interleaved with non-group convolution layer blocks, and the group convolution layer blocks are used to implement the group convolution. In some implementations, the group convolution is a fused group convolution implemented using a fused grouped inverse bottleneck (IBN) layer included within the sequence of layer blocks.

[0009] Generating the augmented feature map can include generating the augmented feature map from the input feature map by applying a group convolution to the input feature map. In some implementations, generating the augmented feature map includes generating an initial augmented feature map from the input feature map by applying a 1×1 convolution to the input feature map. The initial augmented feature map is an h×w feature map having c3 channels, where c3 is greater than c2. Generating the augmented feature map can further include generating the augmented feature map from the initial augmented feature map by applying a group convolution to the initial augmented feature map.

[0010] Other implementations of this and other aspects include corresponding systems, apparatus, and computer programs configured to perform the actions of the methods and encoded on computer storage devices. One or more computer systems may be so configured by software, firmware, hardware, or a combination thereof installed on the systems, and the systems perform the actions during operation. One or more computer programs may be so configured by having instructions that, when executed by a data processing device, cause the devices to perform the actions. [Effects of the Invention]

[0011] The subject matter described herein can be implemented in certain embodiments and can achieve one or more of the following advantages: The group convolution techniques described in this document provide a new convolution architecture with various combinations of group convolution-based neural blocks. Compared to existing uses of group convolution, Convolutional neural blocks can be interleaved with other block types, allowing for finer control over hardware resource utilization metrics and computational efficiency of the exemplary ML hardware accelerator.

[0012] The group convolutional neural block in this architecture is a variation of an inverse bottleneck-style neural block and is implemented using dedicated processors in a variety of devices, such as mobile computing devices or edge computing platforms. The architecture incorporates various group convolution configurations, including fused or grouped variants of baseline inverse bottleneck ("IBN") layers, to implement group convolutions along the channel dimension of the input feature map corresponding to the input image. Group convolution techniques can provide neural architectures with group convolutional layer blocks interleaved with non-group convolutional layer blocks.

[0013] Interleaving non-group convolution and group convolution based neural blocks provides an improved neural architecture for more efficiently processing input images, such as when performing computer vision tasks involving convolutional neural network computations. For example, compared to K x K depth-wise convolutions (i.e., non-group convolutions), neural blocks implementing K x K group convolutions can achieve a more efficient hardware mapping of computations. The mapping is specific to a given hardware layout of arithmetic circuits within a dedicated processor implementing a convolutional neural network. This allows the computation of group convolution layers to be arranged in a manner that is optimized for hardware utilization, processing latency, or the capacity of the integrated circuit's operands (e.g., inputs and weights).

[0014] This architecture can use different types of group convolution-based neural blocks to apply group convolutions to different groupings of inputs along the channel dimension of the input tensor. For example, rather than a one-to-one relationship between input and output channels, the system leverages the concept of blocks to perform group convolutions using different groupings of inputs along the input channels within a group. This can benefit algorithms that can use more information along the input channels, improving the representational power of one or more layers of a computer vision network.

[0015] Group convolution techniques involve automated (or manual) evaluation of different configurations of group convolutional neural network blocks to achieve various types of neural architectures for different computer vision tasks. An exemplary system implementing these techniques can determine a neural architecture that optimizes model performance for constraints such as latency, parameter size, number of computational operations, and model accuracy. Model performance can also be optimized for a given hardware integrated circuit layout of the machine learning accelerator used to run the model.

[0016] The details of one or more implementations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other potential features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a block diagram of an example computing system for performing group convolution on an image. [Figure 2] FIG. 1 is a block diagram illustrating an example grouping used for group folding. [Figure 3] FIG. 1 illustrates exemplary attributes of a machine learning model for different convolution operations. [Figure 4] FIG. 1 is a block diagram illustrating the operations corresponding to different layer blocks of a convolutional neural network. [Figure 5] FIG. 2 is a diagram of an example architecture of a convolutional neural network model that can be used in the example computing system of FIG. 1. [Figure 6] FIG. 1 illustrates an exemplary loop nest for computing full and group convolutions. [Figure 7] FIG. 1 is a flow diagram of an exemplary method used to process images using group convolution. DETAILED DESCRIPTION OF THE INVENTION

[0018] Like reference numbers and designations in the various drawings indicate like elements. Detailed Description 1 is a block diagram of an exemplary computing system 100 for performing group convolution on an input image. The system 100 generally includes an exemplary convolutional neural network 102 configured to process an image 104, i.e., process intensity values ​​of pixels of the image. The convolutional neural network 102 includes an exemplary neural network architecture based on multiple convolutional neural network layers 108. In the example of FIG. 1, the convolutional neural network 102 includes multiple convolutional neural network layers 108. For example, the convolutional neural network 102 includes N layers (or N sets of N layers), where N is an integer greater than 1.

[0019] Various types of CNN architectures 106 can be used to perform various machine learning tasks. For example, the machine learning task can be a computer vision task (also called an "image processing task"). In other words, the neural network can be configured to receive an input image and process the input image to generate a network output for the input image, i.e., to perform some image processing task. As used herein, processing an input image refers to using a neural network to process intensity values ​​of pixels in the image. For example, the task can be image classification, and the output generated by the neural network for a given image can be a score for each of a set of target categories, with each score representing an estimate of the likelihood that the image contains an image of an object belonging to that category.

[0020] As another example, the task may be image embedding generation, where the output generated by the neural network may be a numerical embedding of the input image. As yet another example, the task may be object detection, where the output generated by the neural network may identify locations within the input image, e.g., bounding boxes or other geometric regions within the image where a particular type of object is depicted. As yet another example, the task may be image segmentation, where the output generated by the neural network may define, for each pixel in the input image, to which of multiple categories the pixel belongs. However, more generally, the task may be any of a variety of tasks, including tasks that process inputs other than images.

[0021] Some image processing tasks may involve object detection, data classification, pattern recognition, or image recognition, as well as computational prediction involving data modeling and information clustering. For example, a task may involve object detection, where a CNN processes an image to detect a particular object and, upon detection, produces an output that identifies the object. Another task may involve data / image classification, where a CNN processes an image to determine the classification of the image and classifies the image based on its content. and producing a specific classification output. Another task may include pattern recognition, in which a CNN processes images to identify or recognize specific patterns within the images and produces an output indicating the recognized patterns based on the content of the images. Another task may include general image recognition, in which a CNN processes images to identify or recognize various elements of the images and produces an output indicating the recognized elements based on the content of the images.

[0022] In some implementations, the convolutional neural network 102 is implemented in or accessible by an exemplary mobile device 110. The mobile device 110 may be a smartphone, tablet, electronic notebook, laptop, game console, or related portable computing device. In other implementations, the convolutional neural network 102 is integrated into or accessible by an exemplary cloud-based system, such as a server bank, server group, or multiprocessor system.

[0023] The convolutional neural network 102 can be implemented using one or more machine learning hardware accelerators 112. Each hardware accelerator 112 corresponds to one or more dedicated hardware integrated circuits 114. Generally, the circuits 114 are hardware circuits (e.g., dedicated hardware circuits) that perform neural network calculations. For example, some (or all) of the circuits 114 can be dedicated hardware circuits such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a single-core neural network processor, or a multi-core neural network processor. The circuits 114 can also be dedicated graphics processing units (GPUs).

[0024] The hardware circuitry 114 is operable to accelerate the computation of neural network workloads. In some implementations, the hardware circuitry 114 includes control logic, which may be implemented in hardware, software, or both. The control logic is used to issue instructions for the neural network computation, including retrieving and routing data used in the computation. The circuitry 114 may include memory for storing inputs, input activations, outputs, output activations, and parameters for each layer of the neural network. In some implementations, the circuitry 114 includes dedicated memory, shared memory, or both. For example, the circuitry 114 may include an input / activation memory for storing inputs, input activations, outputs, or output activations, and a parameter memory for storing respective parameter sets for each neural network layer.

[0025] The circuit 114 may include a computational device, such as a hardware matrix unit, an arrangement of computational tiles, or a combination thereof. The computational device is used to perform neural network computations to process inputs through layers of the neural network. In some implementations, each matrix unit or individual computational tile includes one or more arrays of computational cells, such as multiply-accumulate cells, that perform multiplication and accumulation operations. For example, each cell may perform a multiplication of an input and a weight value to generate a product and accumulate the product (e.g., an addition operation) over multiple clock cycles.

[0026] The circuit 114 implements full convolution, depth-wise convolution, and group convolution to convolve filters with different weights on corresponding portions of the input matrix for a given depth in the channel dimension of the input matrix. For example, the mobile device 110 uses the convolutional neural network 102 and the CNN layer 108 of the model to generate an image processing output 120, such as a recognition output or detection output, for a received input 104. For example, the input 104 may be an image of a laptop 122, and the mobile device 110 may use the convolutional neural network The image is processed using the network 102 to detect or recognize that the image contains a depiction of a laptop.

[0027] FIG. 2 is a block diagram including a representation of an input dataset 202 and an exemplary grouping 203 for performing group convolution using inputs from the input dataset. In some implementations, the input dataset 202 is or is derived from a multidimensional matrix structure of inputs. For example, the matrix structure may be an input tensor containing Z channels, each with spatial dimensions X×Y. The matrix structure (or tensor) can represent either a set of inputs, a set of activation inputs, or a set of weight inputs. In some cases, the matrix structure of a set of activation inputs is referred to herein as an input feature map, and the matrix structure of a set of weight inputs is referred to herein as a kernel matrix structure.

[0028] In the example of FIG. 2, the input dataset 202 is a matrix structure (or tensor) with three dimensions: two (X, Y) spatial dimensions and one (Z) channel dimension. With respect to spatial dimensions, in some implementations, these dimensions correspond to the space or location of a set of activation inputs. For example, if the convolutional neural network 102 is processing a two-dimensional image 104, the matrix structure may have two spatial dimensions corresponding to the spatial coordinates of the image, i.e., X and Y coordinates. With respect to the channel dimension, this dimension corresponds to features from the input (e.g., activation input). The channel dimension is described with reference to Z, Zin, or channel dimension. Here, a "channel" may correspond to a color channel of an image.

[0029] The system 100 is configured to determine the group convolution divisions, for example, by referring to the depth level of the channel dimensions of the input dataset 202. Each input channel may have a corresponding depth level. For example, the matrix structure of FIG. 2 has depth levels extending along the Z dimension. As an example, if the exemplary matrix structure 202 represents a 3×3×3 image sent as a set of activation inputs to a convolutional neural network layer, the X and Y dimensions of the image (3×3) may be spatial dimensions, and the Z dimension (3) may be a channel dimension corresponding to R, G, and B values.

[0030] As described above, the system 100 can determine partitions for the group convolutions along the channel dimension of an exemplary input feature map. For example, the system 100 can determine a first partition of the input group 210-1 along the channel dimension and a second partition of the input group 210-2 along the channel dimension. In some implementations, the system 100 determines n groupings 210-n along the channel dimension, where n is an integer greater than or equal to 1. In an example where the input feature map 202 represents a 3×3×3 image to be sent as a set of activation inputs, the first partition defining the input group 210-1 for the group convolution can correspond to features with nine “1” activation inputs (e.g., red values), the second partition defining the input group 210-2 for the group convolution can correspond to features with nine “2” activation inputs (e.g., green values), and the third partition defining the input group 210-3 for the group convolution can correspond to features with nine “3” activation inputs (e.g., blue values).

[0031] As described above, group convolution convolves input feature maps by grouping them along the channel dimension of the input matrix, and each input group 210-n representing a group convolution is associated with a corresponding output group 220-n. The convolutional neural network 102 uses one or more convolutional neural network layers 108 to generate outputs 206 (e.g., classifications) for the received inputs 202. For example, each convolutional neural network layer is associated with a set of kernels 204. The kernels 204 may be partitioned according to the configuration of the group convolution, so that each input group 210-n is convolved with a corresponding kernel / weight matrix to generate a convolutional output 220-n. In the example of FIG. 2, the input group 210-1 is convolved with a corresponding kernel matrix 212. Input group 210-1 is convolved with corresponding kernel matrix 214 to generate convolution output 220-2, while input group 210-2 is convolved with corresponding kernel matrix 214 to generate convolution output 220-2.

[0032] The system 100 is configured to dynamically determine the value of a control parameter g, where g is an integer greater than 1. The system 100 is also configured to determine a group size by calculating Zin / g, where Zin is the number of input channels along the channel dimension of the input tensor and g is the number of groups defined by the control parameter. The control parameter g is used to define the number of group convolutions (e.g., splits). In some cases, the value of g may be dynamically determined by the system 100 or may be predefined by the system 100 for a given operation. For example, the control parameter g, which defines the number of group convolutions, may be predefined (and / or embedded) by a compiler for the system 100 or may be dynamically determined at runtime.

[0033] In some implementations, the system 100 defines the number of group convolutions (e.g., splits) based on the particular type of machine learning task required and sets the value of the control parameter g accordingly. In other implementations, the system 100 defines the number of group convolutions (e.g., splits) based on i) the type of machine learning task being processed, ii) the neural architecture of the convolutional neural network, iii) the computing environment, iv) performance goals, or v) a combination thereof. Exemplary computing environments may include cloud-based computing environments and mobile device computing environments. Performance goals may include speed, latency, hardware utilization, model accuracy, parameter size, or a combination thereof.

[0034] Group convolution can be described as a generalized form of convolution. In some implementations, the system 100 initializes a control parameter g by assigning a particular value to the control parameter. The initialized or assigned value of the control parameter g can be used to control the division of the group convolution. For example, if the system 100 determines that a convolution operation using data across the channel dimensions (e.g., a full convolution) is required, the system 100 sets the value of the control parameter g=1 and triggers and / or performs a full convolution using the associated data in the matrix structure 202.

[0035] Relatedly, system 100 can determine the grouping of depth-separable convolutions required for a given step in a larger neural network computation. For example, if system 100 determines that two or more depth-separable convolutions using data from a portion of the channel dimension are required, system 100 sets a control parameter to a desired value (e.g., g=4) and triggers and / or executes two or more (e.g., four) depth-separable convolutions using the associated data portions in matrix structure 202. In some implementations, the computations for two or more group convolutions are executed sequentially, simultaneously, or a combination thereof. For example, some (or all) of the respective sets of computations for each of two or more depth-separable convolutions can be executed sequentially or in parallel.

[0036] As noted above, the group / grouped convolution techniques described herein provide at least finer control over the utilization metrics and computational efficiency of the hardware resources of an exemplary ML accelerator. In some implementations, these group convolution techniques provide a multipurpose block or control knob that is used to influence and control certain attributes or performance metrics of an exemplary machine learning model. For example, selecting a value for the control parameter g between 1 and the number of channels (z) provides a continuum between two exemplary constraints: full convolution and depth-wise separable convolution, as described in more detail below.

[0037] 3 illustrates exemplary attributes of a machine learning model. Generally, the attributes correspond to various convolution operations performed using the convolutional neural network 102 described above. For example, attribute 302 indicates the number of parameters and multiply-accumulate cells (MACs) used to perform a full convolution operation, attribute 304 indicates the number of parameters and multiply-accumulate cells used to perform a depth-wise convolution operation, and attribute 306 indicates the number of parameters and multiply-accumulate cells used to perform a group convolution operation.

[0038] The control parameter g and the configuration of the group convolutions can be determined and / or adjusted to control the number of parameters (e.g., trainable parameters) used for a given task as well as the number of multiply-accumulate cells used to perform the task's operations. Each of these example attributes 302, 304, 306 of a machine learning model has or can have a corresponding impact on various performance metrics of the model. For example, increasing or decreasing the number of trainable parameters and / or the number of multiply-accumulate cells (or operations) has a corresponding impact on the accuracy, speed, and / or latency of the machine learning model. In another example, while using deep-by-depth convolutions can be a lightweight and low-cost (i.e., less resource-intensive) option compared to full convolutions, performing deep-by-depth convolutions on an ML accelerator's integrated circuit often results in lower utilization of the circuit's hardware resources.

[0039] For example, when performing a depth-wise (or depth-separable) convolution, a standard hardware array of circuit 114 containing tens or hundreds of hardware multiply-accumulate cells may have a 3% utilization of those hardware cells in a given computation cycle, while incurring minimal or low latency. Thus, using a depth-wise convolution can be fast, but also inefficient due to low hardware utilization. Conversely, when performing a full convolution, the hardware array of circuit 114 may have a significantly higher utilization (e.g., 73%), resulting in a majority of the array's multiply-accumulate cells being used in a given computation cycle. Compared to a depth-wise convolution, this higher utilization when performing a full convolution often comes at the cost of significantly higher computation latency.

[0040] As mentioned above, the group convolution technique described herein provides finer control over the utilization metrics and computational efficiency of hardware resources of an exemplary ML hardware accelerator. Selecting a value for the control parameter g between 1 and the number of channels (z) provides a continuum between two exemplary constraints: full convolution (308) and depthwise separable convolution (310). The system 100 can determine the partitioning of group convolutions with reference to the depth level of the channel dimension, as shown in the example of FIG. 2. The control parameter g is used to define the number of group convolutions (e.g., partitions).

[0041] 3 illustrates an example number of parameters 320 and number of MACs 322 for a choice of different values ​​of g (324) between 2 and the number of channels (z) along the continuum between full convolution (308) and depth-wise convolution (310). In this example, the z dimension is 256. Graph 312 illustrates an example of a decrease in the number of trainable parameters and the number of multiplication-accumulation cells (or operations) for a corresponding increase in the value of group convolution (g).

[0042] As described above, the circuit 114 may include a memory with locations corresponding to elements of a tensor that can be traversed or accessed using the control logic of the circuit to compute the output of a layer, such as a group convolutional layer. The elements (e.g., inputs or activations) retrieved from the memory may be needed to compute multiple outputs of the layer. The number of weights (i.e., parameters) may also vary depending on the size of the grouping. In some implementations, the memory may include a tensor that can be traversed or accessed using the control logic of the circuit to compute the output of a layer, such as a group convolutional layer. The transfer of parameters from memory can become a bottleneck, increasing computational latency. In determining a preferred neural network architecture, an example set of search data or simulations can indicate a bottleneck related to parameter transfer time. The disclosed group convolution concepts and group convolution-based neural blocks can then be used to define architectures that reduce the number of parameters and improve or accelerate the computation time of machine learning tasks.

[0043] FIG. 4 is a block diagram illustrating example process blocks 410, 420, and 430. Each process block 410, 420, and 430 includes one or more layer blocks. In general, each process block 410, 420, and 430 may be represented by a different layer block of a convolutional neural network. In the example of FIG. 4, each of process blocks 410, 420, and 430 may be a subset of operations performed for a given convolutional operation. The convolutional operation is performed using a convolutional neural network 102, which may be implemented on the example hardware integrated circuit 114 described above.

[0044] A neural network block can describe a single layer or a component of a neural network that includes multiple layers. A common block widely used in exemplary computer vision models, such as mobile vision models, is the inverse bottleneck (IBN) layer block 402 (“IBN layer 402”). In general, an IBN block may be a macroblock of a larger neural architecture that combines multiple convolutional layers in some way. Multiple types of layers (or blocks), including IBN layers, are used as building blocks to form exemplary classification or object detection networks.

[0045] The IBN layer 402 can include a point-wise convolution (404), a K x K depth-wise convolution (405), and a final point-wise convolution (406). The point-wise convolution expands the channel dimension, and an example of this point-wise convolution is shown in FIG. 4 as "1 x 1 Convolution (Expansion)." A K x K depth-wise convolution kernel is applied to the expanded depth of the channel dimension following the point-wise convolution. The final point-wise convolution (406) projects the expanded channel dimension to a smaller value. An example of this final point-wise convolution is shown in FIG. 4 as "1 x 1 Convolution (Projection)."

[0046] The use of K×K depth-wise convolutions, such as the IBN layer block 402, is very common because computing full convolutions across large or extended channel dimensions after dilation is very costly in terms of processing and computational resources. In some implementations, the pointwise convolutions (404) and the K×K depth-wise convolutions (405) are replaced by a K×K full convolution (fused dilation) process block, representing the fused IBN layer 407. Generally, the fused IBN layer 407 combines the dilation and depth-wise convolution operations into a single fully convolutional neural block.

[0047] Full convolution may involve a large number of parameters / weights and may require a significant proportion of the integrated circuit's hardware computational resources. As previously mentioned, examples of such resources may include multiply-accumulate cells in a hardware computational array (e.g., a systolic array) of circuit 114, a vector unit of integrated circuit 114, or both. In contrast, the disclosed group convolution techniques implemented using the disclosed neural block alternatives, such as blocks 414, 416, 422, and 432 described below, provide an improved approach for increasing the number of trainable parameters for a set of input channels (e.g., large input channels), thereby improving model accuracy, but at a lower computational cost compared to non-group convolution alternatives.

[0048] Referring now to process block 410, a grouped IBN progressive projection (or progressive expansion) block is shown, where the K×K depth-wise convolution (405) described above is replaced with K×K group convolutions (414) or (416). Process block 410 can have a first instance that implements the K×K group convolutions (414) to perform progressive projection of the channel dimension, or a second instance that implements the K×K group convolutions (416) to perform progressive expansion of the channel dimension.

[0049] In a first example of process block 410, the system 100 can generate an augmented feature map from an input feature map (e.g., input 438) by applying a 1×1 convolution (dilation) (404) to the input feature map. The input feature map can be an h×w feature map with c1 channels. The augmented feature map can be an h×w feature map with c2 channels, where c2 is greater than c1. In some implementations, the 1×1 convolution has more output filters than input filters. A K×K group convolution (414) is applied to the augmented feature map to perform a progressive projection of the channel dimension. For example, the convolutional neural network 102 can perform a progressive projection on the augmented feature map using a group convolution implemented in a group convolution layer of the convolutional neural network 102. The grouped IBN progressive projection can provide flexibility to trade off parameters dedicated to the projection with the main K×K convolution operator.

[0050] In this first example of process block 410, the final pointwise convolution (406) projects the expanded channel dimension to a smaller value. Thus, the K×K kernel associated with the group convolution can perform an initial reduction in channel size before the 1×1 projection (406) reduces the channel size to its final value. Each summation block 418 is an optional residual (or skip) connection that can be used to sum the example convolution output 436 with the input 438 provided to a given process block (e.g., 410). An example sum 440 is passed as the output of the operation performed in the corresponding process block.

[0051] In a second example of process block 410, the system 100 can generate an initial augmented feature map from an input feature map (e.g., input 438) by applying a 1×1 convolution (dilation) (404) to the input feature map. This initial augmented feature map can be an h×w feature map with c2 channels, where c2 is greater than c1. The system 100 generates an augmented feature map from the initial augmented feature map by applying a K×K group convolution (416) to the initial augmented feature map. For example, the convolutional neural network 102 can generate an augmented feature map from the initial augmented feature map using group convolutions implemented in a group convolution layer of the convolutional neural network 102. The augmented feature map can be an h×w feature map with c3 channels, where c3 is greater than c2. This grouped IBN incremental dilation operation can provide flexibility in trading off parameters dedicated to the dilation with the main K×K convolution operator. Grouped IBN progressive extension may keep some of the extension layers unfused and allow channel-wise convolution between groups before the main K×K convolution. The final point-wise convolution (406) in process block 410 projects the extended channel dimensions to smaller values.

[0052] Referring now to process block 420, this process block is a fused grouping IBN block in which the above-described 1×1 convolution (dilation) (404) and K×K depth-wise convolution (405) are replaced with a K×K group convolution (422). This K×K group convolution (422) includes the designation "fused and extended" because it can at least replace the point-wise (404) + depth-wise (405) pair and fuse aspects of their operations via the K×K group convolution (422) to extend the channel dimension. Thus, in process block 420, the system 100 applies a K×K group convolution to the input feature map. By applying a loop convolution (422), an augmented feature map can be generated from an exemplary input feature map (e.g., input 438). The exemplary input feature map can be an h×w feature map with a c1 channel. The augmented feature map can be an h×w feature map with a c2 channel, where c2 is greater than c1. The final point-wise convolution (406) in process block 420 projects the augmented channel dimension to a smaller value. As previously mentioned, the corresponding sum 440 is passed as the output of the particular operation performed in process block 420.

[0053] In some implementations, the fused group convolution block 422 provides an alternative to the fused IBN layer 407, enabling more efficient processing along the channel dimension. For example, these efficiencies may be realized in later stages of the computer vision model. In some cases, these later stages correspond to cases where the data resolution associated with convolution along the channel dimension is very large. The processing speed improvements obtained by fused group convolution may be particularly optimized when the process block 420 containing the group convolution operation is implemented using a particular type of dedicated integrated circuit. For example, the dedicated integrated circuit may be a neural network processor that includes a broadcast input bus that broadcasts layer inputs from memory to one or more computational cells of the circuit.

[0054] The fused group convolution block 422 may require a slightly higher number of parameters than the grouped IBN layer 414. On the continuum between the two constraints of full convolution and depthwise separable convolution, the fused group IBN 422 is higher on the continuum. For example, the fused grouped IBN layer 422 may be closer to full convolution along the continuum from depthwise convolution to full continuum.

[0055] Referring now to process block 430, this process block is a grouped IBN block in which the K×K depth-wise convolutions (405) described above are replaced with K×K group convolutions (432). As described above, system 100 applies 1×1 convolutions (404) to inputs 438 to generate the expanded feature maps. The K×K group convolutions (432) are applied in the group convolution layer of convolutional neural network 102. The K×K group convolutions (432) allow the total number of input and output filters to be the same. As with the other process blocks, the final point-wise convolutions (406) in process block 430 project the expanded channel dimension to smaller values, and the corresponding sum 440 is passed as the output of the particular operation performed in process block 430.

[0056] The convolution operations performed in process block 430 can involve smaller expansion ratios compared to the baseline IBN layer. These smaller expansion ratios can reduce the number of parameters. To recover the number of parameters, the convolution operations in process block 430 (and other process blocks) can use group convolutions with K×K kernels that exploit inter-channel information. The K×K group convolutions (432) can be interleaved with other block types that include convolutions along the input channel dimension. This interleaving pattern can mitigate the lack of input channel convolutions between groups.

[0057] In general, each of the architectures of process blocks 410 and 430 replaces K×K depth-wise convolutions with K×K group convolutions. At least one advantage of replacing K×K depth-wise convolutions with K×K group convolutions is that the K×K group convolutions reduce latency and generate more trainable parameters compared to full convolutions. The additional trainable parameters from using K×K group convolutions contribute to improved model accuracy. This improvement in accuracy is achieved with only a small or minimal increase in latency compared to depth-wise convolutions. Cut.

[0058] Replacing depth-wise convolutions with group convolutions may be specific to the convolution operations of certain types of hardware accelerators, such as tensor processing units (TPUs) configured for mobile devices or edge computing applications. In some implementations, K×K group convolutions, relative to K×K depth-wise convolutions, may be configured to achieve a more efficient hardware mapping in terms of the hardware layout of the integrated circuit 114. For example, rather than a one-to-one relationship between input and output channels, group convolutions can leverage the concept of blocks to perform convolutions along input channels in groups. This benefits algorithms that can use more information along the input channels, improving the representational capabilities of one or more layers of a computer vision network.

[0059] As the computation of a machine learning task progresses to deeper layers of a CNN, the channel dimension can become larger. To achieve certain performance improvements, such as output accuracy and computation / processing speed, previous approaches have considered using a fused IBN layer block, such as the fused IBN layer 407 described above. However, the large dimension of each input channel (zin) makes the use of a fused IBN layer impractical due to the cost of performing full convolution and the slow computation speed.

[0060] Compared to traditional approaches, the group convolutions in each of process blocks 410, 420, and 430 provide alternative neural blocks that can each improve model performance while minimizing certain processing penalties. For example, the fused grouped IBN block 422 can be used to achieve improved performance without the latency and augmented / large dataset processing penalties associated with traditional or fused IBN layers. In general, each of the group convolution blocks 414, 416, 422, and 432 is a neural network block that can include one or more group convolution layers. Furthermore, each group convolution block 414, 416, 422, and 432 can be interleaved with other layers or block types that implement convolutions along the input channel dimension. An example of an interleaved neural block is shown in Figure 5.

[0061] The interleaving pattern can mitigate the lack of input channel convolutions between groups. For example, group convolutions use inter-channel information, but such information is limited to groups only, and using groups typically requires a shuffle operation to mix information along the channel dimension. The interleaving pattern also avoids the use of these additional shuffle operators (e.g., ShuffleNet). Similar to blocks 410 and 430, fused group convolution operations, e.g., via block 422, can generate more trainable parameters compared to baseline IBNs and enable processing speed improvements (e.g., faster execution speeds) compared to baseline IBNs and fused IBN layers for certain types of tensor shapes.

[0062] In some implementations, depth-wise convolutions limit input and output channels to the same size, while group convolutions can enable different sizes. For example, a K×K group convolution (414) kernel can perform an initial reduction in channel size before 1×1 projections reduce the channel size to its final value. One assumption here is that when group convolutions reduce channels to their final channel dimensions, thereby eliminating 1×1 projections, performance may be less than optimal (e.g., degraded) due to the small channel depth per group (zo). However, this can be mitigated if group convolutions are natively supported by an integrated circuit architecture that allows for the implementation of incremental expansion. For example, the architecture may require individual 1×1 projections on the integrated circuit. It can include an input bus that can pass input to the MAC.

[0063] The system 100 is operable to select from multiple different types of group convolution blocks. For example, in addition to the group convolution blocks 414, 416, 422, and 432 described above, the system 100 can also select a fused projection grouped convolution block that implements K×K group convolutions. The fused projection grouped convolutions fuse point-wise projections into the K×K main convolutions (instead of fusing point-wise extensions). Depending on the shape of the tensor, the fused projection grouped IBN can provide more trainable parameters while achieving similar processing efficiency compared to the fused IBN. The fused projection grouped IBN keeps some of the projection layers unfused and allows channel-wise convolutions between groups after the main K×K convolutions.

[0064] FIG. 5 is an example architecture 500 of a convolutional neural network for machine learning model 102 that can be used in the example computing system of FIG. 1. Neural architecture 500 can implement multiple respective sets of convolution operations to capture various characteristics of an example input image. In some implementations, system 100 is operable to strategically select and arrange various IBN layer / block options from the grouped and ungrouped IBN options described above with reference to the example of FIG. 4. In some implementations, system 100 is operable to select and arrange operations in a stacked, connected, or combined configuration (i.e., place and combine them together) to form example architecture 500 that can be used to implement large-scale computer vision networks / models.

[0065] 5, architecture 500 includes a sequence of layer blocks, where each of a first subset of layer blocks in the sequence is configured to perform an operation to process an input image. More specifically, architecture 500 includes a first subset of layer blocks 502, a second subset of layer blocks 504, and a third subset of layer blocks 506. In some implementations, at least one subset of layer blocks 502, 504, 506 may include an alternating sequence of two or more different types of neural blocks. For example, the subset of layer blocks 502 may have an alternating sequence including fused IBN layers and fused group IBN layers.

[0066] A fused IBN layer can represent a first individual neural block 512, such as fused IBN layer 407 (described above), that combines dilation and depth-wise convolution operations into a single fully convolutional neural block, while a fused group IBN layer can represent a second individual neural block 514, such as fused group IBN 422, that replaces point-wise (404) + depth-wise (405) pairs and allows for the fusion of aspects of those operations via K × K group convolution (422) to extend the channel dimension. As explained above, this block can provide an alternative to fused IBN layer 407 that allows for more efficient processing along the channel dimension.

[0067] More specifically, the first neural block 512 may be an ungrouped IBN block, and the second neural block 514 may be a grouped IBN block. Each of the first neural block 512 and the second neural block 514 includes one or more convolutional neural network layers. Thus, the layer block 502 may include an alternating sequence of grouped IBN layers and ungrouped IBN layers. For example, the alternating sequence of layer blocks may include grouped convolutional layer blocks interleaved with ungrouped convolutional layer blocks.

[0068] 6 illustrates an exemplary computation loop nest 600. A first computation loop nest 602 represents a loop nest for a full convolution computation, while a second computation loop nest 604 represents a loop nest for a group convolution computation using g groups.

[0069] 7 is a flow diagram of an example method 700 used to process an example image using group convolution. The example image may be image 102 described above, or various other types of digital images and related graphics data. In some implementations, method 700 is part of a technique used to speed up neural network computations, which can also improve the accuracy of image processing output compared to other data processing techniques.

[0070] Method 700 can be implemented or performed using the system 100 described above. Accordingly, descriptions of method 700 may refer to the computing resources of system 100 described above. The steps or actions of method 700 may be enabled by firmware or software instructions programmed and executable by one or more processors of the devices and resources described herein. In some implementations, the steps of method 700 correspond to methods for performing computations to generate outputs of neural network layers using a hardware integrated circuit, such as a dedicated neural network processor or a hardware machine learning accelerator, configured to implement a neural network.

[0071] Referring again to method 700, system 100 receives an input image (702) and processes the input image using an exemplary convolutional neural network (704). The convolutional neural network includes a sequence of layer blocks used to implement group convolutions for processing a digital input image, such as image 102. Individual layer blocks may correspond to group convolution operations performed on a hardware integrated circuit 114 that implements convolutional neural network 108. Layer blocks within the sequence of layer blocks may also include blocks that do not correspond to group convolution operations.

[0072] For example, a sequence of layer blocks can include or be formed from group convolutional layer blocks and non-group convolutional layer blocks. In some implementations, a sequence of layer blocks includes group convolutional layer blocks interleaved with non-group convolutional layer blocks. For example, some (or all) of an individual sequence of layer blocks can include group convolutional layer blocks interleaved among non-group convolutional layer blocks. In other implementations, an individual sequence of layer blocks can have a different arrangement of group convolutional layer blocks and non-group convolutional layer blocks. For example, rather than being interleaved, a sequence of layer blocks can be formed from distinct subsets of consecutive group convolutional layer blocks and consecutive non-group convolutional layer blocks.

[0073] The system 100 can determine the grouping of convolutions based on one or more constraints of a computer vision task or neural network architecture. The system 100 can then determine input groups corresponding to the group convolutions based on the determined grouping. For example, the system 100 can group input feature maps of an input matrix along the channel dimension of the input matrix to form one or more input groups. The input matrix is ​​derived from an input image. The system 100 can associate a corresponding kernel matrix with each input group and convolve the kernel matrix with the corresponding input group to generate the corresponding output group of the output matrix.

[0074] Each of the first subset of layer blocks in the sequence of layer blocks is associated with image processing. The CNN is configured to perform various types of operations to reduce the number of input feature maps. For example, a subset of layer blocks of a sequence included in the CNN is configured to receive input feature maps for the layer blocks (706). In some implementations, the input feature maps for the layer blocks are h×w feature maps with c1 channels. The subset of layer blocks is configured to generate augmented feature maps from the input feature maps using group convolution (708). In some implementations, the augmented feature maps are h×w feature maps with c2 channels, where c2 is greater than c1. The subset of layer blocks is configured to generate reduced feature maps from the augmented feature maps (710). In some implementations, the reduced feature maps are h×w feature maps with c1 channels.

[0075] The subset of layer blocks is configured to generate an output feature map for the layer block from the reduced feature map (712). In some implementations, the subset of layer blocks generates the output feature map by adding the input feature map to the reduced feature map. In some other implementations, the subset of layer blocks generates an output feature map that directly corresponds to the reduced feature map. For example, in these implementations, the output feature map is equal to the reduced feature map.

[0076] Embodiments of the subject matter and functional operations described herein can be implemented in digital electronic circuitry, tangibly embodied computer software or firmware, computer hardware, or one or more combinations thereof, including the structures disclosed herein and structural equivalents thereof. Embodiments of the subject matter described herein can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by or control the operation of a data processing apparatus.

[0077] Alternatively or additionally, the program instructions may be encoded on an artificially generated propagated signal, such as a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to an appropriate receiver device for execution by a data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more thereof.

[0078] The term "computing system" encompasses all kinds of apparatus, devices, and machines for processing data, such as, for example, a programmable processor, a computer, or multiple processors or computers. An apparatus can include special-purpose logic circuitry such as an FPGA (field-programmable gate array) or an ASIC (application-specific integrated circuit). In addition to hardware, an apparatus can also include code that creates an execution environment for the computer program in question, such as code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations thereof.

[0079] A computer program (which may also be called or described as a program, software, software application, module, software module, script, or code) may be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and may be deployed in any form, either as a stand-alone program or as a module, component, subroutine, or other device suitable for use in a computing environment.

[0080] A computer program may, but need not, correspond to a file in a file system. A program can be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subprograms, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are at one site or distributed across multiple sites and interconnected by a communications network.

[0081] The processes and logic flows described herein may be performed by one or more programmable computers executing one or more computer programs that perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, or an apparatus may be implemented as, special purpose logic circuitry, such as an FPGA (field programmable gate array), an ASIC (application specific integrated circuit), or a GPGPU (general purpose graphics processing unit).

[0082] A computer suitable for executing a computer program may be based, for example, on a general-purpose microprocessor, a special-purpose microprocessor, or both, or on any other type of central processing unit. Typically, the central processing unit receives instructions and data from a read-only memory, a random-access memory, or both. Some computer elements include a central processing unit for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer also includes one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or is operatively coupled to receive data from and / or transfer data to these storage devices. However, such devices are not required for a computer. Furthermore, a computer may be incorporated into another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name just a few. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, by way of example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal or removable disks), magneto-optical disks, CD-ROM and DVD-ROM disks, etc. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0083] To enable user interaction, embodiments of the subject matter described herein can be implemented on a computer that includes a display device (e.g., an LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) that allow the user to provide input to the computer. Other types of devices can also be used to provide user interaction. For example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and input from the user can be received in any form, such as acoustic, speech, or tactile input. Additionally, a computer can interact with a user by sending and receiving documents to devices used by the user. For example, a computer can send a web page to a web browser on a user's client device in response to a request received from the web browser.

[0084] Embodiments of the subject matter described herein may be implemented in a computing system including back-end components such as, for example, a data server, or a computing system including middleware components such as, for example, an application server, or a computing system including front-end components such as, for example, a client computer with a graphical user interface or web browser through which a user can interact with an implementation of the subject matter described herein, or may be implemented in any combination of one or more of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, such as a communications network. Examples of communications networks include local area networks ("LANs") and wide area networks ("WANs") such as the Internet.

[0085] A computing system may include clients and servers. Generally, clients and servers are remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0086] While this specification contains many specific implementation details, these should not be construed as limiting the scope of the invention or what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of a particular invention. Certain features described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable subcombination. Furthermore, even if features may be described above as functioning in a certain combination and originally claimed as such, one or more features from a claimed combination may, in some cases, be deleted from the combination, and the claimed combination may be directed to a subcombination or variations of the subcombination.

[0087] Similarly, although the figures depict operations in a particular order, this should not be understood as requiring that such operations be performed in the particular or sequential order shown, or that all of the operations shown be performed, to achieve desirable results. In some situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the program components and systems described may generally be integrated into a single software product or packaged into multiple software products.

[0088] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the following claims. For example, the actions recited in the claims may be performed in a different order and still achieve desirable results. By way of example, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing may be advantageous.

Claims

1. 1. A method implemented by one or more computers, the method comprising: obtaining an input image; and processing the input image using a convolutional neural network, the convolutional neural network comprising a sequence of layer blocks, each of a first subset of the layer blocks in the sequence comprising: receiving an input feature map for the layer block, the input feature map for the layer block being an h×w feature map having a c1 channel, and the first subset further comprising: generating an augmented feature map from the input feature map using group convolution, the augmented feature map being an h×w feature map having c2 channels, c2 being greater than c1, and the first subset further comprising: generating a reduced feature map from the expanded feature map, the reduced feature map being an h×w feature map having a c1 channel, the first subset further comprising: generating an output feature map for the layer block from the reduced feature map.

2. Generating the augmented feature map involves: generating an initial augmented feature map from the input feature map by applying a 1×1 convolution to the input feature map, the initial augmented feature map being an h×w feature map having c2 channels, and generating the augmented feature map further comprises: The method of claim 1 , comprising generating the augmented feature map from the initial augmented feature map by applying the group convolution to the initial augmented feature map.

3. The method of claim 2 , wherein the 1×1 convolution has more output filters than input filters.

4. The method of claim 2 , wherein the group convolution has the same total number of input filters and output filters.

5. The sequence of layer blocks is: The method of claim 1 , comprising group convolutional layer blocks interleaved with non-group convolutional layer blocks, the group convolutional layer blocks being used to implement the group convolution.

6. The method of claim 1 , wherein the group convolution is a fused group convolution implemented using a fused grouped inverse bottleneck (IBN) layer included within the sequence of layer blocks.

7. Generating the augmented feature map involves: The method of claim 1 , comprising generating the augmented feature map from the input feature map by applying the group convolution to the input feature map.

8. Generating the augmented feature map involves: generating an initial augmented feature map from the input feature map by applying a 1×1 convolution to the input feature map, wherein the initial augmented feature map is an h×w feature map having c3 channels, c3 being greater than c2; and generating the augmented feature map further comprises: Applying the group convolution to the initial dilated feature map The method of claim 1 , comprising generating the augmented feature map from a feature map.

9. a processing device and a non-transitory machine-readable storage device storing instructions executable by the processing device to perform operations, the operations comprising: obtaining an input image; and processing the input image using a convolutional neural network, the convolutional neural network comprising a sequence of layer blocks, each of a first subset of the layer blocks in the sequence comprising: receiving an input feature map for the layer block, the input feature map for the layer block being an h×w feature map having a c1 channel, and the first subset further comprising: generating an augmented feature map from the input feature map using group convolution, the augmented feature map being an h×w feature map having c2 channels, c2 being greater than c1, and the first subset further comprising: generating a reduced feature map from the expanded feature map, the reduced feature map being an h×w feature map having a c1 channel, the first subset further comprising: generating an output feature map for the layer block from the reduced feature map.

10. Generating the augmented feature map involves: generating an initial augmented feature map from the input feature map by applying a 1×1 convolution to the input feature map, the initial augmented feature map being an h×w feature map having c2 channels, and generating the augmented feature map further comprises: The system of claim 9 , further comprising generating the augmented feature map from the initial augmented feature map by applying a group convolution to the initial augmented feature map.

11. The system of claim 10 , wherein the 1×1 convolution has more output filters than input filters.

12. The system of claim 10 , wherein the group convolution has the same total number of input filters and output filters.

13. The sequence of layer blocks is:

10. The system of claim 9, comprising group convolutional layer blocks interleaved with non-group convolutional layer blocks, the group convolutional layer blocks being used to implement the group convolution.

14. 10. The system of claim 9, wherein the group convolution is a fused group convolution implemented using a fused grouped inverse bottleneck (IBN) layer included within the sequence of layer blocks.

15. Generating the augmented feature map involves: The system of claim 9 , further comprising generating the augmented feature map from the input feature map by applying a group convolution to the input feature map.

16. Generating the augmented feature map involves: generating an initial augmented feature map from the input feature map by applying a 1×1 convolution to the input feature map, the initial augmented feature map being an h×w feature map having c3 channels, c3 being greater than c2, to generate the augmented feature map. Furthermore, The system of claim 9 , further comprising generating the augmented feature map from the initial augmented feature map by applying a group convolution to the initial augmented feature map.

17. A non-transitory machine-readable storage device that stores instructions, the instructions being executable by a processing device to cause operations to be performed, the operations comprising: obtaining an input image; and processing the input image using a convolutional neural network, the convolutional neural network comprising a sequence of layer blocks, each of a first subset of the layer blocks in the sequence comprising: receiving an input feature map for the layer block, the input feature map for the layer block being an h×w feature map having a c1 channel, and the first subset further comprising: generating an augmented feature map from the input feature map using group convolution, the augmented feature map being an h×w feature map having c2 channels, c2 being greater than c1, and the first subset further comprising: generating a reduced feature map from the expanded feature map, the reduced feature map being an h×w feature map having a c1 channel, the first subset further comprising: generating an output feature map for the layer block from the reduced feature map.

18. Generating the augmented feature map involves: generating an initial augmented feature map from the input feature map by applying a 1×1 convolution to the input feature map, the initial augmented feature map being an h×w feature map having c2 channels, and generating the augmented feature map further comprises:

20. The machine-readable storage device of claim 17, further comprising generating the augmented feature map from the initial augmented feature map by applying the group convolution to the initial augmented feature map.

19. The sequence of layer blocks is:

20. The machine-readable storage device of claim 17, comprising group convolutional layer blocks interleaved with non-group convolutional layer blocks, the group convolutional layer blocks being used to implement the group convolution.

20. the grouped convolution is a fused grouped convolution implemented using a fused grouped inverse bottleneck (IBN) layer included within the sequence of layer blocks; 20. The machine-readable storage device of claim 17, wherein generating an extended feature map comprises generating the extended feature map from the input feature map by applying the group convolution to the input feature map.

Images