Compression neural network
By identifying and removing redundant channels in neural networks using a graph-based approach, the method addresses high computational and memory demands, enhancing efficiency in neural network implementation.
Patent Information
- Application Number
- CN202410696874.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-31
- Filing Date
- 2024-05-31
- Publication Date
- 2024-12-03
AI Technical Summary
Neural networks require significant computational resources and memory bandwidth due to large coefficient sets, particularly in fully connected and convolutional layers, leading to high processing demands and memory bandwidth requirements.
A method and system for compressing neural networks by identifying and removing redundant channels through a graph representation of data flow, forming vertex subsets for different layer types, and traversing the graph to identify and eliminate redundant paths and channels.
Reduces computational requirements and memory bandwidth by eliminating redundant channels, enabling more efficient implementation of neural networks on hardware accelerators.
Smart Images

Figure CN119067180A_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims priority to UK patent application GB 2308120.1 filed on May 31, 2023, which is incorporated herein by reference in its entirety. Technical Field
[0003] The present disclosure relates to methods and processing systems for compressing neural networks. Background Art
[0004] A neural network (NN) is a form of artificial network that includes multiple layers (eg, "interconnected layers") that can be used for machine learning applications. In particular, neural networks can be used to perform signal processing applications, including but not limited to image processing.
[0005] Figure 1 An exemplary neural network 100 including a plurality of layers 102-1, 102-2, 102-3 is shown. The plurality of layers 102-1, 102-2, 102-3 are arranged in sequence. Each layer 102-1, 102-2, 102-3 receives a set of input activation values. The set of input activation values received by a layer may be referred to as input activation data for the layer. Each layer 102-1, 102-2, 102-3 processes its corresponding received input activation data by performing an operation according to the layer to produce output data. The output data is provided to a subsequent layer as the input activation data for the layer (e.g., the output data is the "output activation data"), or is output as the final output data of the neural network. For example, in the neural network 100, the first layer 102-1 receives input activation data 104 that is initially input to the neural network 100, and processes the input activation data by performing a first operation according to the first layer 102-1 to produce output activation data. The output activation data of the first layer 102-1 becomes the input activation data of the second subsequent layer 102-2, which processes the input activation data by performing a second operation according to the second layer 102-2 to generate output activation data. The output activation data of the second layer 102-2 becomes the input activation data of the third subsequent layer 102-3, which processes the input activation data by performing a third operation according to the third layer 102-3 to generate output data. The output data of the third layer 102-3 is output as the output data 106 of the neural network.
[0006] Each layer of a neural network can be one of a number of different types. The type of operation performed on the input activation data of the layer depends on the type of layer. Fully connected layers (sometimes called dense layers or linear layers), convolutional layers, addition layers, flattening layers, pooling layers, and activation layers (such as rectified linear activation unit (ReLu) layers) are example types of neural network layers. It will be apparent to those skilled in the art that this is not an exhaustive list of example neural network layer types.
[0007] Certain types of neural network layers use sets of coefficients associated with those layers to perform operations on sets of input activation values received by those layers. Fully connected layers and convolutional layers are examples of layer types that use sets of coefficients in this way.
[0008] In a fully connected layer, a fully connected operation may be performed by performing a matrix multiplication between a coefficient matrix comprising a set of coefficients of the fully connected layer and an input matrix comprising a set of input activation values received by the fully connected layer. The purpose of a fully connected layer is to cause a dimensionality change between the activation data set input to this layer and the activation data set output from this layer. The coefficient matrix comprising the set of coefficients of the fully connected layer may have a dimension C out ×C in That is, the number of rows of the matrix can represent the number of output channels of the fully connected layer (“C out ”), and the number of columns of the matrix can represent the number of input channels of the fully connected layer (“C in ”). In the fully connected layer, a matrix multiplication WX=Y can be performed, where: W is a matrix that includes a set of coefficients and has dimension C out ×C in The coefficient matrix of C; X is the input matrix including the input activation value set and has dimension M×N, where C in = M; and Y is a set of output values and has dimension C out Alternatively, the coefficient matrix comprising the coefficient set of the fully connected layer may have dimension C in ×C out That is, the number of rows of the matrix can represent the number of input channels of the fully connected layer (“C in ”), and the number of columns of the matrix can represent the number of output channels of the fully connected layer (“C out ”). In this alternative, in the fully connected layer, a matrix multiplication XW=Y can be performed, where: X is the input matrix including the set of input activation values and having dimension M×N; W is the matrix including the set of coefficients and having dimension C in ×C out The coefficient matrix, C in =N where; and Y is a set of output values and has dimensions M×C outThe matrix multiplication involves performing a plurality of element-wise multiplications between the coefficients of the coefficient matrix and the activation values of the input matrix. The results of the element-wise multiplications may be summed (e.g., accumulated) to form output data values of the output matrix. It will be apparent to those skilled in the art that other types of neural network layers also perform matrix multiplications using coefficient matrices comprising sets of coefficients.
[0009] In a convolutional layer, a convolution operation is performed using a set of input activation values received by the convolutional layer and a set of coefficients of the convolutional layer. Figure 2a shows the data structures used in the example convolutional layer. Figure 2a In the example, the input activation data of the convolutional layer can have dimension C in ×H a ×W a In other words, the input activation data can be arranged as C in input channels, each of which has spatial dimension H a ×W a —— Among them, H a and W a are the height and width dimensions respectively. Figure 2a In the figure, the input activation data is shown to include three input channels (i.e., C in =3). The coefficient sets of the convolutional layer can be arranged in one or more input channels and one or more output channels. The output channels can alternatively be referred to as filters. Figure 2a In the equation, the coefficient set has dimension C out ×C in ×H w ×W w The number of input channels in the coefficient set of the convolutional layer corresponds to (e.g., is equal to) the number of input channels in the input activation data to be convolved with the coefficient set (e.g., in Figure 2a In the example shown, C in =3). Each input channel of each filter of the coefficient set input to the convolutional layer has a spatial dimension H w ×W w —— Among them, H w and W w are the height and width dimensions, respectively. Each input channel of each filter includes a corresponding subset of the set of coefficients of the convolutional layer. Each coefficient is contained in (e.g., is included by or is part of) an input channel and a filter. C out The dimension (e.g., the number of output channels or the number of filters) is Figure 2a , but represents the number of channels generated in the output data by performing a convolution operation using a coefficient set.
[0010] Figure 2b An example convolutional layer 200 is schematically shown. Figure 2b , the convolutional layer 200 is arranged to perform a convolution operation using input activation data 202 and a coefficient set 204. Figure 2b Also shown is the use of a bias (e.g., "offset bias") 208 within the convolutional layer 200 - this is optional. Figure 2b , activation data 202 input to layer 200 is arranged in three input channels 1, 2, 3. The number of input channels in coefficient set 204 corresponds to (e.g., is equal to) the number of input channels in activation data 202 to be convolved with coefficient set 204. Therefore, coefficient set 204 is arranged in three input channels 1, 2, 3. Coefficient set 204 is also arranged in four filters (e.g., output channels) A, B, C, D. The number of filters in coefficient set 204 corresponds to (e.g., is equal to) the number of channels in output data 206. Each filter A, B, C, D in coefficient set 204 is responsible for forming a corresponding channel A, B, C, D of output data 206. Each coefficient is contained in one input channel and one filter (e.g., included by or part of one input channel and one filter). For example, coefficient 210 is included in input channel 1 and filter A. In convolutional layer 200, input activation data 202 is convolved with coefficient set 204 to generate output data 206 having four data channels A, B, C, D. More specifically, the first input channel of input activation data 202 is convolved with the first input channel of each filter in coefficient set 204. For example, returning to Figure 2a , with spatial dimension H a ×W a The first input channel of the input activation data 202 may be associated with the coefficient set 204 having a spatial dimension H w ×W w The first input channel of each filter is convolved in directions s and t in multiple steps. Return to Figure 2b , in a similar manner, the second input channel of the input activation data 202 is convolved with the second input channel of each filter in the coefficient set 204, and the third input channel of the input activation data 202 is convolved with the third input channel of each filter in the coefficient set 204. Convolving each input channel of the input activation data with each input channel of each filter involves performing multiple element-by-element multiplications between the activation values of each input channel of the input activation data and the coefficients of each input channel of each filter. The results of the element-by-element multiplications can be summed (e.g., accumulated) to form output data values for each channel of the output data 206. If the convolution layer 200 is not configured to use a bias, the output data 206 will be the output of the convolution layer. Figure 2b, the output data 206 is the intermediate output data to be combined with the bias 208. Each of the four filters (e.g., output channels) A, B, C, D of the coefficient set 204 input to the convolution layer 200 is associated with a corresponding bias A, B, C, D. In the convolution layer 200, the biases A, B, C, D are added to the corresponding data channels A, B, C, D of the intermediate data 206 to generate output data 212 having four data channels A, B, C, D. In this case, the output data 212 is the output of the convolution layer 200.
[0011] The coefficient sets used by certain layers of a typical neural network (e.g., fully connected layers and / or convolutional layers) typically include a large number of coefficients. A layer with a large coefficient set may place greater computational demands on the processing elements of the neural network accelerator that implements this layer. This is because this layer may require those processing elements to perform a large number of multiplication and accumulation operations to generate the output of this layer. In addition, when a neural network is implemented at a neural network accelerator, the coefficient sets are typically stored in "off-chip" memory. The neural network accelerator can implement a layer of a neural network by reading the coefficient sets of the layer at runtime. A large amount of memory bandwidth may be required to read a large number of coefficient sets from the off-chip memory. The memory bandwidth required to read in the coefficient set can be referred to as "weight bandwidth". It is desirable to reduce the processing requirements and weight bandwidth required to implement a neural network at a neural network accelerator. Summary of the invention
[0012] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
[0013] According to a first aspect of the present invention, a computer-implemented method for compressing a neural network is provided, the method comprising: receiving a neural network comprising a plurality of layers; forming a graph representing a data flow through the plurality of layers of the neural network, the graph comprising: a plurality of vertices, each vertex of the plurality of vertices representing an output channel of a layer of the plurality of layers of the neural network; and one or more edges, each of the one or more edges representing a potential non-zero data flow between corresponding output channels represented by a corresponding pair of vertices; identifying one or more redundant channels included by the plurality of layers of the neural network by traversing the graph; and outputting a compressed neural network, wherein the identified one or more redundant channels do not exist in the compressed neural network.
[0014] A redundant channel may be a channel included by a layer of the plurality of layers of a neural network that may be removed from the neural network without changing an output of the neural network.
[0015] The graph may include: a plurality of vertex subsets, each vertex subset in the plurality of vertex subsets representing a corresponding layer in a plurality of layers of a neural network, each vertex subset in the plurality of vertex subsets including one or more vertices, each vertex in the one or more vertices representing an output channel of the corresponding layer of the neural network; and one or more edges, each of the one or more edges: connecting two vertices, the two vertices being included in different vertex subsets of the graph; and representing a potential non-zero data flow between corresponding channels of the corresponding layers of the neural network represented by these vertices.
[0016] A vertex subset among multiple vertex subsets may represent a fully connected layer of multiple layers of a neural network, each of one or more vertices included by the vertex subset may represent a corresponding output channel of the fully connected layer, and forming a graph may include: determining a matrix representing a coefficient set of the fully connected layer, the matrix including one or more elements representing non-zero coefficients and one or more elements representing zero coefficients; for each of the one or more elements representing non-zero coefficients: identifying: an output channel of the fully connected layer including the non-zero coefficient; an input channel of the fully connected layer including the non-zero coefficient; and an output channel of a previous layer among the multiple layers of the neural network corresponding to the identified input channel of the fully connected layer; and connecting vertices in the vertex subset representing the identified output channel of the fully connected layer to vertices in different vertex subsets among the multiple vertex subsets representing the identified output channel of the previous layer using edges.
[0017] A vertex subset among multiple vertex subsets may represent a convolutional layer among multiple layers of a neural network, each of one or more vertices included by the vertex subset may represent a corresponding output channel of the convolutional layer, and forming the graph may include: determining a matrix representing a coefficient set of the convolutional layer, the matrix including one or more elements representing non-zero values and one or more elements representing zero values; for each of the one or more elements representing non-zero values: identifying: an output channel of the convolutional layer; an input channel of the convolutional layer; and an output channel of a previous layer among the multiple layers of the neural network corresponding to the identified input channel of the convolutional layer; and connecting vertices in the vertex subset representing the identified output channel of the convolutional layer to vertices in different vertex subsets among the multiple vertex subsets representing the identified output channel of the previous layer using edges.
[0018] The convolution layer may include a set of coefficients arranged in one or more filters, each of the one or more filters is arranged in one or more channels, each channel of each filter includes a corresponding subset of the set of coefficients of the convolution layer, and determining the matrix may include: for each channel of each filter: determining whether the channel of the filter includes a non-zero coefficient; and in response to determining that the channel of the filter includes at least one non-zero coefficient, using elements in the matrix representing non-zero values to represent the channel of the filter; or in response to determining that the channel of the filter includes only zero coefficients, using elements in the matrix representing zero values to represent the channel of the filter.
[0019] The vertex subset in the multiple vertex subsets may also include a bias vertex, wherein the bias vertex represents one or more biases of a subsequent layer in the multiple layers of the neural network to the layer represented by the vertex subset in the multiple layers of the neural network, and the bias vertex is connected to one or more vertices in the vertex subset representing the subsequent layer by one or more edges, each of the edges representing a non-zero bias in the one or more biases represented by the bias vertex, and the bias vertex is associated with a corresponding output channel in the one or more output channels represented by the vertex subset representing the subsequent layer.
[0020] A vertex subset among multiple vertex subsets can represent an addition layer among multiple layers of a neural network, the vertex subset includes a number of vertices equal to the number of channels of each activation data set among multiple activation data sets that the addition layer is configured to sum, each activation data set among the multiple activation data sets has the same number of channels, each vertex included by the vertex subset represents a corresponding summation operation performed between corresponding channel sets of the multiple activation data sets so that each vertex included by the vertex subset represents a corresponding output channel of the addition layer; and each vertex included by the vertex subset can be connected to vertices in different vertex subsets through corresponding edges, the vertex represents an output channel of a previous layer among the multiple layers of the neural network, the output channel corresponds to a channel in the corresponding channel set of the multiple activation data sets, and the summation operation represented by the vertex is performed between the sets of the corresponding channels.
[0021] A vertex subset among multiple vertex subsets can represent a flattening layer among multiple layers of a neural network, the vertex subset includes n vertex groups, n is equal to the number of channels of data of an activation data set on which the flattening layer is configured to perform a flattening operation, each vertex group includes m vertices, m is equal to the number of values in each data channel of the activation data set, each vertex included by the vertex subset represents a corresponding output channel of the flattening layer; and each vertex included by each vertex group in the vertex subset can be connected to a vertex in a different vertex subset by a corresponding edge, the vertex represents an output channel of a previous layer among multiple layers of the neural network, the output channel corresponding to a channel of the activation data set on which a portion of the flattening operation represented by the vertex group is performed.
[0022] The neural network may include a pooling layer or an activation layer, and the plurality of subsets of vertices may exclude the subset of vertices representing this layer of the neural network.
[0023] The plurality of vertex subsets may be arranged in a sequence representing a sequence in which the plurality of layers of the neural network are arranged, and identifying one or more redundant paths may include: assigning a first state to each of the incoming edges of the vertices in the vertex subset representing the last layer in sequence of the plurality of layers of the neural network; traversing the sequence of vertex subsets from the vertex subset representing the second to last layer in sequence of the plurality of layers of the neural network to the vertex subset representing the first layer in sequence of the plurality of layers of the neural network, thereby evaluating each of the one or more vertices in each vertex subset to determine whether the vertex has at least one outgoing edge assigned the first state, and: if the vertex has at least one outgoing edge assigned the first state, assigning the first state to each of the incoming edges of the vertex; if the vertex does not have at least one outgoing edge assigned the first state, not assigning the first state to each of the incoming edges of the vertex. each incoming edge is assigned a first state; then, a sequence of vertex subsets is traversed from a vertex subset representing a first layer in sequence of a plurality of layers of a neural network to a vertex subset representing a second to last layer in sequence of a plurality of layers of a neural network, thereby evaluating each vertex in one or more vertices in each vertex subset to determine whether the vertex has at least one incoming edge assigned the first state, and: if the vertex has at least one incoming edge assigned the first state, then each of the outgoing edges of the vertex is assigned the first state; if the vertex does not have at least one incoming edge assigned the first state, then each of the outgoing edges of the vertex is not assigned the first state; and then, one or more vertices that do not have any outgoing edges assigned the first state are identified, the one or more identified vertices representing one or more redundant channels included by the plurality of layers of the neural network.
[0024] The graph may also include one or more output edges representing outputs from the multiple layers of the neural network, the one or more output edges extending from corresponding one or more vertices in the subset of vertices representing the sequentially last layer in the multiple layers of the neural network; and identifying the one or more redundant paths may include: assigning a first state to each outgoing edge in the output edges; and traversing the sequence of vertex subsets from the subset of vertices representing the sequentially last layer in the multiple layers of the neural network to the subset of vertices representing the sequentially first layer in the multiple layers of the neural network.
[0025] The plurality of vertex subsets may be arranged in a sequence representing a sequence in which the plurality of layers of the neural network are arranged, and identifying the one or more redundant paths may include: assigning a first state to each of the incoming edges of one or more vertices included in the vertex subset representing a sequentially first layer of the plurality of layers of the neural network; traversing the sequence of vertex subsets from the vertex subset representing the sequentially first layer of the plurality of layers of the neural network to the vertex subset representing the sequentially second to last layer of the plurality of layers of the neural network, thereby evaluating each of the one or more vertices in each vertex subset to determine whether the vertex has at least one incoming edge assigned the first state, and: if the vertex has at least one incoming edge assigned the first state, assigning the first state to each of the outgoing edges of the vertex; if the vertex does not have at least one outgoing edge assigned the first state, not assigning the first state to the outgoing edges of the vertex; each outgoing edge in the edge is assigned a first state; then, a sequence of vertex subsets is traversed from a vertex subset representing the second to last layer in order of the plurality of layers of the neural network to a vertex subset representing the first layer in order of the plurality of layers of the neural network, thereby evaluating each vertex in one or more vertices in each vertex subset to determine whether the vertex has at least one outgoing edge assigned the first state, and: if the vertex has at least one outgoing edge assigned the first state, assigning the first state to each incoming edge of the vertex; if the vertex does not have at least one outgoing edge assigned the first state, causing each incoming edge of the vertex to not be assigned the first state; and then, identifying one or more vertices that do not have any outgoing edges assigned the first state, the one or more identified vertices representing one or more redundant channels included by the plurality of layers of the neural network.
[0026] The graph may also include an input vertex subset representing inputs to multiple layers of the neural network, the input vertex subset including one or more input vertices, each of the one or more input vertices representing a channel of inputs to the multiple layers of the neural network; and identifying one or more redundant channels may include: assigning a first state to each of the input edges of the one or more input vertices included by the input vertex subset; and traversing a sequence of vertex subsets from the input vertex subset to a vertex subset representing a second-to-last layer in sequence of the multiple layers of the neural network.
[0027] The plurality of vertex subsets may be arranged in a sequence representing a sequence in which the plurality of layers of the neural network are arranged, and identifying the one or more redundant channels may include: assigning a first state to each of the incoming edges of the vertices in the vertex subset representing the sequentially last layer of the plurality of layers of the neural network; traversing the sequence of vertex subsets from the vertex subset representing the sequentially second-to-last layer of the plurality of layers of the neural network to the vertex subset representing the sequentially first layer of the plurality of layers of the neural network, thereby evaluating each of the one or more vertices in each vertex subset to determine whether the vertex has at least one outgoing edge assigned the first state, and: if the vertex has at least one outgoing edge assigned the first state, assigning the first state to each of the incoming edges of the vertex; if the vertex does not have at least one outgoing edge assigned the first state, not assigning the first state to each of the incoming edges of the vertex; and subsequently, identifying one or more vertices that do not have any outgoing edges assigned the first state, the one or more identified vertices representing one or more redundant channels included by the plurality of layers of the neural network.
[0028] The plurality of vertex subsets may be arranged in a sequence representing a sequence in which the plurality of layers of the neural network are arranged, and identifying the one or more redundant channels may include: assigning a first state to each of the incoming edges of one or more vertices included by the vertex subset representing a sequentially first layer of the plurality of layers of the neural network; traversing the sequence of vertex subsets from the vertex subset representing the sequentially first layer of the plurality of layers of the neural network to the vertex subset representing the sequentially second to last layer of the plurality of layers of the neural network, evaluating each of the one or more vertices in each vertex subset to determine whether the vertex has at least one incoming edge assigned the first state, and: if the vertex has at least one incoming edge assigned the first state, assigning the first state to each of the outgoing edges of the vertex; if the vertex does not have at least one outgoing edge assigned the first state, not assigning the first state to each of the outgoing edges of the vertex; and subsequently, identifying one or more vertices that do not have any outgoing edges assigned the first state, the one or more identified vertices representing the one or more redundant channels included by the plurality of layers of the neural network.
[0029] The edges may be outgoing edges of a first vertex and / or incoming edges of a second vertex; the incoming edges of a vertex may represent potential non-zero data flow into an output channel represented by the vertex; and the outgoing edges of a vertex may represent potential non-zero data flow from an output channel represented by the vertex.
[0030] One or more output channels of a last layer in a sequence of multiple layers of a neural network may not be identified as redundant channels.
[0031] An output channel of an additive layer of multiple layers of a neural network may only be identified as a redundant channel when the vertex representing the output channel does not have any outgoing edges assigned the first state, and all vertices representing output channels from previous layers connected to the vertex by edges also do not have any outgoing edges assigned the first state.
[0032] An output channel of a flattening layer of the plurality of layers of a neural network may only be identified as a redundant channel when all m vertices included by the vertex group including the vertex representing the output channel do not have any outgoing edges assigned a first state.
[0033] The method may also include storing the compressed neural network for subsequent implementation. The method may also include outputting a computer-readable description of the compressed neural network, the computer-readable description causing the compressed neural network to be executed when implemented at a system for implementing a neural network. The method may also include configuring hardware logic to implement the compressed neural network, wherein the hardware logic includes a neural network accelerator. The method may also include performing image processing using the compressed neural network.
[0034] According to a second aspect of the present invention, a processing system for compressing a neural network is provided, the processing system comprising at least one processor, the at least one processor being configured to: receive a neural network comprising a plurality of layers; form a graph representing a data flow through the plurality of layers of the neural network, the graph comprising: a plurality of vertices, each vertex of the plurality of vertices representing an output channel of a layer of the plurality of layers of the neural network; and one or more edges, each of the one or more edges representing a potential non-zero data flow between corresponding output channels represented by a corresponding pair of vertices; identify one or more redundant channels included by the plurality of layers of the neural network by traversing the graph; and output a compressed neural network, wherein the identified one or more redundant channels do not exist in the compressed neural network.
[0035] The processing system may further include a memory, and the at least one processor may further be configured to write the compressed neural network into the memory for subsequent implementation.
[0036] The at least one processor may also be configured to configure hardware logic to implement the compressed neural network. The hardware logic may include a neural network accelerator.
[0037] The processing system may be embodied in hardware on an integrated circuit. A method of manufacturing a processing system at an integrated circuit manufacturing system may be provided. An integrated circuit definition data set may be provided that, when processed in the integrated circuit manufacturing system, configures the system to manufacture the processing system. A non-transitory computer-readable storage medium may be provided on which a computer-readable description of the processing system is stored that, when processed in the integrated circuit manufacturing system, causes the integrated circuit manufacturing system to manufacture an integrated circuit embodying the processing system.
[0038] An integrated circuit manufacturing system may be provided, the integrated circuit manufacturing system comprising: a non-transitory computer-readable storage medium having stored thereon a computer-readable description of a processing system; a layout processing system configured to process the computer-readable description to generate a circuit layout description of an integrated circuit embodying the processing system; and an integrated circuit production system configured to manufacture the processing system based on the circuit layout description.
[0039] A computer program code for executing any of the methods described herein may be provided. A non-transitory computer readable storage medium may be provided on which computer readable instructions are stored, which when executed in a computer system, cause the computer system to execute any of the methods described herein.
[0040] As will be apparent to the skilled person, the above features may be combined as appropriate and may be combined with any aspect of the examples described herein. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Examples will now be described in detail with reference to the accompanying drawings, in which:
[0042] Figure 1 An example neural network is shown.
[0043] Figure 2a shows the data structures used in an example convolutional layer.
[0044] Figure 2b An example convolutional layer is shown.
[0045] Figure 2c A pair of example convolutional layers are shown.
[0046] Figure 3 An example system for implementing a neural network is shown.
[0047] Figure 4 A processing system for compressing a neural network is shown.
[0048] Figure 5 A method for compressing a neural network is shown.
[0049] Figure 6a An example matrix representing the set of coefficients for a fully connected layer is shown.
[0050] Figure 6b An example matrix representing the set of coefficients for a fully connected layer is shown.
[0051] Figure 7a Represents a sequence of fully connected or convolutional layers.
[0052] Figure 7b Shown is the expression through Figure 7a Diagram of the data flow of multiple layers represented in .
[0053] Figure 8a Represents multiple layers including a planarization layer.
[0054] Figure 8b Shown is the expression through Figure 8a Diagram of the data flow of multiple layers represented in .
[0055] Figure 9a Represents multiple layers including additive layers.
[0056] Figure 9b Shown is the expression through Figure 9a Diagram of the data flow of multiple layers represented in .
[0057] Fig.10a Represents a plurality of layers including a convolutional layer configured to use a bias.
[0058] Fig.10b It is shown how the bias of a layer can be represented in a diagram representing the flow of data through multiple layers.
[0059] Fig.11a Represents multiple layers including pooling layers or activation layers.
[0060] Fig.11b Shown is the expression through Fig.11a The first diagram shows the data flow of multiple layers.
[0061] Fig.11c Shown is the expression through Fig.11a A second alternative diagram of the data flow of multiple layers represented in .
[0062] Figures 12a to 12i It is shown how, in a preferred method, a graph representing data flow through multiple layers of a neural network may be traversed to identify one or more redundant channels included by the multiple layers.
[0063] Figures 13a to 13gIt is shown how, in a first preferred approach, a graph representing data flow through multiple layers of a neural network may be traversed to identify one or more redundant channels included by the multiple layers.
[0064] Figures 14a to 14g It is shown how, in a second preferred method, a graph representing data flow through multiple layers of a neural network may be traversed to identify one or more redundant channels included by the multiple layers.
[0065] Fig.15a Represents a neural network consisting of a sequence of fully connected or convolutional layers.
[0066] Fig.15b Represents a compressed neural network comprising a sequence of fully connected or convolutional layers in which one or more redundant channels are absent.
[0067] Fig.16 A computer system is shown in which the processing system is implemented; and
[0068] Fig.17 An integrated circuit manufacturing system for producing an integrated circuit embodying a processing system is shown.
[0069] The accompanying drawings illustrate various examples. It will be appreciated by the skilled person that the element boundaries (e.g., boxes, groups of boxes, or other shapes) shown in the accompanying drawings represent an example of boundaries. In some examples, it may be the case that an element can be designed as multiple elements, or multiple elements can be designed as one element. Where appropriate, common reference numerals are used throughout the accompanying drawings to indicate similar features. DETAILED DESCRIPTION
[0070] The following description is presented by way of example to enable one skilled in the art to make and use the invention.The present invention is not limited to the embodiments described herein, and various modifications to the disclosed embodiments will be apparent to those skilled in the art.
[0071] Embodiments will now be described by way of example only.
[0072] Neural networks can be used to perform image processing. Examples of image processing techniques that can be performed by neural networks include: image super-resolution processing, semantic image segmentation processing, object detection, and image classification. For example, in an image super-resolution processing application, image data representing one or more lower resolution images can be input to a neural network, and the output of this neural network can be image data representing one or more higher resolution images. In another example, in an image classification application, image data representing one or more images can be input to a neural network, and the output of the neural network can be data indicating the probability (or probability set) that each of these images belongs to a specific classification (or classification set). It should be understood that the principles described herein are not limited to being used to compress neural networks for performing image processing. For example, the principles described herein can be used to compress neural networks to perform speech recognition / speech to text applications or any other suitable type of application. The technician will understand how to configure the neural network to perform any of the processing techniques mentioned in this paragraph, so for the sake of brevity, these techniques will not be discussed in further detail.
[0073] A neural network can be defined by a software model. For example, the software model can define a sequence of layers of the neural network (e.g., the number of layers, the order of the layers, and the connectivity between the layers), and define each of the layers in the sequence according to the operation it is configured to perform (and optionally the set of coefficients it will use). In general, a neural network can be implemented in hardware, software, or any combination thereof.
[0074] Figure 3 An example system 300 for implementing a neural network in hardware is shown. The system 300 includes hardware components (e.g., hardware logic, such as processing elements implemented in hardware) and software components (e.g., firmware, and processes and tasks for execution at the hardware logic). The system 300 includes hardware logic for implementing a neural network. The hardware logic may include a neural network accelerator (NNA) 302. In other words, the system 300 may include a hardware neural network accelerator (NNA). That is, the system 300 may include a neural network accelerator (NNA) implemented in hardware. The neural network accelerator (NNA) 302 may be configured to perform operations to be performed by a layer of a neural network. The neural network accelerator (NNA) 302 includes a plurality of configurable resources that enable implementation of different types of neural network layers, such as fully connected layers and convolutional layers. In other words, implementing a neural network may include configuring the hardware logic included by the system 300 to process data input to the system according to the layers of the neural network.
[0075] A neural network accelerator (NNA) is a hardware accelerator designed to accelerate the processing of a neural network. As known to those skilled in the art, a hardware accelerator is hardware designed to perform a specific set of one or more functions more efficiently than a general-purpose processing unit, such as a central processing unit (CPU). Therefore, compared to a general-purpose CPU that can be configured to perform any number of functions, an accelerator can only perform a limited set of one or more functions. An NNA includes one or more hardware accelerators designed to accelerate one or more neural network operations. Therefore, a graphics processing unit (GPU) having one or more hardware accelerators designed to accelerate one or more neural network operations can be understood as an NNA.
[0076] In more detail, system 300 includes an input 301 for receiving input data. The input data received at input 301 includes input activation data. For example, when the implemented neural network is configured to perform image processing, the input activation data may include image data representing one or more images. For example, for RGB images, the image data may be in the format C in ×H a ×W a , where H a and W a is the image across the three input color channels C in (ie, R, G, and B) pixel dimensions. The input data received at input 301 also includes coefficient sets for each layer of the neural network using the coefficient sets. The coefficient sets may also be referred to as weights.
[0077] Input data received at input 301 may be written to a memory 304 included in system 300. Memory 304 may be accessible by neural network accelerator (NNA) 302. Memory 304 may be a system memory accessible to neural network accelerator (NNA) 302 via a data bus. Neural network accelerator (NNA) 302 may be implemented on a chip (e.g., a semiconductor die and / or an integrated circuit package), and memory 304 may not be physically located on the same chip (e.g., a semiconductor die and / or an integrated circuit package) as neural network accelerator (NNA) 302. Thus, memory 304 may be referred to as "off-chip memory" and / or "external memory". Memory 304 may be coupled to input buffer 306 at neural network accelerator (NNA) 302 to provide input activation data to neural network accelerator (NNA) 302. Memory 304 may be coupled to coefficient buffer 330 at neural network accelerator (NNA) 302 to provide coefficient sets to neural network accelerator (NNA) 302.
[0078] Input buffer 306 may be arranged to store input activation data required by neural network accelerator (NNA) 302. Coefficient buffer 330 may be arranged to store coefficient sets required by neural network accelerator (NNA) 302. Input buffer 306 may contain some or all of the input activation data associated with one or more operations performed at neural network accelerator (NNA) 302 on a given cycle, as described herein. Coefficient buffer 330 may include some or all of the coefficient sets associated with one or more operations processed at neural network accelerator (NNA) 302 on a given cycle, as described herein. Figure 3 The various buffers of the neural network accelerator (NNA) 302 shown may be implemented in any suitable manner, for example, as any number of data storage repositories local to the neural network accelerator (NNA) 302 (e.g., on the same semiconductor die and / or disposed within the same integrated circuit package) or accessible to the neural network accelerator (NNA) 302 via a data bus or other interconnect.
[0079] exist Figure 3 In the embodiment, the system 300 also includes a memory 328 for storing at least some of the coefficient sets and / or input activation data required for the neural network implemented at the neural network accelerator (NNA) 302. The neural network accelerator (NNA) 302 can be implemented on a chip (e.g., a semiconductor die and / or an integrated circuit package), and the memory 328 can be physically located on the same chip (e.g., a semiconductor die and / or an integrated circuit package) as the neural network accelerator (NNA) 302. As such, the memory 328 can be referred to as "on-chip memory" and / or "local memory". Data stored in the on-chip memory 328 can be accessed by the neural network accelerator (NNA) 302 without consuming memory bandwidth to the off-chip memory 304. That is, the on-chip memory 328 may not have enough storage space to simultaneously store all the input data required to implement the neural network. Therefore, a subset of the input data can be periodically written from the off-chip memory 304 to the on-chip memory 328. The coefficient buffer 330 may be configured to receive coefficient sets from the on-chip memory 328 to reduce the bandwidth between the off-chip memory 304 and the coefficient buffer 330. The input buffer 306 may be configured to receive input activation data from the on-chip memory 328 to reduce the bandwidth between the off-chip memory 304 and the input buffer 306.
[0080] exist Figure 3In the figure, the neural network accelerator (NNA) 302 includes an input buffer 306, a coefficient buffer 330, a plurality of multiplication engines 308, a plurality of accumulators 310, an accumulation buffer 312, and an output buffer 316. Each multiplication engine 308, together with its corresponding accumulator 310 and its share of the resources of the accumulation buffer 312, represents a processing element 314 of the neural network accelerator (NNA) 302. Figure 3 Three processing elements 314 are shown in FIG. 3 , but in general any number of processing elements may be present.
[0081] Each processing element 314 may receive a set of input activation values from the input buffer 306 and a set of coefficients from the coefficient buffer 330. By operating on the set of input activation values and the set of coefficients, the processing element 314 may be used to implement certain types of neural network layers, such as fully connected and / or convolutional layers. The processing element 314 of the neural network accelerator (NNA) 302 may be an independent processing subsystem of the neural network accelerator (NNA) 302 that may operate in parallel. Each processing element 314 includes a multiplication engine 308 configured to perform multiplication between the set of coefficients and the input activation value. In an example, the multiplication engine 308 may be configured to perform a fully connected operation (e.g., when implementing a fully connected layer) or a convolution operation (e.g., when implementing a convolutional layer) between the set of coefficients and the input activation value. The multiplication engine 308 may perform these operations by means of each multiplication engine 308 including a plurality of multipliers, each multiplier being configured to multiply the coefficient and the corresponding input activation value to produce a multiplication output value. The multipliers may be followed, for example, by a tree of adders arranged to compute the sum of the multiplication outputs in a manner dictated by the operations performed by the layer. In some examples, these multiply-accumulate calculations may be pipelined.
[0082] As described herein, neural networks are often described as including many layers. A relatively large number of multiplication-accumulation calculations must often be performed at the neural network accelerator (NNA) 302 in order to run operations to be performed by certain types of layers of the neural network (such as fully connected layers and / or convolutional layers). This is because the input activation data and coefficient sets of these types of layers are often very large. Since more than one pass of the multiplication engine 308 may be used to produce the complete output of the operation (e.g., because the multiplication engine 308 may only receive and process a portion of the coefficient set and input activation values), the neural network accelerator (NNA) may include multiple accumulators 310. Each accumulator 310 receives the output of the multiplication engine 308 and adds the output to the previous output of the multiplication engine 308 associated with the same operation. Depending on the implementation of the neural network accelerator (NNA) 302, the multiplication engine 308 may not process the same operation in consecutive cycles, so an accumulation buffer 312 may be provided to store the partially accumulated output of a given operation. The appropriate partial result may be provided to the accumulator 310 by the accumulation buffer 312 in each cycle.
[0083] The accumulation buffer 312 may be coupled to the output buffer 316 to allow the output buffer 316 to receive output activation data of intermediate layers of the neural network operated at the neural network accelerator (NNA) 302, and output data of a final layer (e.g., a layer that performs the final operation of the network implemented at the neural network accelerator (NNA) 302). The output buffer 316 may be coupled to the on-chip memory 328 and / or the off-chip memory 304, and the output data stored in the output buffer 316 (e.g., output activation data input as input activation data to a subsequent layer, or final output data output by the neural network) may be written to the on-chip memory and / or the off-chip memory.
[0084] In general, the neural network accelerator (NNA) 302 may also include any other suitable processing logic for implementing different types of neural network layers. For example, the neural network accelerator (NNA) 302 may include: processing logic (e.g., reduction logic) for implementing a pooling layer that performs operations such as maximum pooling or average pooling on an activation data set; processing logic (e.g., activation logic) for implementing an activation layer that applies an activation function such as a sigmoid function or a step function to an activation data set; processing logic (e.g., addition logic) for implementing an addition layer that sums data sets output by two or more other layers; and / or processing logic for implementing a flattening layer that reduces the dimensionality of a data set. A skilled person will understand how to provide suitable processing logic for implementing these types of neural network layers. For simplicity, such processing logic is not described in detail. Figure 3It will be apparent to one skilled in the art that this is not an exhaustive list of other suitable processing logic that may be included by the neural network accelerator (NNA) 302.
[0085] As described herein, the coefficient sets used by certain layers of a typical neural network (e.g., fully connected layers and / or convolutional layers) typically include a large number of coefficients. A neural network accelerator, such as neural network accelerator 302, can implement the layers by reading input activation values and coefficient sets of the layers of the neural network at runtime, such as directly reading from off-chip memory 304 or reading via on-chip memory 328, as described herein with reference to Figure 3 As described. A large amount of memory bandwidth may be required to read a large number of coefficient sets from memory. The memory bandwidth required to read in a coefficient set may be referred to as "weight bandwidth". The memory bandwidth required to read in an activation value set may be referred to as "activation bandwidth". In addition, the time it takes to read a large coefficient set from memory may also affect the time it takes for the neural network to perform the operations performed by the layer. This may affect the inference time (e.g., latency) of the neural network. Large coefficient sets may also impose large computational demands on the processing elements of a neural network accelerator that implements the neural network, such as the processing element 314 of the neural network accelerator (NNA) 302 described herein. That is, layers with large coefficient sets may require those processing elements to perform a large number of multiplication and accumulation operations.
[0086] Furthermore, the inventors have observed that typically most of the coefficients in the coefficient set of a layer (e.g., a fully connected or convolutional layer) of a typical neural network are equal to zero (e.g., "zero coefficients" or "0"). This is especially true in trained neural networks, as the training process can typically drive most of the coefficients to zero. Performing element-wise multiplication between an input activation value and a zero coefficient will inevitably produce a zero output value, regardless of the value of the input activation value.
[0087] In fact, the inventors have observed that multiple channels (e.g., input or output channels) comprised by a set of coefficients of one or more layers (e.g., fully connected or convolutional layers) of a neural network may comprise only zero coefficients (e.g., may not include any non-zero coefficients). This may be a result of a training process performed on this neural network. These channels may be referred to as redundant channels. This is because channels comprising only zero coefficients may be removed from the neural network without changing the output of this neural network, since all of the element-by-element multiplications performed using the zero coefficients comprised by the channels result in zero output values. These redundant channels comprising only zero coefficients may also make other channels comprised by other (e.g., adjacent or neighboring) layers of the neural network redundant channels that may be removed from the neural network without changing the output of the neural network. Reference Figure 2c This can be further understood.
[0088] Figure 2c A pair of example convolutional layers are shown - a first convolutional layer 200-1 followed by a second convolutional layer 200-2. For simplicity, Figure 2c The convolutional layers shown in are not configured to use bias. The first convolutional layer 200-1 is arranged to perform a convolution operation using input activation data 202-1 and coefficient set 204-1 to generate output activation data 206-1, as will be referred to herein. Figure 2a and Figure 2b The description is as follows. Figure 2c , the output activation data 206-1 generated by the first convolutional layer 200-1 is the input activation data 202-2 received by the second convolutional layer 200-2. The convolutional layer 200-2 is arranged to perform a convolution operation using the input activation data 202-2 and the coefficient set 204-2 to generate the output activation data 206-2, as will be referred to herein. Figure 2a and Figure 2b The description is understood as such.
[0089] In the example, the fourth input channel 228 (shown with cross-hatching) of the coefficient set 204-2 of the second convolutional layer 200-2 may include only zero coefficients. This may be the result of the training process performed on the neural network. Thus, the input channel 228 is a redundant channel. This is because all of the element-by-element multiplications performed using the zero coefficients included by the channel 228 result in zero output values. It can be seen that the fourth channel 222 (shown with cross-hatching) of the input activation data 202-2 of the second convolutional layer 200-2 is also a redundant channel. This is because when the channel 222 is convolved with the input channel 228, even if the channel 222 includes multiple non-zero values, these non-zero values will all be multiplied by the zero coefficient of the dedicated zero coefficient of the input channel 228. As described above, the output activation data 206-1 generated by the first convolutional layer 200-1 is the input activation data 202-2 received by the second convolutional layer 200-2. Thus, the fourth channel 224 (shown as cross-hatching) of the output activation data 206-1 generated by the first convolutional layer 200-1 is a redundant channel. As described herein, each output channel (e.g., filter) in the coefficient set 204-1 of the first convolutional layer 200-1 is responsible for forming a corresponding channel of the output activation data 206-1. In this example, the fourth output channel (e.g., filter) 220 (shown as cross-hatching) of the coefficient set 204-1 is responsible for forming the fourth channel 224 of the output activation data 206-1. Thus, it can be seen that the fourth channel (e.g., filter) 220 is also a redundant channel. This is regardless of the number and / or magnitude of the non-zero coefficient values included by the fourth output channel (e.g., filter) 220. This is because, no matter how many non-zero activation values there are in the channels 222 / 224 that the filter 220 is responsible for forming, all of these non-zero values will be multiplied by the zero coefficients of the dedicated zero coefficients of the input channel 228 in the second convolutional layer 200-2. Therefore, each of channels 220, 222 / 224, and 228 can be removed from convolutional layers 220-1 and 200-2 without changing the output of this neural network.
[0090] In short, in another example, the fourth output channel (e.g., filter) 220 (shown as cross-hatched) of coefficient set 204-1 includes only zero coefficients, which will also cause each of channels 220, 222 / 224 and 228 to be a redundant channel (regardless of the number of non-zero coefficients included by channel 228), as understood by those skilled in the art by applying equivalent logic to the channels applied in the previous paragraph.
[0091] Figure 2cA pair of example convolutional layers is shown to illustrate how redundant channels in one layer can cause channels in another layer to be redundant. It should be understood that this principle applies to more than just pairs of convolutional layers. In some examples: redundant channels in a fully connected layer can cause channels in another fully connected layer to be redundant; redundant channels in a convolutional layer can cause channels in a pooling layer or activation layer to be redundant; in some cases, redundant channels in a convolutional layer can cause channels in an addition layer to be redundant; in some cases, redundant channels in a convolutional layer can cause multiple channels in a flattening layer to be redundant. It will be apparent to one skilled in the art that this is not an exhaustive list of scenarios where redundant channels in one layer can cause channels in another layer to be redundant.
[0092] What is undesirable is to incur weight bandwidth, inference time (e.g., latency), and computational requirement deficiencies caused by performing operations using redundant coefficient channels that include only zero coefficient values, which inevitably result in zero output values for all of the element-by-element multiplications performed using these coefficient values. That is, the redundant channel incurs weight bandwidth, inference time (e.g., latency), and computational requirement "costs" during processing, but does not affect the output of the neural network. What is also undesirable is to incur an activation bandwidth "cost" for a channel that reads activation values from memory, which inevitably results in zero output values for all of the element-by-element multiplications performed using the activation values of this activation channel and the zero coefficients of this coefficient channel for only this activation channel that is operated on by the redundant coefficient channel, and thereby does not affect the output of the neural network. It is also undesirable to incur inference time (e.g., latency), computational requirements, activation bandwidth, and optional weighted bandwidth “cost” for performing operations on channels of a layer (e.g., a fully connected layer, a convolutional layer, an addition layer, a flattening layer, a pooling layer, an activation layer, or any other suitable type of layer) (channels that are responsible only for forming output activation data for subsequent operations (e.g., convolutions or multiplications) by redundant channels that only include zero values in subsequent layers of the neural network).
[0093] Methods and processing systems are described herein for compressing neural networks in order to solve one or more of the problems described in the preceding paragraphs.
[0094] Figure 4 A processing system 400 for compressing a neural network according to the principles described herein is shown. The processing system 400 includes at least one processor 404 and a memory 406.
[0095] At least one processor 404 may be implemented in hardware, software, or any combination thereof. At least one processor 404 may be a microprocessor, a controller, or any other suitable type of processor for processing computer executable instructions. At least one processor 404 may be configured to perform a method for compressing a neural network according to the principles described herein (e.g., as will be described herein with reference to Figure 5 described method).
[0096] The memory 406 is accessible by at least one processor 404. The memory 406 may be a system memory accessible to the at least one processor 404 via a data bus. The at least one processor 404 may be implemented on a chip (e.g., a semiconductor die and / or an integrated circuit package), and the memory 406 may not be physically located on the same chip (e.g., a semiconductor die and / or an integrated circuit package) as the at least one processor 404. Thus, the memory 406 may be referred to as “off-chip memory” and / or “external memory”. Alternatively, the at least one processor 404 may be implemented on a chip (e.g., a semiconductor die and / or an integrated circuit package), and the memory 406 may be physically located on the same chip (e.g., a semiconductor die and / or an integrated circuit package) as the at least one processor 404. Thus, the memory 406 may be referred to as “on-chip memory” and / or “local memory”. Alternatively, Figure 4 The memory 406 shown in may again represent any combination of "on-chip" and "off-chip" memory, such as one or more "on-chip" memories and one or more "off-chip" memories arranged in a memory hierarchy.
[0097] The memory 406 may store a method for executing a method for compressing a neural network according to the principles described herein (e.g., as will be referred to herein). Figure 5 Memory 406 may store a neural network received at processing system 400 for compression (e.g., a software model defining the neural network, as described herein). As at least one processor executes the computer-executable instructions stored by memory 406, memory 406 may store the compressed neural network output. Memory 406 may store the compressed neural network for subsequent implementation (e.g., at a system for implementing a neural network).
[0098] Processing system 400 may be used to configure system 300 for implementing a neural network. Figure 4 The system 300 shown in FIG. 3 may have the same Figure 3That is, system 300 may include hardware logic for implementing a neural network, the hardware logic including a neural network accelerator (NNA) 302. At least one processor 404 may configure the hardware logic to implement the compressed neural network output as at least one processor performs a method for compressing a neural network according to the principles described herein (e.g., as will be described herein with reference to Figure 5 The results of the method described).
[0099] Figure 5 A method for compressing a neural network is shown. Figure 5 The method of compressing a neural network shown in is a computer-implemented method. Figure 4 The processing system 400 for compressing a neural network shown in FIG. 4 may be configured to perform Figure 5 The method for compressing neural networks shown in .
[0100] In step S502, a neural network including a plurality of layers is received. The received neural network may include any number of layers - for example, 2 layers, 5 layers, 15 layers, 100 layers, or any other suitable number of layers. The received neural network may be defined by a software model. For example, the software model may define a sequence of layers in the plurality of layers of the received neural network (e.g., the number of layers, the order of the layers, and the connectivity between the layers), and define each of the layers in the sequence according to the operation it is configured to perform (and optionally, the set of coefficients it will use). The plurality of layers of the received neural network may include at least one of any one or more of the following: a fully connected layer, a convolutional layer, an addition layer, a flat layer, a pooling layer, an activation layer, and / or any other suitable type of layer. It should be understood that the received neural network need not include all of these types of layers. That is, the received neural network may not include fully connected layers, convolutional layers, addition layers, flat layers, pooling layers, and / or activation layers. As an example, the received neural network may include four convolutional layers, one pooling layer, one flattening layer, and two fully connected layers—or any other suitable number and combination of suitable layer types. The neural network (e.g., a software model defining the neural network) may be in Figure 4 The data is received at the processing system 400 and stored in the memory 406 .
[0101] The received neural network may be a trained neural network. That is, as will be appreciated by the skilled person, the received neural network may have been previously trained iteratively by: processing training data in a forward pass; evaluating the accuracy of the output of the forward pass; and updating the coefficient sets of the layers in a backward pass. As described herein, the training process may typically drive towards zero a large portion of the coefficient sets used by the fully connected and / or convolutional layers of the neural network.
[0102] In step S504, a graph is formed that represents data flow (e.g., potential non-zero data flow) through multiple layers of the neural network. Alternatively, the graph can be described as representing input-output dependencies between multiple layers of the neural network. The graph can be a multi-part graph. The graph includes multiple vertices. Each vertex in the multiple vertices represents an output channel of a layer in the multiple layers of the neural network. The graph also includes one or more edges. Each edge in the one or more edges represents a potential non-zero data flow between corresponding output channels represented by a corresponding vertex pair. Alternatively, the edges can be described as representing input-output dependencies between corresponding output channels represented by the corresponding vertex pair. Figure 4 The at least one processor 404 shown in FIG. 4 may be configured to perform step S504 .
[0103] In more detail, the graph may include multiple vertex subsets. Each vertex subset in the multiple vertex subsets may represent a corresponding layer in the multiple layers of the neural network. Each vertex subset in the multiple vertex subsets may include one or more vertices. Each vertex in the one or more vertices may represent an output channel of the corresponding layer of the neural network. Each of the one or more edges may connect two vertices, which are included by different vertex subsets of the graph. Each of the one or more edges may represent a data flow (e.g., a potential non-zero data flow) between corresponding channels of the corresponding layer of the neural network represented by those vertices.
[0104] As described herein, the plurality of layers of the received neural network may include at least one of any one or more of the following: a fully connected layer, a convolutional layer, an addition layer, a flattening layer, a pooling layer, an activation layer, and / or any other suitable type of layer. Different types of neural network layers may be represented in the figure in different ways. Figures 6a to 11c Describes examples of different types of neural network layers and how they can be represented.
[0105] The received multiple layers of the neural network may include a fully connected layer. A vertex subset of the multiple vertex subsets included by the graph may represent the fully connected layer in the multiple layers of the neural network. Each vertex of the one or more vertices included by the vertex subset may represent a corresponding output channel of the fully connected layer. As described herein, in a fully connected layer, a fully connected operation may be performed by performing a matrix multiplication between a coefficient matrix including a coefficient set of the fully connected layer and an input matrix including a set of input activation values received by the fully connected layer. The purpose of the fully connected layer is to cause a dimensional change between an activation data set input to this layer and an activation data set output from this layer.
[0106] To represent a fully connected layer in a graph, a matrix representing a set of coefficients of the fully connected layer may be determined. The determined matrix may include the set of coefficients of the fully connected layer. The determined matrix may be a coefficient matrix of the fully connected layer. The matrix may include one or more elements representing non-zero coefficients and one or more elements representing zero coefficients. Figure 6a This step can be further understood.
[0107] Figure 6a An example matrix 602 representing a set of coefficients for a fully connected layer is shown. The example matrix 602 includes a plurality of elements representing non-zero coefficients and a plurality of elements representing zero coefficients. Figure 6a , the elements of the matrix 602 representing non-zero coefficients are marked with "x", while the elements of the matrix 602 representing zero coefficients are not marked. For example, the elements located in the 1st row and the 1st column and the 1st row and the 2nd column of the matrix 602 represent non-zero coefficients, while the other elements located in the 1st row of the matrix 602 represent zero coefficients. A zero coefficient is a coefficient having a value equal to zero. A non-zero coefficient is a coefficient having a value not equal to zero.
[0108] As described in this paper, in a fully connected layer, a matrix multiplication WX=Y can be performed, where W is a matrix having a dimension C and a set of coefficients. out ×C in The coefficient matrix of . Therefore, Figure 6a As shown, each row of the determined matrix can represent the corresponding output channel (“C out ”), and each column of the determined matrix can represent the corresponding input channel of the fully connected layer (“C in ”). That is to say, Figure 6a The matrix 602 shown in represents a set of coefficients of a fully connected layer having five output channels and four input channels. Alternatively, as described herein, in a fully connected layer, a matrix multiplication XW=Y may be performed, where W is a matrix including the set of coefficients and having dimension C. in ×C out In this alternative (not shown in the figure), each row of the determined matrix can represent the corresponding input channel (“C in ”), and each column of the determined matrix can represent the corresponding output channel of the fully connected layer (“C out ”).
[0109] For each of the one or more elements of the determined matrix representing non-zero coefficients, the following can be identified: an output channel of a fully connected layer including the non-zero coefficient; an input channel of the fully connected layer including the non-zero coefficient; and an output channel of a previous layer of multiple layers of a neural network that corresponds to (e.g., serves as or is responsible for forming activation data to be operated on by) the identified input channel of the fully connected layer. It should be understood that the previous layer can be a layer of the same type (e.g., a fully connected layer in this example) or a different type of layer (e.g., a convolutional layer, an addition layer, a flattening layer, a pooling layer, an activation layer, and / or any other suitable type of layer). For each of the one or more elements of the determined matrix representing non-zero coefficients, an edge can then be used to connect a vertex in a subset of vertices representing the identified output channel of the fully connected layer to a vertex in a different subset of vertices in the multiple subsets of vertices representing the identified output channels of the previous layer. Reference Figure 7a and Figure 7b This step can be further understood.
[0110] Figure 7a can represent a sequence of fully connected layers. Figure 7a In FIG. 7 , three matrices 702 are shown representing a sequence of three fully connected layers (layers (k-1), k, and (k+1)). Figure 7a In , layer (k-1) is the previous layer of the neural network relative to layer k. The activation data set output by layer (k-1) can be the activation data set input to layer k. Figure 7a , layer k is the previous layer of the neural network relative to layer (k+1). The activation data set output by layer k can be the activation data set input to layer (k+1). It should be understood that there may be additional layers before layer (k-1) and / or after layer (k+1), such as Figure 7a . It should also be understood that there may be additional layers (such as pooling layers and / or activation layers) between the fully connected layer (k-1) and the fully connected layer k, and / or between the fully connected layer k and the fully connected layer (k+1) that cannot change the number of channels between the input activation data sets they receive and the output activation data sets they generate, as will be described in further detail herein.
[0111] Matrix 702k has reference Figure 6a The same properties as described for matrix 602. Figure 7a , matrix 702k represents the coefficient set of the fully connected layer k with four input channels and five output channels. Figure 6a Based on the principle described above, the matrix 702(k-1) can be determined for the fully connected layer (k-1). Figure 7a, matrix 702(k-1) represents the set of coefficients of a fully connected layer (k-1) having three input channels and four output channels (corresponding to the four input channels of layer k). Figure 6a Based on the principle described above, the matrix 702(k+1) can be determined for the fully connected layer (k+1). Figure 7a , matrix 702(k+1) represents the set of coefficients for a fully connected layer (k+1) having five input channels (corresponding to the five output channels of layer k) and three output channels.
[0112] Figure 7b Shown is the expression through Figure 7a Graph of data flow for multiple layers represented in . The graph includes three vertex subsets 719(k-1), 720k and 721(k+1) representing layers (k-1), k and (k+1), respectively.
[0113] For example, in more detail, the vertex subset 720k represents Figure 7a The vertex subset 720k includes five vertices to Each vertex represents one of the five output channels of layer k. That is, vertex represents the first output channel of layer k, vertex represents the second output channel of layer k, vertex Represents the third output channel of layer k, vertex represents the fourth output channel of layer k, and vertex represents the fifth output channel of layer k. Vertex subset 719(k-1) represents Figure 7a Vertex subset 719(k-1) includes four vertices to Each vertex represents a corresponding one of the four output channels of layer (k-1). That is, the vertex Represents the first output channel of layer (k-1), vertex Represents the second output channel of layer (k-1), vertex represents the third output channel of layer (k-1), and vertex Represents the fourth output channel of layer (k-1).
[0114] As described herein, matrix 702k includes elements representing non-zero coefficients located in row 1 and column 1 (marked with an "x"). The non-zero coefficients can be identified as being included by the first output channel of layer k (e.g., because the elements representing the non-zero coefficients are located in row 1 of matrix 702k). The non-zero coefficients can be identified as being included by the first input channel of layer k (e.g., because the elements representing the non-zero coefficients are located in column 1 of matrix 702k). The first output channel of the previous layer (k-1) can be identified as corresponding to the identified first input channel of layer k (e.g., responsible for forming activation data to be operated on by the first input channel). Therefore, as Figure 7b As shown, edge 722a may be used to represent a vertex in vertex subset 720k of the identified first output channel of layer k. Connected to the vertices in the vertex subset 719(k-1) representing the identified first output channel of the previous layer (k-1) In this manner, edge 722a represents an input-output dependency between the first output channel of the previous layer (k-1) and the first output channel of layer k. In particular, edge 722a represents a potential non-zero data flow between the first output channel of the previous layer (k-1) and the first output channel of layer k. Edges representing potential non-zero data flow between two channels can subsequently be evaluated (e.g., by traversing the graph as described in further detail herein) to determine whether non-zero data actually flows between those channels when the neural network is executed. In other words, the absence of an edge between two vertices indicates that there is no input-output dependency between the channels represented by those vertices—that is, there is no non-zero data flow between those channels when the neural network is executed. Edge 722a can be referred to as being relative to the vertex outgoing edge, because it represents the edge coming from the vertex The potential non-zero data flow of the output channel represented by . Edge 722a can be referred to as relative to vertex The incoming edge, because it represents the edge entering the vertex Represents a potentially non-zero data stream for the output channel.
[0115] A similar "identification" and "edge formation" process may be performed for each of the other five elements (marked with "x") in matrix 702k that represent non-zero coefficients - for the sake of brevity, this document will not be described in detail - to form edges 722b, 722c, 722c, 722e, and 722f. For elements (not marked with "x") in matrix 702k that represent zero coefficients, no edges may be formed. In this manner, a graph representing the data flow (e.g., potential non-zero data flow) between layers (k-1) and k may be formed, in this example, these layers are fully connected layers.
[0116] Vertex subset 721(k+1) represents Figure 7aVertex subset 721(k+1) includes three vertices to Each vertex represents a corresponding one of the three output channels of layer (k+1). That is, the vertex Represents the first output channel of layer (k+1), vertex represents the second output channel of layer (k+1), and vertex A similar “identification” and “edge formation” process may be performed on each of the elements (labeled with “x”) representing non-zero coefficients in matrix 702(k+1)—not described in detail herein for the sake of brevity—to form edges (at Figure 7b , but without any reference numerals assigned to them). For elements in matrix 702(k+1) that represent zero coefficients (not marked with an "x"), no edges may be formed. In this manner, a graph may be formed that represents the flow of data (e.g., potentially non-zero data flow) between layers k and (k+1), which in this example are fully connected layers.
[0117] The received plurality of layers of the neural network may include a convolutional layer. A subset of vertices in a plurality of subsets of vertices included by the graph may represent the convolutional layer in the plurality of layers of the neural network. Each vertex in the one or more vertices included by the subset of vertices may represent a corresponding output channel of the convolutional layer. As described herein, in the convolutional layer, a convolution operation may be performed using a set of input activation values received by the convolutional layer and a set of coefficients of the convolutional layer. For example, as described herein with reference to Figure 2a and Figure 2b As described above, with spatial dimension H a ×W a The Nth input channel of the input activation data 202 may be associated with the coefficient set 204 having a spatial dimension H w ×W w Convolution of each filter with the Nth input channel.
[0118] To represent the convolutional layer in the figure, a matrix representing a set of coefficients for the convolutional layer may be determined. The matrix may include one or more elements representing non-zero values and one or more elements representing zero values. Determining the matrix representing the set of coefficients for the convolutional layer may include, for each input channel of each filter (e.g., reference Figure 2b, for input channel 1 of filter A, input channel 2 of filter A, and so on for input channel 3 of filter D), determine whether this input channel of the filter includes a non-zero coefficient. In response to determining that the input channel of the filter includes at least one non-zero coefficient, the input channel of the filter can be represented by an element representing a non-zero value in a matrix. In response to determining that the input channel of the filter includes only zero coefficients (i.e., excluding any non-zero coefficients within a subset of the coefficient set of the convolutional layer), the input channel of the filter can be represented by an element representing a zero value in a matrix. For completeness, binary "1" can be used to represent non-zero values in a matrix, and binary "0" can be used to represent zero values within a matrix, but this is not necessary. For example, binary "0" can be used to represent non-zero values in a matrix, and binary "1" can be used to represent zero values within a matrix. It should also be understood that any other arbitrarily selected value can be used to represent non-zero values or zero values within a matrix. As an illustrative example, one or more bits indicating an even number can be used to represent non-zero values in a matrix, and one or more bits indicating an odd number can be used to represent zero values within a matrix. Reference Figure 6b This step can be further understood.
[0119] Figure 6b An example matrix 606 representing a set of coefficients for a convolutional layer is shown. Figure 6b , matrix 606 represents a matrix having dimension C out ×C in ×H w ×W w The coefficient set C of the convolutional layer out ×C in plane. In order to make the out ×C in ×H w ×W w The four-dimensional coefficient set of the convolutional layer can be visualized, for example, by replacing w ×W w , the coefficients of each two-dimensional input channel of each filter are represented as corresponding one-dimensional coefficient sequences to make H w ×W w The dimension is "flattened". These one-dimensional coefficient sequences are Figure 6b As shown in Figure 1, one dimension (labeled H) spanning the three-dimensional cuboid representing the coefficient set of the convolutional layer w W w ). C out ×C in Plane Figure 6b The other two dimensions of this three-dimensional cuboid are shown in . Therefore, it can be said that each one-dimensional coefficient sequence H w W w In C out ×C in In the plane, there is (Cout ,C in ) coordinates. As described herein, each element of matrix 606 represents an input channel of a filter of a convolutional layer. Figure 6b The elements marked with "x" in the matrix 606 in C that represent non-zero values represent out ×C in The plane has this coordinate (C out ,C in )'s one-dimensional coefficient sequence H w W w At least one of the coefficients in is a non-zero coefficient. Figure 6b The unmarked zero-valued elements in the matrix 606 in C out ×C in Which coordinate (C out ,C in )'s one-dimensional coefficient sequence H w W w None of the coefficients in is non-zero.
[0120] Matrix 606 includes a number of elements marked with "x" representing non-zero values, and a number of elements not marked representing zero values. Figure 6b , the convolutional layer has a set of coefficients arranged in five filters (e.g., output channels) (e.g., represented by the five rows of the matrix 606), each of which is arranged in four input channels (e.g., represented by the four columns of the matrix 606). For example, the first row of the matrix 606 may represent the first filter of the convolutional layer, and each column in the first row represents one of the four input channels of the first filter. Thus, the elements located in the first row and the first column and the first row and the second column of the matrix 606 represent the input channels of the first filter including at least one non-zero coefficient, while the other elements located in the first row of the matrix 606 represent the input channels of the first filter including only zero coefficients.
[0121] exist Figure 6b In , each row of the matrix 606 is described as representing a corresponding output channel (e.g., filter) of the convolutional layer, and each column of the matrix 606 represents a corresponding input channel of the convolutional layer. It should be understood that, alternatively, the matrix (not shown in the figure) can be determined so that each row of the matrix represents a corresponding input channel of the convolutional layer, and each column of the matrix represents a corresponding output channel (e.g., filter) of the convolutional layer. After defining the matrix to be filled in this way, the elements of the matrix can be filled accordingly by evaluating whether the input channel of the filter represented by each element includes a non-zero coefficient.
[0122] Once a matrix representing the set of coefficients for a convolutional layer has been determined as described herein, this matrix can be used to form a graph representing the flow of data through the convolutional layer in much the same way that a matrix representing the set of coefficients for a fully connected layer as described herein can be used to form a graph representing the flow of data through the fully connected layer.
[0123] That is, for each of the one or more elements of the determined matrix representing non-zero values, the following can be identified: the output channel of the convolutional layer; the input channel of the convolutional layer; and the output channel of a previous layer of the multiple layers of the neural network that corresponds to the identified input channel of the convolutional layer (e.g., as or responsible for forming the activation data to be operated by it). It should be understood that the previous layer can be a layer of the same type (e.g., a convolutional layer in this example) or a different type of layer (e.g., a fully connected layer, an addition layer, a flattening layer, a pooling layer, an activation layer, and / or any other suitable type of layer). For each of the one or more elements of the determined matrix representing non-zero values, an edge can then be used to connect a vertex in the subset of vertices representing the identified output channel of the convolutional layer to a vertex in a different subset of vertices in the multiple subsets of vertices representing the identified output channels of the previous layer. Reference Figure 7a and Figure 7b This step can be further understood.
[0124] Figure 7a can represent a sequence of convolutional layers. Figure 7a In FIG. 7 , three matrices 706 are shown representing a sequence of three convolutional layers (layers (k-1), k, and (k+1)). Figure 7a In , layer (k-1) is the previous layer of the neural network relative to layer k. The activation data set output by layer (k-1) can be the activation data set input to layer k. Figure 7a , layer k is the previous layer of the neural network relative to layer (k+1). The activation data set output by layer k can be the activation data set input to layer (k+1). It should be understood that there may be additional layers before layer (k-1) and / or after layer (k+1), such as Figure 7a . It should also be understood that there may be additional layers (such as pooling layers and / or activation layers) between convolution layer (k-1) and convolution layer k, and / or between convolution layer k and convolution layer (k+1) that cannot change the number of channels between the input activation data sets they receive and the output activation data sets they generate, as will be described in further detail herein.
[0125] Matrix 706k has the same Figure 6b The same properties as described for matrix 606. Figure 7a, matrix 706k represents the set of coefficients for convolutional layer k with four input channels and five output channels. Figure 6b Based on the principle described above, the matrix 706(k-1) can be determined for the convolution layer (k-1). Figure 7a , matrix 706(k-1) represents the set of coefficients for convolutional layer (k-1) having three input channels and four output channels (corresponding to the four input channels of layer k). Figure 6b Based on the principle described above, the matrix 706(k+1) can be determined for the convolution layer (k+1). Figure 7a , matrix 706(k+1) represents the set of coefficients for convolutional layer (k+1) having five input channels (corresponding to the five output channels of layer k) and three output channels.
[0126] Figure 7b Shown is the expression through Figure 7a Graph of data flow for multiple layers represented in . The graph includes three vertex subsets 719(k-1), 720k and 721(k+1) representing layers (k-1), k and (k+1), respectively.
[0127] For example, in more detail, the vertex subset 720k represents Figure 7a The vertex subset 720k includes five vertices to Each vertex represents one of the five output channels of layer k. That is, vertex represents the first output channel of layer k, vertex represents the second output channel of layer k, vertex Represents the third output channel of layer k, vertex represents the fourth output channel of layer k, and vertex represents the fifth output channel of layer k. Vertex subset 719(k-1) represents Figure 7a Vertex subset 719(k-1) includes four vertices to Each vertex represents a corresponding one of the four output channels of layer (k-1). That is, the vertex Represents the first output channel of layer (k-1), vertex Represents the second output channel of layer (k-1), vertex represents the third output channel of layer (k-1), and vertex Represents the fourth output channel of layer (k-1).
[0128] As described herein, the matrix 706k includes an element representing a non-zero value located in the 1st row and the 1st column (marked with an "x"). The non-zero value can be identified as corresponding to the first output channel of layer k (e.g., because the element representing the non-zero value is located in the 1st row of the matrix 706k). The non-zero value can also be identified as corresponding to the first output channel of layer k (e.g., because the element representing the non-zero value is located in the 1st column of the matrix 706k). The first output channel of the previous layer (k-1) can be identified as corresponding to the identified first input channel of layer k (e.g., responsible for forming activation data to be operated by the first input channel). Therefore, as Figure 7b As shown, edge 722a may be used to represent a vertex in vertex subset 720k of the identified first output channel of layer k. Connected to the vertices in the vertex subset 719(k-1) representing the identified first output channel of the previous layer (k-1) In this manner, edge 722a represents an input-output dependency between the first output channel of the previous layer (k-1) and the first output channel of layer k. In particular, edge 722a represents a potential non-zero data flow between the first output channel of the previous layer (k-1) and the first output channel of layer k. Edges representing potential non-zero data flow between two channels can subsequently be evaluated (e.g., as described in further detail herein) to determine whether non-zero data actually flows between those channels when the neural network is executed. In other words, the absence of an edge between two vertices indicates that there is no input-output dependency between the channels represented by those vertices—that is, there is no non-zero data flow between those channels when the neural network is executed. Edge 722a can be referred to as being relative to the vertex outgoing edge, because it represents the edge coming from the vertex The potential non-zero data flow of the output channel represented by . Edge 722a can be referred to as relative to vertex The incoming edge, because it represents the edge entering the vertex Represents a potentially non-zero data stream for the output channel.
[0129] A similar "identification" and "edge formation" process may be performed for each of the other five elements (marked with "x") in matrix 706k that represent non-zero values - not described in detail herein for the sake of brevity - to form edges 722b, 722c, 722c, 722e, and 722f. For elements (not marked with "x") in matrix 706k that represent zero values, no edge may be formed. In this manner, a graph representing the data flow (e.g., potential non-zero data flow) between layers (k-1) and k may be formed, which in this example are convolutional layers.
[0130] Vertex subset 721(k+1) represents Figure 7aVertex subset 721(k+1) includes three vertices to Each vertex represents a corresponding one of the three output channels of layer (k+1). That is, the vertex represents the first output channel (e.g., filter) of layer (k+1), vertex represents the second output channel (e.g., filter) of layer (k+1), and vertex , representing the third output channel (e.g., filter) of layer (k+1). A similar “identification” and “edge formation” process may be performed for each of the elements (labeled with “x”) representing non-zero values in matrix 706(k+1)—not described in detail herein for the sake of brevity—to form edges (at Figure 7b , but without being assigned reference numerals). For elements in matrix 706(k+1) that represent zero values (not marked with an "x"), no edges may be formed. In this manner, a graph may be formed that represents the flow of data (e.g., potentially non-zero data flow) between layers k and (k+1), which in this example are convolutional layers.
[0131] It should be understood that the (multiple) incoming edges of one or more vertices in the vertex subset representing the first layer in sequence of the multiple layers of the neural network may be connected to corresponding vertices in the input vertex subset representing the input to the multiple layers of the neural network. The input vertex subset may include one or more input vertices, each of the one or more input vertices representing a channel of input to the multiple layers of the neural network. The input to the multiple layers of the neural network may be an activation data set initially input to the neural network including the multiple layers, or an activation data set output by a previous layer of the neural network that is not included in the received multiple layers of the neural network to be compressed. In an example, the input to the multiple layers of the neural network may be an activation data set initially input to the neural network, and the first layer in sequence of the multiple layers of the neural network may be a fully connected layer or a convolutional layer. As described herein, where a matrix representing a set of coefficients of the fully connected layer or convolutional layer has been determined, for each of the one or more elements of the determined matrix representing non-zero values, the following may be identified: an output channel of the fully connected layer or convolutional layer; an input channel of the fully connected layer or convolutional layer; and an output channel of an activation data set initially input into a neural network that corresponds to (e.g., as activation data to be operated on) the identified input channel of the fully connected layer or convolutional layer. For each of the one or more elements of the determined matrix representing non-zero values, an edge may then be used to connect a vertex in a subset of vertices representing the identified output channel of the fully connected layer or convolutional layer to a vertex in a subset of input vertices representing the identified channel of the activation data set.
[0132] As referenced in this article Figure 2b The convolutional layer can be configured to use a bias such that each filter (e.g., output channel) of the coefficient set of the convolutional layer is associated with a corresponding bias. Figure 7b A diagram is shown that can represent data flow through multiple convolutional layers that are not configured to use biases. In an example where the multiple layers of the received neural network include a convolutional layer that is configured to use biases, a subset of vertices in a plurality of subsets of vertices can also include a bias vertex representing one or more biases of the convolutional layer that follows a layer in the multiple layers of the neural network represented by the subset of vertices. The bias vertex can be connected to one or more vertices in the subset of vertices representing the convolutional layer via one or more edges, each of the edges representing a non-zero bias in the one or more biases represented by the bias vertex, the bias vertex being associated with a corresponding output channel in the one or more output channels represented by the subset of vertices representing the convolutional layer. Reference Fig.10a and Fig.10b The representation of the bias in the figure can be further understood.
[0133] Fig.10a Represents a plurality of layers including a convolutional layer configured to use a bias. Fig.10a In , two matrices representing a sequence of two convolutional layers (layers k and (k+1)) are shown. Fig.10a , layer (k+1) is the subsequent layer of the neural network relative to layer k. The activation data set output by layer k can be the activation data set input to layer (k+1). It should be understood that there may be additional layers before layer k and / or after layer (k+1), such as Fig.10a . It should also be understood that there may be additional layers (such as pooling layers and / or activation layers) between convolutional layer k and convolutional layer (k+1) that cannot change the number of channels between the input activation data sets they receive and the output activation data sets they generate, as will be described in further detail herein.
[0134] exist Fig.10a , layer k is a convolutional layer. Matrix 1006k represents the coefficient set of layer k. Matrix 1006k has the same Figure 6b Matrix 1006k has the same properties as matrix 606 described in reference 1. Figure 7a The same properties as described for matrix 706k. Fig.10a , layer (k+1) is a convolutional layer configured to use a bias. The convolutional layer configured to use a bias does not affect the determination of the matrix representing the coefficient set of the convolutional layer. Thus, using the reference in this article Figure 6bBased on the principle described above, a matrix 1006(k+1) can be determined for the convolution layer (k+1). The matrix 1006(k+1) has the same Figure 7a The same properties are described for matrix 706(k+1).
[0135] Fig.10b It is shown how the bias of a layer may be represented in a graph representing the flow of data through a plurality of layers. The graph comprises two vertex subsets 1020k and 1021(k+1) representing layers k and (k+1) respectively.
[0136] Vertex subset 1020k representation Fig.10a The vertex subset 1020k includes five vertices to Each vertex represents Fig.10a The vertex subset 1021(k+1) represents one of the five output channels of layer k shown in FIG. Fig.10a The vertex subset 1021(k+1) includes three vertices to Each vertex represents Fig.10a The edges between the vertices in the vertex subsets 1020k and 1021(k+1) (in Fig.10b , but without any reference numerals assigned thereto) may be used in conjunction with reference Figure 7a and Figure 7b The edges between the vertices of the vertex subsets 720k and 721(k+1) are formed according to the determined matrix 1006(k+1) in the same manner as described above according to the determined matrix 706(k+1).
[0137] In this example, layer (k+1) is a convolutional layer configured to use a bias. Thus, the vertex subset 1020k representing layer k also includes a bias vertex representing the bias of layer (k+1). In this example, the biases for layer (k+1) include a zero bias associated with a first output channel (e.g., filter) of layer (k+1), and non-zero biases associated with a second output channel and a third output channel (e.g., filter) of layer (k+1). A zero bias is a bias having a value equal to zero. A non-zero bias is a bias having a value not equal to zero. Thus, the bias vertex Vertices that are not connected by an edge to the first output channel of layer (k+1) Offset Vertex Connected to the vertex representing the second output channel of layer (k+1) by the first edge 1034a And offset vertices Connected to the vertex representing the third output channel of layer (k+1) by the second edge 1034b In this manner, a graph can be formed that represents the data flow (e.g., potentially non-zero data flow) between layers k and (k+1), where in this example, the layers are convolutional layers, where layer (k+1) is configured to use a bias of –.
[0138] Those skilled in the art will also appreciate that the fully connected layer can be configured to use biases in a manner similar to the manner in which the convolutional layer can be configured to use biases. Fig.10a and Fig.10b The same principle described by the example shown, the bias of the fully connected layer can be represented in the graph formed in step S504. That is, in the example in which the multiple layers of the received neural network include a fully connected layer configured to use a bias, the vertex subset in the multiple vertex subsets can also include a bias vertex representing one or more biases of the fully connected layer, and the fully connected layer is after the layer in the multiple layers of the neural network represented by the vertex subset. The bias vertex can be connected to one or more vertices in the vertex subset representing the fully connected layer by one or more edges, each of the edges representing a non-zero bias in the one or more biases represented by the bias vertex, and the bias vertex is associated with a corresponding output channel in the one or more output channels represented by the vertex subset representing the fully connected layer.
[0139] It should also be understood that if a sequentially first layer of multiple layers of a neural network is configured to use biases, the input vertex subset may also include bias vertices representing one or more biases of this sequentially first layer of multiple layers of the neural network in accordance with the principles described herein.
[0140] The plurality of layers of the received neural network may include a flattening layer. A flattening layer is a type of neural network layer configured to operate on a received activation data set having a plurality of dimensions so as to generate a flattened activation data set having a number of dimensions less than the number of dimensions of the received activation data set. The flattening layer is typically located in a sequence of neural network layers between a convolutional layer and a fully connected layer. For completeness, in some rare examples, a flattening layer may be located in a sequence of neural network layers between two convolutional layers or between two fully connected layers.
[0141] A vertex subset of a plurality of vertex subsets included by a graph may represent a flattening layer in a plurality of layers of a neural network. The vertex subset may include n vertex groups, n being equal to the number of channels of data in an activation data set on which the flattening layer is configured to perform a flattening operation. Each vertex group may include m vertices, m being equal to the number of values in each channel of data in the activation data set. Each vertex included by the vertex subset representing the flattening layer may represent a corresponding output channel of the flattening layer. Each vertex included by each vertex group in the vertex subset representing the flattening layer may be connected to a vertex in a different vertex subset by a corresponding edge, the vertex representing an output channel of a previous layer in the plurality of layers of the neural network, the output channel corresponding to a channel of the activation data set on which a portion of the flattening operation represented by the vertex group is performed. Reference Figure 8a and Figure 8b The examples shown can further understand the representation of the flattening layer in the figure.
[0142] Figure 8a represents a plurality of layers including a planarization layer. In particular, Figure 8a represents an example layer sequence including convolutional layers, flattening layers, and fully connected layers. Figure 8a , layer (k-1) is a convolutional layer. Layer (k-1) is the previous layer of the neural network relative to layer k. The activation data set 812 output by layer (k-1) can be the activation data set input to layer k. Figure 8a In , layer k is the flattening layer. Layer k is the previous layer of the neural network relative to layer (k+1). The activation data set output by layer k can be the activation data set input to layer (k+1). Figure 8a In , layer (k+1) is a fully connected layer. It should be understood that there may be additional layers before layer (k-1) and / or after layer (k+1), such as Figure 8a . It should also be understood that there may be additional layers (such as pooling layers and / or activation layers) between the convolution layer (k-1) and the flattening layer k, and / or between the flattening layer k and the fully connected layer (k+1) that cannot change the number of channels between the input activation data sets they receive and the output activation data sets they generate, as will be described in further detail herein.
[0143] Use this article as a reference Figure 6b Based on the principle described above, the matrix 806(k-1) can be determined for the convolution layer (k-1). Figure 8a In , matrix 806(k-1) represents the set of coefficients for convolutional layer (k-1) with three output channels and five input channels. Figure 8a , the convolutional layer outputs an activation dataset 812 having three channels, each channel having a spatial dimension H a ×W a.exist Figure 8a Middle, H a =2 and W w =2. That is, each channel of the output activation data 812 includes four values (i.e., because 2×2=4). Figure 8a In , layer k is a flattening layer with 12 output channels. That is, in Figure 8a , the layer k output includes a one-dimensional sequence of values (e.g., 12 values (i.e., because 3×2×2=12)) of the output activation data set 812. Figure 8a ). Use the references herein Figure 6a Based on the principle described above, the matrix 802(k+1) can be determined for the fully connected layer (k+1). Figure 8a , matrix 802(k+1) represents the set of coefficients for a fully connected layer (k+1) having twelve input channels (corresponding to the twelve output channels of layer k) and two output channels.
[0144] Figure 8b Shown is the expression through Figure 8a Graph of data flow for multiple layers represented in . The graph includes three vertex subsets 819(k-1), 820k and 821(k+1) representing layers (k-1), k and (k+1), respectively.
[0145] Vertex subset 819(k-1) represents Figure 8a The convolutional layer (k-1) shown in FIG. The vertex subset 819 (k-1) includes three vertices to Each vertex represents a corresponding one of the three output channels of layer (k-1). That is, the vertex Represents the first output channel of layer (k-1), vertex represents the second output channel of layer (k-1), and vertex Represents the third output channel of layer (k-1).
[0146] Vertex subset 820k representation Figure 8a The flattened layer k shown in FIG. The vertex subset 820k includes three vertex groups 824-1, 824-2, and 824-3 (i.e., n=3). This is because Figure 8a The activation data set 812 output by the convolutional layer (k-1) shown in FIG. 8 includes three channels. Each vertex group 824-1, 824-2, and 824-3 includes four vertices (i.e., m=4). This is because Figure 8a Each channel of the activation data set 812 shown in FIG. 8 includes four values (ie, because 2×2=4). That is, vertex group 824-1 includes four vertices. to Vertex group 824-2 includes four vertices to And vertex group 824-3 includes four vertices to Each vertex included in vertex group 824-1 to Connected to the vertex representing the first output channel of layer (k-1) via corresponding edges 826a-d Vertex group 824-1 may represent the flattening of the first channel of activation data set 812 into a one-dimensional sequence of four values—this is represented by Figure 8a Each vertex included in vertex group 824-2 to Connected to the vertex representing the second output channel of layer (k-1) via corresponding edges 826e-h Vertex group 824-2 may represent the flattening of the second channel of activation data set 812 into a one-dimensional sequence of four values—this is represented by Figure 8a Each vertex included in vertex group 824-3 to Connected to the vertex representing the third output channel of layer (k-1) via corresponding edge 826i-l Vertex group 824-3 may represent the flattening of the third channel of activation data set 812 into a one-dimensional sequence of four values—this is represented by Figure 8a , and a portion of the flattening operation performed by the flattening layer k shown in FIG. In this manner, a graph can be formed that represents the data flow (e.g., the potential non-zero data flow) between layers (k-1) and k, which in this example include the convolutional layer (k-1) and the flattening layer k.
[0147] Vertex subset 821(k+1) represents Figure 8a The fully connected layer (k+1) shown in FIG. The vertex subset 821 (k+1) includes two vertices and Each vertex represents a corresponding one of the two output channels of layer (k+1). Figure 7a and Figure 7b According to the principle described above, according to the determined matrix 802(k+1), the vertices included in the vertex subset 821(k+1) can be connected through the edge (in Figure 8b , but not assigned a reference numeral) are connected to the vertices included by the vertex subset 820k. In this manner, a graph representing the data flow (e.g., potentially non-zero data flow) between layers k and (k+1) can be formed, in this example, these layers include the flattening layer k and the fully connected layer (k+1).
[0148] As described herein, in some rare examples, a flattening layer may be located in a sequence of neural network layers between two convolutional layers or between two fully connected layers. It should be understood that such a flattening layer may be located in a sequence of neural network layers between two convolutional layers or between two fully connected layers. Figure 8a and Figure 8b The same principles described for the example planarization layer shown are represented in the diagram formed in step S504.
[0149] The plurality of layers of the received neural network may include an addition layer. An addition layer is a type of neural network layer configured to perform a summation operation between respective sets of channels of a plurality of activation data sets, each of the plurality of data sets having the same number of channels. In an example, the addition layer may be used to sum activation data sets output by a plurality of convolutional layers so as to output the activation data sets to a subsequent convolutional layer. In another example, the addition layer may be used to sum activation data sets output by a plurality of fully connected layers so as to output the activation data sets to a subsequent fully connected layer.
[0150] A subset of vertices among a plurality of vertex subsets included by a graph may represent an addition layer among a plurality of layers of a neural network. The subset of vertices may include a plurality of vertices equal to a plurality of channels in each of a plurality of activation data sets that the addition layer is configured to sum, each of the plurality of activation data sets having the same number of channels. Each vertex included by the subset of vertices may represent a respective summation operation performed between respective sets of channels of the plurality of activation data sets, such that each vertex included by the subset of vertices represents a respective output channel of the addition layer. Each vertex included by the subset of vertices representing the addition layer may be connected to a vertex in a different subset of vertices by a respective edge, the vertex representing an output channel of a previous layer among the plurality of layers of the neural network, the output channel corresponding to a channel in respective sets of channels of the plurality of activation data sets, the summation operation represented by the vertex being performed between respective sets of channels. Reference Figure 9a and Figure 9b The examples shown can further understand the representation of the additive layers in the figure.
[0151] Figure 9a represents multiple layers including additive layers. In particular, Figure 9a represents an example layer sequence including a pair of convolutional layers, an addition layer, and a further convolutional layer. Figure 9a In , layer (k-1a) is a convolutional layer. Layer (k-1a) is the previous layer of the neural network relative to layer k. The activation data set output by layer (k-1a) is the activation data set input to layer k. Figure 9aIn , layer (k-1b) is a convolutional layer. Layer (k-1b) is the previous layer of the neural network relative to layer k. The activation data set output by layer (k-1b) is the activation data set input to layer k. The activation data set output by layer (k-1b) has the same number of channels as the activation data set output by layer (k-1a). Figure 9a In , layer k is an additive layer. Layer k is the previous layer of the neural network relative to layer (k+1). The activation data set output by layer k can be the activation data set input to layer (k+1). Figure 8a In , layer (k+1) is a convolutional layer.
[0152] It should be understood that there may be additional layers before layer (k-1a), before layer (k-1b) and / or after layer (k+1), such as Figure 9a . It should also be understood that there may be additional layers (such as pooling layers and / or activation layers) between the addition layer k and the convolution layer (k+1) that cannot change the number of channels between the input activation data sets they receive and the output activation data sets they generate, as will be described in further detail herein.
[0153] Use this article as a reference Figure 6b Based on the principles described above, a matrix 906(k-1a) can be determined for the convolutional layer (k-1a). The matrix 906(k-1a) represents the coefficient set of the convolutional layer (k-1a) having four input channels and five output channels. Figure 6b Based on the principles described above, a matrix 906(k-1b) can be determined for the convolutional layer (k-1b). The matrix 906(k-1b) represents the set of coefficients for the convolutional layer (k-1b) having five input channels and five output channels (equal to the number of output channels of the layer (k-1a)). Figure 6b Based on the principles described above, a matrix 906(k+1) can be determined for the convolutional layer (k+1). The matrix 906(k+1) represents a set of coefficients for the convolutional layer (k+1) having five input channels (corresponding to the five output channels of layer k) and three output channels.
[0154] exist Figure 9a In , layer k is an additive layer with five output channels (equal to the number of output channels of layers (k-1a) and (k-1b)). That is, in Figure 9a, layer k is configured to perform a summation operation between corresponding channel sets of activation datasets output by layers (k-1a) and (k-1b) so as to output an activation dataset having five channels. That is, layer k is configured to perform a summation operation between: the first channel of the first activation dataset corresponding to the first output channel of layer (k-1a) and the first channel of the second activation dataset corresponding to the first output channel of layer (k-1b), so as to generate the first channel of the output activation dataset; the second channel of the first activation dataset corresponding to the second output channel of layer (k-1a) and the second channel of the second activation dataset corresponding to the second output channel of layer (k-1b), so as to generate the second channel of the output activation dataset; the third channel of the first activation dataset corresponding to the third output channel of layer (k-1a) and the third channel of the second activation dataset corresponding to the third output channel of layer (k-1b), so as to generate the third channel of the output activation dataset; the fourth channel of the first activation dataset corresponding to the fourth output channel of layer (k-1a) and the fourth channel of the second activation dataset corresponding to the fourth output channel of layer (k-1b), so as to generate the fourth channel of the output activation dataset; and the fifth channel of the first activation dataset corresponding to the fifth output channel of layer (k-1a) and the fifth channel of the second activation dataset corresponding to the fifth output channel of layer (k-1b), so as to generate the fifth channel of the output activation dataset.
[0155] Figure 9b Shown is the expression through Figure 9a Graph of data flow for multiple layers represented in . The graph includes four vertex subsets 919(k-1a), 919(k-1b), 920k and 921(k+1) representing layers (k-1a), (k-1b), k and (k+1) respectively.
[0156] Vertex subset 919(k-1a) includes five vertices to Each vertex represents a corresponding one of the five output channels of layer (k-1a). (For ease of illustration, Figure 9b In other words, the vertex Represents the first output channel of layer (k-1a), vertex Represents the second output channel of layer (k-1a), vertex Represents the third output channel of layer (k-1a), vertex represents the fourth output channel of layer (k-1a), and vertex Represents the fifth output channel of layer (k-1a).
[0157] Vertex subset 919(k-1b) includes five vertices to Each vertex represents a corresponding one of the five output channels of layer (k-1b). (For ease of illustration, Figure 9b In other words, the vertex Represents the first output channel of layer (k-1b), vertex Represents the second output channel of layer (k-1b), vertex Represents the third output channel of layer (k-1b), vertex represents the fourth output channel of layer (k-1b), and vertex Represents the fifth output channel of layer (k-1b).
[0158] Vertex subset 920k representation Figure 9a The additive layer k shown in Figure 9a The activation data sets output by layers (k-1a) and (k-1b) are shown in FIG. 1 , and the addition layer k is configured to sum, both of which include five channels. Thus, the vertex subset 920k includes five vertices to Each vertex represents a corresponding summation operation performed between the corresponding set of channels of the activation data sets output by layers (k-1a) and (k-1b), so that each vertex to represents the corresponding output channel of the additive layer k. That is, vertex represents the summation operation between the first channel of the activation data set output by layer (k-1a) and the first channel of the activation data set output by layer (k-1b), so that the vertex represents the first output channel of the additive layer k; vertex represents the summation operation between the second channel of the activation data set output by layer (k-1a) and the second channel of the activation data set output by layer (k-1b), so that the vertex represents the second output channel of the addition layer k; vertex represents the summation operation between the third channel of the activation data set output by layer (k-1a) and the third channel of the activation data set output by layer (k-1b), so that the vertex represents the third output channel of the additive layer k; vertex represents the summation operation between the fourth channel of the activation data set output by layer (k-1a) and the fourth channel of the activation data set output by layer (k-1b), so that the vertex represents the fourth output channel of the additive layer k; and vertex represents the summation operation between the fifth channel of the activation data set output by layer (k-1a) and the fifth channel of the activation data set output by layer (k-1b), so that the vertex represents the fifth output channel of the addition layer k.
[0159] vertex Connect to the vertices through corresponding edges and Each of these vertices represents the vertex corresponding to the summed in the sum operation. represents the first channel of the activation dataset output by layers (k-1a) and (k-1b). Connect to the vertices through corresponding edges and Each of these vertices represents the vertex corresponding to the summed in the sum operation. represents the second channel of the activation dataset output by layers (k-1a) and (k-1b). Connect to the vertices through corresponding edges and Each of these vertices represents the vertex corresponding to the summed in the sum operation. The third channel of the activation dataset output by layers (k-1a) and (k-1b) represents the third output channel of layers (k-1a) and (k-1b). Connect to the vertices through corresponding edges and Each of these vertices represents the vertex corresponding to the summed in the sum operation. The fourth channel of the activation dataset output by layers (k-1a) and (k-1b) represents the fourth output channel of layers (k-1a) and (k-1b). Connect to the vertices through corresponding edges and Each of these vertices represents the vertex corresponding to the summed in the sum operation. represents the fifth channel of the activation dataset output by layers (k-1a) and (k-1b). Figure 9b , the edges mentioned in this paragraph are designated as reference numerals 930a-j. In this manner, a graph can be formed that represents the data flow (e.g., potential non-zero data flow) between layers (k-1a), (k-1b), and k, which in this example include two convolutional layers (k-1a) and (k-1b) and an addition layer k.
[0160] Vertex subset 921(k+1) represents Figure 9a The vertex subset 921(k+1) includes three vertices to Each vertex represents a corresponding one of the three output channels of layer (k+1). Figure 7a and Figure 7b According to the principle described above, according to the determined matrix 902 / 906(k+1), the vertices included in the vertex subset 921(k+1) are connected by the edge (in Figure 9b , but not assigned a reference numeral) are connected to the vertices included by vertex subset 920k. In this manner, a graph representing data flow (e.g., potentially non-zero data flow) between layers k and (k+1) can be formed, in this example, these layers include additive layer k and convolutional layer (k+1).
[0161] As described herein, an addition layer may alternatively be used to sum activation data sets output by multiple fully connected layers in order to output activation data sets to subsequent fully connected layers. It should be understood that such an addition layer may be used in accordance with the same principles as those described herein. Figure 9a and Figure 9b The same principle described for the example additive layer shown is represented in the diagram formed in step S504.
[0162] The plurality of layers of the received neural network may include a pooling layer. A pooling layer is a type of neural network layer configured to perform an operation such as maximum pooling or average pooling on an activation data set. For example, a pooling operation includes dividing each channel of the activation data set into a plurality of activation value groups, each group including a plurality of activation values, and representing each activation value group with a corresponding single value. In a maximum pooling operation, the single value representing the group is the maximum (e.g., maximum magnitude) activation value within the group. In an average pooling operation, the single value representing the group is the average (e.g., mean, median, or mode) of the plurality of activation values within the group. Pooling layers may be used to reduce a network having dimension C. in ×H a ×W a The activation data set channels H a and W a That is, the pooling layer does not change the dimension C in ×H a ×W a The activation data set channels C in Thus, because the pooling layer cannot change the number of channels between the input activation data set it receives and the output activation data set it generates, it does not need to be represented in the graph formed in step S504. That is, the neural network received in step S502 may include a pooling layer, and the plurality of vertex subsets included by the graph formed in step S504 do not need to include a vertex subset representing this pooling layer of the neural network.
[0163] The plurality of layers of the received neural network may include an activation layer. An activation layer is a type of neural network layer configured to apply an activation function (such as a sigmoid function or a step function) to each of the activation values included by the activation data set. The activation layer does not change the activation function having dimension C. in ×H a ×W a The activation data set channels C in Thus, because the activation layer cannot change the number of channels between the input activation data set it receives and the output activation data set it generates, it does not need to be represented in the graph formed in step S504. That is, the neural network received in step S502 may include an activation layer, and the plurality of vertex subsets included by the graph formed in step S504 do not need to include a vertex subset representing this activation layer of the neural network.
[0164] refer to Figures 11a to 11c The illustrated example can further understand why it is not necessary to represent the pooling layer and / or activation layer in the diagram formed in step S504.
[0165] Fig.11a represents multiple layers including pooling layers or activation layers. In particular, Fig.11a A sequence of layers including a first convolutional layer, a pooling or activation layer, and a second convolutional layer is shown.
[0166] exist Fig.11a In , two matrices representing convolutional layers (layers k and (k+1)) are shown. Fig.11a In , the convolution layer k can be considered as the previous layer of the neural network relative to the pooling layer or activation layer P / A. The activation data set output by the convolution layer k can be the activation data set input to the pooling layer or activation layer P / A. Fig.11a In , the pooling layer or activation layer P / A can be considered as the previous layer of the neural network relative to the convolution layer (k+1). The activation data set output by the pooling layer or activation layer P / A can be the activation data set input to the convolution layer (k+1). It should be understood that there may be additional layers before layer k and / or after layer (k+1), such as Fig.11a ” are indicated by ellipsis (“…”) in the .
[0167] Matrix 1106k represents the coefficient set of convolution layer k. Matrix 1106k has the same Figure 7a 1 has the same properties as the matrix 706k described above. Matrix 1106(k+1) represents the set of coefficients of convolutional layer (k+1). Matrix 1106(k+1) has the same properties as reference Figure 7a The same properties are described for matrix 706(k+1).
[0168] Fig.11b Shown is the expression through Fig.11a A first diagram of the data flow of multiple layers represented in . The first diagram includes three vertex subsets 1120k, 1136P / A and 1121(k+1) representing convolution layer k, pooling layer or activation layer P / A and convolution layer (k+1) respectively.
[0169] Vertex subset 1120k represents Fig.11a The convolutional layer k shown in FIG. The vertex subset 1120k includes five vertices to Each vertex represents Fig.11a The vertex subset 1136P / A represents one of the five output channels of the convolution layer k shown in FIG. Fig.11a 1120k. As described herein, pooling layers and activation layers cannot change the number of channels between the input activation data sets they receive and the output activation data sets they generate. Therefore, vertex subset 1136P / A includes the same number of vertices as vertex subset 1120k. That is, vertex subset 1136P / A includes five vertices. to Each vertex represents Fig.11a 1136P / A is connected to a corresponding vertex in vertex subset 1120k by a corresponding edge 1138a-e. That is, the vertex representing the first output channel of the pooling layer or activation layer P / A Connected to the vertex representing the first output channel of convolutional layer k via edge 1138a The vertex representing the second output channel of the pooling layer or activation layer P / A Connected to the vertex representing the second output channel of convolutional layer k via edge 1138b The vertex representing the third output channel of the pooling layer or activation layer P / A Connected to the vertex representing the first output channel of convolutional layer k via edge 1138c The vertex representing the fourth output channel of the pooling layer or activation layer P / A Connected to the vertex representing the fourth output channel of convolutional layer k via edge 1138d and the vertex representing the fifth output channel of the pooling layer or activation layer P / A Connected to the vertex representing the first output channel of convolutional layer k via edge 1138e In this manner, a graph may be formed that represents the data flow (e.g., potentially non-zero data flow) between layers k and P / A, which in this example include convolutional layer k and pooling layer or activation layer P / A.
[0170] Vertex subset 1121(k+1) represents Fig.11a The convolutional layer (k+1) shown in FIG. The vertex subset 1121 (k+1) includes three vertices to Each vertex represents Fig.11a The edges between the vertices of the vertex subset 1136P / A and 1121(k+1) (in Fig.11b , but without any reference numerals) can be compared with the reference layer by considering the pooling layer or activation layer P / A as the previous layer. Figure 7a and Figure 7b The edges between the vertices in the vertex subsets 720k and 721(k+1) are formed in a similar manner according to the determined matrix 1106(k+1) according to the determined matrix 706(k+1). In this manner, a graph representing the data flow (e.g., potential non-zero data flow) between layers P / A and (k+1) can be formed, in this example, these layers include the pooling layer or activation layer P / A and the convolution layer (k+1).
[0171] Fig.11c Shown is the expression through Fig.11a A second alternative graph for the data flow of the multiple layers represented in . The second graph includes two vertex subsets 1120k and 1121(k+1) representing convolution layer k and convolution layer (k+1), respectively. As described herein, because pooling layers and activation layers cannot change the number of channels between the input activation data sets they receive and the output activation data sets they generate, they do not need to be represented in the graph formed in step S504. Therefore, Fig.11a The pooling layer or activation layer P / A shown in FIG. Fig.11c In the second graph shown in . It may be advantageous not to represent pooling layers or activation layers in the graph formed in step S504, as this may reduce the time and / or processing power required to traverse this graph in step S506, as will be further described herein.
[0172] In more detail, the vertex subset 1120k represents Fig.11a The convolutional layer k shown in FIG. The vertex subset 1120k includes five vertices to Each vertex represents Fig.11a The vertex subset 1121(k+1) represents one of the five output channels of the convolutional layer k shown in FIG. Fig.11a The convolutional layer (k+1) shown in FIG. The vertex subset 1121 (k+1) includes three vertices to Each vertex represents Fig.11a A corresponding one of the three output channels of layer (k+1) shown in .
[0173] In the example where a pooling layer or activation layer is inserted between the first convolutional layer and the second convolutional layer, the first convolutional layer can be considered to be the previous layer of the neural network relative to the second convolutional layer. This is because, as described in the article, pooling layers and activation layers cannot change the number of channels between the input activation data sets they receive and the output activation data sets they generate. This means that the Nth output channel of the pooling layer or activation layer must correspond to the Nth output channel of the convolutional layer before this pooling layer or activation layer. It can be seen from this that the Nth output channel of the convolutional layer before the pooling layer or activation layer must also correspond to the Nth input channel of the convolutional layer after this pooling layer or activation layer. Therefore, in Fig.11a In the figure, convolutional layer k can also be considered as the previous layer of the neural network relative to convolutional layer (k+1).
[0174] Thus, the edges between the vertices of vertex subsets 1120k and 1121(k+1) (in Fig.11c , but without any reference numerals) can be compared with reference Figure 7a and Figure 7b The edges between the vertices in the vertex subsets 720k and 721(k+1) are formed in a similar manner based on the determined matrix 1106(k+1) as described above based on the determined matrix 706(k+1).
[0175] It should be understood that the pooling layer or activation layer can alternatively be adjacent to (e.g., before or after) any other type of layer (e.g., a fully connected layer), or between two of any other type of layers (e.g., two fully connected layers). It should also be understood that for purposes of this document, reference is made to Fig.11a To him Fig.11c For similar reasons as described above, a pooling layer and / or activation layer that is adjacent to (e.g., before or after) any other type of layer (e.g., a fully connected layer), or between two of any other type of layers (e.g., two fully connected layers) need not be represented in the graph formed in step S504. Figures 11a to 11c As can be understood by those skilled in the art, in the example in which a pooling layer or an activation layer is inserted between a first layer (e.g., a layer of any suitable layer type, such as a fully connected layer or a convolutional layer) and a second layer (e.g., a layer of any suitable layer type, such as a fully connected layer or a convolutional layer), for the purpose of forming the graph in step S504, the first layer can be considered as a previous layer of the neural network relative to the second layer.
[0176] Use the references in this article Figures 6a to 11cThe described principles can form a graph in step S504 that represents data flow (e.g., potential non-zero data flow) through multiple layers of a neural network including at least one of any one or more of the following: a fully connected layer, a convolutional layer, an addition layer, a flattening layer, a pooling layer, and / or an activation layer.
[0177] Back to Figure 5 In step S506, one or more redundant channels included by the plurality of layers of the neural network are identified by traversing the graph formed in step S504. A redundant channel is a channel included by a layer in the plurality of layers of the neural network that can be removed from the neural network without changing the output of the neural network. In other words, a redundant channel is a channel included by a layer in the plurality of layers of the neural network that does not affect the output of the neural network - for example, as described herein with reference to Figure 2c The one or more redundant channels included by the multiple layers of the neural network can be identified by traversing the graph to evaluate each edge representing potential non-zero data flow between two channels, thereby determining whether non-zero data actually flows between those channels when the neural network is executed, and identifying the one or more redundant channels included by the multiple layers of the neural network based on these determinations. In other words, traversing the graph can include sequentially evaluating each of the multiple vertices based on any incoming and / or outgoing edges of the vertex to determine whether the output channel represented by the vertex can be removed from the neural network without changing the output of the neural network. Figure 4 The at least one processor 404 shown in FIG. 4 may be configured to perform step S506. Figures 12a to 12i , Figures 13a to 13f and Figures 14a to 14f Several example implementations of step S506 are described.
[0178] Figures 12a to 12i It is shown how, in a preferred method, a graph representing data flow through multiple layers of a neural network may be traversed to identify one or more redundant channels included by the multiple layers.
[0179] Fig.15a Represents a neural network consisting of a sequence of fully connected or convolutional layers. Fig.12a Shown is the expression through Fig.15a Diagram of the data flow of multiple layers represented in . Fig.12a The vertex subsets shown in FIG. 1 , including (i) vertices H, I, J, K, and L, (ii) vertices M, N, O, P, and Q, and (iii) vertices R, S, and T, represent fully connected layers or convolutional layers (k-1), k, and (k+1), respectively, as shown in FIG. Fig.15a As shown. Multiple vertex subsets are arranged in a sequence representing a sequence in which multiple layers of a neural network are arranged. Fig.12a The subset of vertices shown and the edges connecting them can be found in the reference Figure 7a and Figure 7b The principle of description Fig.15a 1502 / 1506 (k-1), 1502 / 1506k and 1502 / 1506 (k+1) are formed - for the sake of simplicity, this is not repeated here.
[0180] Fig.12a The diagram shown in also includes a subset of input vertices 1240 representing inputs to the multiple layers of the neural network. The subset of input vertices may include one or more input vertices, each of the one or more input vertices representing a channel of input to the multiple layers of the neural network. The input to the multiple layers of the neural network may be an activation data set initially input to the neural network including the multiple layers, or an activation data set output by a previous layer of the neural network that is not included in the multiple layers of the received neural network to be compressed. Fig.12a , the input vertex subset 1240 includes three vertices E, F, and G—these three vertices correspond to Fig.15a 1240 . It should be understood that the presence of an input vertex subset (e.g., input vertex subset 1240) to the graph is optional—as explained in further detail herein.
[0181] Fig.12a The graph shown in also includes one or more output edges representing outputs from the plurality of layers of the neural network. The outputs from the plurality of layers of the neural network may be an activation data set ultimately output from the neural network including the plurality of layers, or an input to a subsequent layer of the neural network not included in the plurality of layers of the received neural network to be compressed. The one or more output edges extend from corresponding one or more vertices in the vertex subset of the sequentially last layer representing the plurality of layers of the neural network. Fig.12a , outgoing edge 1242 extends from vertices R, S, and T—these vertices represent Fig.15a The three output channels of the last fully connected or convolutional layer in the order of the multiple layers shown. It should be understood that the presence of one or more output edges (e.g., output edge 1242) to the graph is optional - as explained in further detail herein.
[0182] In a preferred method, in order to identify one or more redundant channels, each of the output edges may be assigned a first state. Assigning the first state to each of the output edges may include marking each of the output edges with a first value. For example, this Figure 12b, where each of the output edges is marked with a "1". The first value can be "1", although this does not have to be the case. For example, the first value can alternatively be "0". It should be understood that any other arbitrarily selected value can be used as the first value. It should also be understood that it is not necessary to use a digital label to assign a first state to an edge. For example, a flag can be associated with the edge to record that it has been assigned the first state; and / or a mask corresponding to all of the edges in the graph can be maintained that can be used to record the current state of each edge. The states assigned to the edges can be stored in a memory, for example Figure 4 Memory 406 is shown in FIG.
[0183] Next, the sequence of vertex subsets can be traversed. The vertex subset representing the last layer in the order of multiple layers of the neural network (e.g., Figures 12a to 12i The vertex subset including vertices R, S, T in ) to the vertex subset of the first layer in the order of multiple layers representing the neural network (for example, Figures 12a to 12i A sequence of vertex subsets is traversed by evaluating each of one or more vertices in each vertex subset to determine whether the vertex has at least one outgoing edge assigned a first state. If the vertex has at least one outgoing edge assigned a first state, each of the incoming edges of the vertex is assigned the first state. If the vertex does not have at least one outgoing edge assigned a first state, each of the incoming edges of the vertex is not assigned the first state. For example, if a vertex does not have at least one outgoing edge assigned a first state, each of the incoming edges of the vertex may be assigned a different state (e.g., a second state, such as by not marking those incoming edges at all or by marking those incoming edges with different values (e.g., "0")). This step can be described as traversing the graph in reverse topological sort order.
[0184] For completeness, an edge can be an outgoing edge of a first vertex and / or an incoming edge of a second vertex. An incoming edge of a vertex represents a potential non-zero data flow into an output channel represented by the vertex. An outgoing edge of a vertex represents a potential non-zero data flow from an output channel represented by the vertex. For example, an edge between vertex M and vertex R is an incoming edge of vertex R and an outgoing edge of vertex M.
[0185] For example, in Fig.12c In , the vertex subset including vertices R, S, and T is evaluated. Each of vertices R, S, and T has an outgoing edge marked with "1". Therefore, each of the incoming edges of vertices R, S, and T is marked with "1". Next, in Fig.12dIn , the subset of vertices including vertices M, N, O, P, and Q is evaluated. Each of vertices M, P, and Q has an outgoing edge marked with "1". Therefore, each of the incoming edges of vertices M, P, and Q is marked with "1" - even though vertex P does not have any incoming edges to mark. Each of vertices N and O has no outgoing edge marked with "1" (in fact, these vertices do not have any outgoing edges). Therefore, each of the incoming edges of vertices N and O is not marked with "1". Next, in Fig.12e , the subset of vertices including vertices H, I, J, K, and L is evaluated. Each of vertices H, I, and K has an outgoing edge marked with "1". Therefore, each of the incoming edges of vertices H, I, and K is marked with "1" - even though vertex I does not have any incoming edges to mark. Each of vertices J and L has no outgoing edge marked with "1" (in fact, vertex L does not have any outgoing edges). Therefore, each of the incoming edges of vertices J and L is not marked with "1".
[0186] Those skilled in the art will appreciate that if step S506 is performed by adding a vertex subset of the last layer in the order of the multiple layers representing the neural network (e.g., Figures 12a to 12i The same result of this reverse topological sorting order traversal step can be achieved by assigning the first state to each of the incoming edges of the vertices in the vertex subset including the vertices R, S, T in the vertices ... Figure 12b ), each of the incoming edges of the vertices in the subset of vertices representing the sequentially last layer in the plurality of layers of the neural network will then be assigned a first state (e.g., Fig.12c In this example, traversing the graph in reverse topological sort order may include traversing the graph from the second-to-last layer in the plurality of layers representing the neural network (e.g., Figures 12a to 12i ) to the first layer of the plurality of layers representing the neural network (e.g., Figures 12a to 12i The method further comprises: traversing a sequence of vertex subsets (a vertex subset including vertices H, I, J, K, and L in the vertex subset), thereby evaluating each of the one or more vertices in each vertex subset to determine whether the vertex has at least one outgoing edge assigned a first state. If the vertex has at least one outgoing edge assigned a first state, each of the incoming edges of the vertex is assigned the first state. If the vertex does not have at least one outgoing edge assigned a first state, each of the incoming edges of the vertex is not assigned the first state.
[0187] Next (e.g., subsequently), the sequence of vertex subsets may be traversed again. In a preferred example, the vertex subset of the first layer in the order of the multiple layers representing the neural network (e.g.,Figures 12a to 12i ) to a vertex subset of the second-to-last layer in the plurality of layers representing a neural network (e.g., Figures 12a to 12i A sequence of vertex subsets is traversed by evaluating each of one or more vertices in each vertex subset to determine whether the vertex has at least one incoming edge assigned a first state. If the vertex has at least one incoming edge assigned a first state, each of the outgoing edges of the vertex is assigned the first state. If those outgoing edges were previously assigned the first state, assigning the first state to those outgoing edges may include maintaining the previously assigned first state. If the vertex does not have at least one incoming edge assigned the first state, each of the outgoing edges of the vertex is not assigned the first state. For example, if those outgoing edges were previously assigned the first state, this first state may not be assigned (for example, the first value with which the edge was previously marked may be deleted). Alternatively or additionally, these outgoing edges may be assigned a different state (for example, a second state, such as by marking with a different value (for example, "0")). This step can be described as traversing the graph in a topologically sorted order.
[0188] For example, in Fig.12f In , the subset of vertices including vertices H, I, J, K, and L is evaluated. Each of vertices H and K has an outgoing edge marked with "1". Therefore, each of the outgoing edges of vertices H and K is marked with "1". Each of vertices I, J, and L has no incoming edge marked with "1" (in fact, vertex I does not have any incoming edges). Therefore, each of the outgoing edges of vertices I, J, and L is not marked with "1" - even though vertex L does not have any outgoing edges to mark. In particular, the previous label "1" on the edge between vertices I and M is deleted. Next, in Figure 12g , the subset of vertices including vertices M, N, O, P, and Q is evaluated. Each of vertices M and Q has an incoming edge marked with "1". Therefore, each of the outgoing edges of vertices M and Q is marked with "1". Each of vertices N, O, and P has no incoming edge marked with "1" (in fact, vertex P does not have any incoming edges). Therefore, each of the outgoing edges of vertices N, O, and P is not marked with "1" - even though vertices N and O do not have any outgoing edges to mark. In particular, the previous label "1" on the edge between vertices P and S is deleted.
[0189] Next (e.g., subsequently), one or more vertices that do not have any outgoing edges assigned a first state may be identified. The one or more identified vertices represent one or more redundant channels included by multiple layers of the neural network. For example, in Figure 12hIn the graph, vertices I, J, L, N, O, and P may be identified as not having any outgoing edges marked with the first value. Thus, vertices I, J, L, N, O, and P may be identified as representing redundant paths. Optionally, one or more vertices representing redundant paths may be removed from the graph. For example, Fig.12i A graph is shown with vertices I, J, L, N, O, and P removed.
[0190] It should be understood that one or more vertices in the input vertex subset (e.g., Figures 12a to 12i Vertices E, F, and G in (e.g., ) cannot be identified as representing redundant channels—e.g., even if these vertices are not assigned any outgoing edges of the first state. This allows the inputs to the multiple layers of the compressed neural network to maintain the same number of channels as the inputs to the multiple layers of the received neural network. It should also be understood that one or more output channels of the sequentially last layer in the multiple layers of the neural network (e.g., Figures 12a to 12i ) may not be identified as redundant channels - for example, even if the vertices representing these channels do not have any outgoing edges assigned the first state. This allows the outputs of the multiple layers of the compressed neural network to maintain the same number of channels as the outputs of the multiple layers of the received neural network.
[0191] Alternatively, in another preferred method (not shown in the figure), the graph can be traversed in a topological sorting order and then in a reverse topological sorting order. That is, each of the one or more vertices in the input vertex subset can be provided with a corresponding input edge, and each of the input edges of the one or more input vertices included in the input vertex subset can be assigned a first state. In this method, the outgoing edge of any bias vertex can also be assigned a first state. Next, the vertex subset sequence can be traversed from the input vertex subset to the vertex subset of the second to last layer in the order of multiple layers representing the neural network, and each of the one or more vertices in each vertex subset is evaluated to determine whether the vertex has at least one incoming edge assigned to the first state. If the vertex has at least one incoming edge assigned to the first state, each of the outgoing edges of the vertex is assigned the first state. If the vertex does not have at least one incoming edge assigned to the first state, each of the outgoing edges of the vertex is not assigned the first state. Those skilled in the art will appreciate that if step S506 begins by assigning a first state to each of the incoming edges of one or more vertices included in the vertex subset representing the first layer in the plurality of layers of the neural network, the same result of this topologically sorted order traversal step can be achieved. This is because it is inevitable when performing the described method that, having assigned the first state to each of the outgoing edges of the incoming edges of the input vertex subset and any bias vertex, each of the incoming edges of one or more vertices included in the vertex subset representing the first layer in the plurality of layers of the neural network will subsequently be assigned the first state. In this example, traversing the graph in a topologically sorted order may include traversing a sequence of vertex subsets from the vertex subset representing the first layer in the plurality of layers of the neural network to the vertex subset representing the second to last layer in the plurality of layers of the neural network, thereby evaluating each of the one or more vertices in each vertex subset to determine whether the vertex has at least one incoming edge assigned the first state. If the vertex has at least one incoming edge assigned the first state, each of the outgoing edges of the vertex is assigned the first state. If the vertex does not have at least one incoming edge assigned to the first state, each of the outgoing edges of the vertex is not assigned to the first state. Next (e.g., subsequently), the sequence of vertex subsets may be traversed in reverse topological sort order by traversing the sequence of vertex subsets from the vertex subset representing the second-to-last layer in the plurality of layers of the neural network to the vertex subset representing the first layer in the plurality of layers of the neural network, thereby evaluating each of the one or more vertices in each vertex subset to determine whether the vertex has at least one outgoing edge assigned to the first state. If the vertex has at least one outgoing edge assigned to the first state, each of the incoming edges of the vertex is assigned to the first state. If the vertex does not have at least one outgoing edge assigned to the first state, each of the incoming edges of the vertex is not assigned to the first state.Next, one or more vertices that do not have any outgoing edges assigned the first state may be identified. The one or more identified vertices represent one or more redundant channels included by the plurality of layers of the neural network. It should be understood that one or more vertices in the subset of input vertices may not be identified as representing redundant channels—e.g., even if these vertices do not have any outgoing edges assigned the first state. It should also be understood that one or more output channels of a sequentially last layer in the plurality of layers of the neural network may not be identified as redundant channels—e.g., even if the vertices representing these channels do not have any outgoing edges assigned the first state.
[0192] Figures 13a to 13g It is shown how, in a first preferred approach, a graph representing data flow through multiple layers of a neural network may be traversed to identify one or more redundant channels included by the multiple layers. Fig.13a Shown is the expression through Fig.15a As described in this paper, Fig.13a With Fig.12a The same properties of the graph shown in . In the first preferred approach, the graph can only be traversed in reverse topological sort order.
[0193] First, to identify one or more redundant channels, each of the output edges can be assigned a first state. Fig.13b It is shown in the figure, which has the same Figure 12b The same properties as shown in Fig.
[0194] Next, we can select a subset of vertices from the last layer in the order of multiple layers representing the neural network (e.g., Figures 13a to 13g The vertex subset including vertices R, S, T in ) to the vertex subset of the first layer in the order of multiple layers representing the neural network (for example, Figures 13a to 13g The method of claim 1 traverses a sequence of vertex subsets, wherein each of one or more vertices in each vertex subset is evaluated to determine whether the vertex has at least one outgoing edge assigned a first state. If the vertex has at least one outgoing edge assigned a first state, each of the incoming edges of the vertex is assigned the first state. If the vertex does not have at least one outgoing edge assigned a first state, each of the incoming edges of the vertex is not assigned the first state. For example, see Figures 13c to 13e The steps of the graph traversal shown in FIG are similar to those described in this article. Figure 12c to Figure 12e The same properties are shown in the steps of the traversal of the graph.
[0195] Those skilled in the art will appreciate that if step S506 is performed by adding a vertex subset of the last layer in the order of the multiple layers representing the neural network (e.g., Figures 13a to 13gThe same result of this reverse topological sorting order traversal step can be achieved by assigning the first state to each of the incoming edges of the vertices in the vertex subset including the vertices R, S, T in the vertices ... Fig.13b ), each of the incoming edges of the vertices in the subset of vertices representing the last layer in the order of the plurality of layers of the neural network will then be assigned a first state (e.g., Fig.13c In this example, traversing the graph in reverse topological sort order may include traversing the graph from the second-to-last layer in the plurality of layers representing the neural network (e.g., Figures 13a to 13g ) to the first layer of the plurality of layers representing the neural network (e.g., Figures 13a to 13g The method further comprises: traversing a sequence of vertex subsets (a vertex subset including vertices H, I, J, K, and L in the vertex subset), thereby evaluating each of the one or more vertices in each vertex subset to determine whether the vertex has at least one outgoing edge assigned a first state. If the vertex has at least one outgoing edge assigned a first state, each of the incoming edges of the vertex is assigned the first state. If the vertex does not have at least one outgoing edge assigned a first state, each of the incoming edges of the vertex is not assigned the first state.
[0196] Next (e.g., subsequently), one or more vertices that do not have any outgoing edges assigned a first state may be identified. The one or more identified vertices represent one or more redundant channels included by multiple layers of the neural network. It should be understood that one or more vertices in the input vertex subset (e.g., Figures 13a to 13g Vertices E, F, and G in (e.g., ) cannot be identified as representing redundant channels—e.g., even if these vertices are not assigned any outgoing edges of the first state. It should also be understood that one or more output channels of the last layer in a sequence of multiple layers of a neural network (e.g., Figures 13a to 13g ) may not be identified as redundant pathways—for example, even if the vertices representing these pathways do not have any outgoing edges assigned the first state. Fig.13f In the graph, vertices J, L, N, and O can be identified as not having any outgoing edges marked with the first value. Thus, by performing the first optimization method, vertices J, L, N, and O can be identified as representing redundant paths. Optionally, one or more vertices representing redundant paths can be removed from the graph. For example, Figure 14g The graph is shown with vertices J, L, N, and O removed.
[0197] It is understood that the first preferred method may identify fewer redundant channels than the preferred method. Figures 13a to 13g The first preferred method shown in does not identify vertices I and P as representing redundant channels. Figures 12a to 12i In the preferred method shown in , these vertices are identified as representing redundant paths. For this reason, the steps of the first preferred method can be repeated in order to identify additional vertices representing redundant paths (e.g., from Figure 13g That is, the first preferred method can be performed iteratively, for example for a plurality of iterations, until no further vertices representing redundant channels are identified.
[0198] Figures 14a to 14g It is shown how, in a second preferred method, a graph representing data flow through multiple layers of a neural network may be traversed to identify one or more redundant channels included by the multiple layers.
[0199] Fig.14a Shown is the expression through Fig.15a As described in this paper, Fig.14a With Fig.12a That is, relative to Fig.12a As shown in the figure, Fig.14a The graph shown in additionally includes a subset of input vertices (e.g. Fig.14a 1446 in ), and does not include output edges representing outputs from multiple layers (e.g. Fig.12a 1242 in ). In the second preferred approach, the graph can only be traversed in topological sort order.
[0200] First, to identify one or more redundant paths, each of the incoming edges of the input vertex subset may be assigned a first state. Fig.14b , where each of the incoming edges of vertices E, F, and G is marked with "1". In the second optimization method, the outgoing edge of any offset vertex ( Figures 14a to 14g ) may also be assigned a first state.
[0201] Next, we can select a subset of input vertices (e.g. Figures 14a to 14g ) to the vertex subset of the second-to-last layer in the plurality of layers representing the neural network (e.g., Figures 14a to 14gThe method further comprises: traversing a sequence of vertex subsets (a vertex subset including vertices M, N, O, P, and Q in the above method), thereby evaluating each of one or more vertices in each vertex subset to determine whether the vertex has at least one incoming edge assigned a first state. If the vertex has at least one incoming edge assigned a first state, each of the outgoing edges of the vertex is assigned the first state. If the vertex does not have at least one incoming edge assigned a first state, each of the outgoing edges of the vertex is not assigned the first state.
[0202] For example, in Fig.14c In , the input vertex subset including vertices E, F, and G is evaluated. Each of vertices E, F, and G has an incoming edge marked with "1". Therefore, each of the outgoing edges of vertices E, F, and G is marked with "1". Next, in Fig.14d In , the subset of vertices including vertices H, I, J, K, and L is evaluated. Each of vertices H, J, K, and L has an outgoing edge marked with "1". Therefore, each of the outgoing edges of vertices H, J, K, and L is marked with "1" - even though vertex L does not have any outgoing edges to mark. Vertex I has no incoming edges marked with "1" (in fact, vertex I does not have any incoming edges). Therefore, each of the outgoing edges of vertex I is not marked with "1". Next, in Fig.14e , the subset of vertices including vertices M, N, O, P, and Q is evaluated. Each of vertices M, N, and Q has an incoming edge marked with "1". Therefore, each of the outgoing edges of vertices M, N, and Q is marked with "1" - even though vertex N does not have any outgoing edges to mark. Each of vertices O and P has no incoming edge marked with "1" (in fact, vertex P does not have any incoming edges). Therefore, each of the outgoing edges of vertices O and P is not marked with "1" - even though vertex O does not have any outgoing edges to mark.
[0203] Those skilled in the art will appreciate that if step S506 is performed by providing a subset of vertices of the first layer in the order of the plurality of layers representing the neural network (e.g., Figures 14a to 14g The same result of this topologically sorted order traversal step can be achieved by starting with assigning the first state to each of the incoming edges of one or more vertices included in the vertex subset (the vertex subset including vertices H, I, J, K and L in the topologically sorted order traversal step). This is because when performing the described method, it is inevitable that after assigning the first state to each of the outgoing edges of any bias vertex and each of the incoming edges of the input vertex subset (for example, Fig.14b ), each of the incoming edges of one or more vertices comprised by the subset of vertices representing the sequentially first layer of the plurality of layers of the neural network will then be assigned a first state (e.g., Fig.14c In this example, traversing the graph in topologically sorted order may include traversing the graph from a sequentially first layer of multiple layers representing a neural network (e.g., Figures 14a to 14g ) to a vertex subset of the second-to-last layer in the plurality of layers representing a neural network (e.g., Figures 14a to 14g The method further comprises: traversing a sequence of vertex subsets (a vertex subset including vertices M, N, O, P, and Q in the above method), thereby evaluating each of the one or more vertices in each vertex subset to determine whether the vertex has at least one incoming edge assigned a first state. If the vertex has at least one incoming edge assigned a first state, each of the outgoing edges of the vertex is assigned the first state. If the vertex does not have at least one incoming edge assigned a first state, each of the outgoing edges of the vertex is not assigned the first state.
[0204] Next (e.g., subsequently), one or more vertices that do not have any outgoing edges assigned a first state may be identified. The one or more identified vertices represent one or more redundant channels included by multiple layers of the neural network. It should be understood that one or more vertices in the input vertex subset (e.g., Figures 14a to 14g Vertices E, F, and G in (e.g., ) cannot be identified as representing redundant channels—e.g., even if these vertices are not assigned any outgoing edges of the first state. It should also be understood that one or more output channels of the last layer in a sequence of multiple layers of a neural network (e.g., Figures 14a to 14g ) may not be identified as redundant pathways—for example, even if the vertices representing these pathways do not have any outgoing edges assigned the first state. Fig.14f In the graph, vertices I, L, N, O, and P may be identified as not having any outgoing edges assigned the first state. Thus, vertices I, L, N, O, and P may be identified as representing redundant paths. Optionally, one or more vertices representing redundant paths may be removed from the graph. For example, Figure 14g A graph is shown with vertices I, J, L, N, O, and P removed.
[0205] It is understandable that the second optimization method can identify fewer redundant channels than the optimization method. Figures 14a to 14g The first preferred method shown in FIG. 1 does not identify vertex J as representing a redundant channel, but rather Figures 12a to 12i In the preferred method shown in , this vertex is identified as representing a redundant path. For this reason, the steps of the second preferred method can be repeated in order to identify additional vertices representing redundant paths (e.g., from Figure 14g That is, the second preferred method may be performed iteratively, for example for a plurality of iterations, until no further vertices representing redundant channels are identified.
[0206] For ease of explanation, Figures 12a to 15bThe multiple layers of the neural network represented in do not include any convolutional layers or fully connected layers that are configured to use biases. A skilled person will have no difficulty applying the principles described herein to a neural network that includes convolutional layers or fully connected layers that are configured to use biases. That is, it should be understood that "bias vertices" in this article are distinguished from "vertices" - because they represent biases, not output channels. When traversing the graph, although the outgoing edges of the biased vertices can be assigned states as a result of evaluating other vertices, the biased vertices themselves are not evaluated. Biased vertices cannot be identified as representing redundant channels - for example, even if these biased vertices do not have any outgoing edges assigned to the first state.
[0207] For ease of explanation, Figures 12a to 15b The multiple layers of the neural network represented in do not include any flattening layers. A skilled person will have no difficulty in applying the principles described herein to a neural network including a flattening layer. That is, it should be understood that when all m vertices included by the vertex group including the vertex representing the output channel do not have any outgoing edges assigned a first state, that output channel of the flattening layer of the multiple layers of the neural network can only be identified as a redundant channel. This is to prevent inappropriate removal of subsets of these channels from causing dimensional mismatches in subsequent layers of the neural network. For example, returning to reference Figure 8b , after traversing the graph, if the vertex in vertex group 824-1 and does not have any outgoing edges assigned the first state, then the vertex Can only be identified as indicating redundant channels.
[0208] For ease of explanation, Figures 12a to 15b The multiple layers of the neural network represented in do not include any additive layers. A skilled person will have no difficulty in applying the principles described herein to a neural network including additive layers. That is, it should be understood that the output channels of the additive layers of the multiple layers of the neural network can only be identified as redundant channels when the vertex representing the output channel does not have any outgoing edges assigned to the first state, and all vertices representing the output channels from the previous layer connected to the vertex by an edge also do not have any outgoing edges assigned to the first state. This is to prevent improper removal of the output channels of the additive layer from causing dimensional mismatch in subsequent layers of the neural network. For example, returning to reference Figure 9b , after traversing the graph, if the vertex There are no outgoing edges assigned the first state, and the vertex and Connected to the vertex by an edge that is also not assigned a first state Then the vertex can only be identified as representing redundant channels. Further, vertices representing output channels from the previous layer connected by edges to vertices representing output channels of the additive layer can only be identified as representing redundant channels when all of these vertices do not have any outgoing edges assigned the first state. This is to prevent improper removal of subsets of these channels from causing dimensional mismatches in subsequent additive layers. For example, returning to reference Figure 9b , after traversing the graph, if the vertex There are no outgoing edges assigned the first state, and the vertex and Connected to the vertex by an edge that is also not assigned a first state Then the vertex and Any one of them can only be identified as representing a redundant channel.
[0209] Back to Figure 5 In step S508, the compressed neural network is outputted, wherein the identified one or more redundant channels are not present in the compressed neural network. In other words, the compressed neural network does not perform operations associated with the removed redundant channels. For example, for the redundant channels in the convolutional layer (e.g., for Figure 2b ), when implementing the compressed neural network, there is no need to perform the convolution operation associated with this redundant channel. This also means that when implementing the compressed neural network, the coefficients included by this redundant channel do not need to be stored in the memory (or transmitted to the accelerator). In another example, for the redundant channel in the addition layer, when implementing the compressed neural network, there is no need to perform the summation operation associated with this redundant channel. This also means that when implementing the compressed neural network, the activation value added by this redundant channel does not need to be transmitted to the accelerator. As described herein, a redundant channel is a channel included by a layer in multiple layers of a neural network that can be removed from the neural network without changing the output of the neural network. Therefore, the output of the compressed neural network for a given input is the same as the output of the received neural network for the given input. Figure 4 The at least one processor 404 shown in FIG. 4 may be configured to perform step S508 .
[0210] refer to Fig.15a and Fig.15b Step S508 can be further understood. Fig.15a represents the received neural network comprising a sequence of fully connected or convolutional layers. As described herein, the preferred method may Figures 12a to 12i Vertices I, J, L, N, O, and P shown in FIG. 1 are identified to represent redundant channels. Fig.15b represents a compressed neural network consisting of a sequence of fully connected or convolutional layers in which one or more redundant channels are absent. That is, Fig.15b, the rows I, J, L, N, O, and P representing the output channels represented by vertices I, J, L, N, O, and P and the columns representing the corresponding input channels in the previous layer have been removed from the matrices representing the fully connected or convolutional layers (k-1), k, and (k+1). The fully connected or convolutional layer (k-1) still includes three input channels E, F, and G. The fully connected or convolutional layer (k+1) still includes three output channels R, S, and T. Therefore, the inputs to and outputs from the multiple layers are not changed.
[0211] That is to say, according to the reference in this article Figure 5 The described method of compressing a received neural network is advantageous because, when the compressed neural network is implemented, it does not incur weight bandwidth, activation bandwidth, inference time (e.g., latency), and computational requirements disadvantages associated with performing operations using redundant coefficient channels. This allows the compressed neural network to perform fewer multiple accumulation operations, thereby reducing its weight bandwidth, activation bandwidth, inference time (e.g., latency), and computational requirements relative to executing (e.g., implementing) the received neural network. In a manner similar to "pure" data compression techniques (e.g., where a received data set is compressed according to a compression scheme so as to output a compressed data set that requires fewer bits to store), according to the present invention, the compressed neural network is compressed according to a compression scheme described in reference to the present invention. Figure 5 The described method of compressing the received neural network is also advantageous because fewer bits are required to store the compressed neural network compared to the received neural network. This is because bits associated with (e.g., defining) the identified redundant channels do not need to be stored. This means that the compressed neural network has a smaller memory footprint (e.g., consumes less space in memory) when stored.
[0212] Step S508 may include storing the compressed neural network for subsequent implementation. Figure 4 , at least one processor 404 may write the compressed neural network into the memory 406 for subsequent implementation. Step S508 may include outputting a computer-readable description of the compressed neural network, which is useful when used in a system for implementing the neural network (e.g., Figure 3 When implemented at the system 300 shown in FIG. 5 , the compressed neural network is executed. Step S508 may include configuring the system for implementing the neural network to implement the compressed neural network.
[0213] Step S508 may include configuring hardware logic to implement the compressed neural network. The hardware logic may include a neural network accelerator. For example, referring to Figure 4 , at least one processor 404 can be configured to configure hardware logic included in the system 300 for implementing a neural network to implement a compressed neural network.
[0214] The compressed neural network output in step S508 may be used. The compressed neural network output in step S508 may be used to perform image processing. The compressed neural network output in step S508 may receive image data representing one or more images and perform image processing on the received image data. As a non-limiting example, the compressed neural network may be used to perform one or more of image super-resolution processing, semantic image segmentation processing, object detection, and image classification. For example, in an image super-resolution processing application, image data representing one or more lower resolution images may be input to a neural network, and the output of the neural network may be image data representing one or more higher resolution images. In another example, in an image classification application, image data representing one or more images may be input to a neural network, and the output of the neural network may be data indicating the probability (or set of probabilities) that each of the images belongs to a particular classification (or set of classifications).
[0215] There is a synergy between the method of compressing a neural network described herein and implementing a compressed neural network in hardware - i.e., implementing the compressed neural network by configuring hardware logic including a neural network accelerator (NNA). This is because the method of compressing a neural network is intended to improve the implementation of a compressed neural network at a system where coefficient sets would be stored in an off-chip memory, and the layers of the compressed neural network would be executed by reading those coefficient sets from the off-chip memory into the hardware logic including the neural network accelerator (NNA) at runtime. That is, the method described herein is particularly advantageous when used to compress a neural network for thereby implementing in hardware.
[0216] Fig.16 16 shows a computer system that can implement the neural network system described herein. The computer system includes a CPU 1602, a GPU 1604, a memory 1606 (e.g., equivalent to Figure 3 Memory 304 shown in FIG. 1 ), neural network accelerator (NNA) 1608 (e.g., equivalent to Figure 3 ), and other devices 1614, such as a display 1616, a speaker 1618, and a camera 1622 (e.g., equivalent to Figure 3 16). In some examples, camera 1622 may input activation data directly into neural network accelerator (NNA) 1608. In some examples, the NNA may be a GPU with one or more hardware accelerators for accelerating one or more neural network operations. It should be understood that one or more of the depicted components may be omitted from the system. The components of the computer system may communicate with each other via communication bus 1620.
[0217] Figure 3 , Figure 4 and Fig.16 The system is shown as comprising several functional blocks. This is merely schematic and is not intended to define a strict division between different logical elements of such an entity. Each functional block may be provided in any suitable manner. It should be understood that intermediate values described herein as being formed by the system need not be physically generated by the system at any point in time, and may simply represent logical values that conveniently describe the processing performed by the system between its inputs and outputs.
[0218] The processing system described herein can be embodied in hardware on an integrated circuit. The processing system described herein can be configured to perform any of the methods described herein. In general, any of the functions, methods, techniques or components described above can be implemented in software, firmware, hardware (e.g., fixed logic circuits) or any combination thereof. The terms "module", "functionality", "component", "element", "unit", "block" and "logic" can be used herein to generally represent software, firmware, hardware or any combination thereof. In the case of software implementation, a module, functionality, component, element, unit, block or logic represents a program code, which performs a specified task when executed on a processor. The algorithms and methods described herein can be performed by one or more processors that execute code, which prompts the processor to perform the algorithm / method. Examples of computer-readable storage media include random access memory (RAM), read-only memory (ROM), optical disk, flash memory, hard disk storage, and other memory devices that can use magnetic, optical and other technologies to store instructions or other data and can be accessed by a machine.
[0219] As used herein, the terms computer program code and computer readable instructions refer to any kind of executable code for a processor, including code expressed in a machine language, an interpreted language, or a scripting language. Executable code includes binary code, machine code, byte code, code that defines an integrated circuit (such as a hardware description language or a netlist), and code expressed in a programming language code such as C, Java, or OpenCL. Executable code can be, for example, any kind of software, firmware, script, module, or library that, when properly executed, processed, interpreted, compiled, run in a virtual machine or other software environment, causes a processor of a computer system supporting the executable code to perform the tasks specified by the code.
[0220] A processor, computer or computer system may be any kind of device, machine or special purpose circuit, or a collection or part thereof, which has processing capabilities so that instructions can be executed. A processor may be or include any kind of general or special purpose processor, such as a CPU, GPU, NNA, system on chip, state machine, media processor, application specific integrated circuit (ASIC), programmable logic array, field programmable gate array (FPGA), etc. A computer or computer system may include one or more processors.
[0221] The present invention is also intended to cover software (e.g., HDL (Hardware Description Language) software) that defines the configuration of hardware as described herein, which is used to design integrated circuits or to configure programmable chips to perform desired functions. That is, a computer-readable storage medium may be provided on which is encoded a computer-readable program code in the form of an integrated circuit definition data set, which, when processed (i.e., run) in an integrated circuit manufacturing system, configures the system to manufacture a processing system configured to perform any of the methods described herein, or to manufacture a processing system that includes any of the devices described herein. The integrated circuit definition data set may be, for example, an integrated circuit description.
[0222] Thus, a method of manufacturing a processing system as described herein at an integrated circuit manufacturing system may be provided. Furthermore, an integrated circuit definition data set may be provided which, when processed in an integrated circuit manufacturing system, causes the method of manufacturing a processing system to be performed.
[0223] The integrated circuit definition data set may be in the form of computer code, for example as a netlist, code for configuring a programmable chip, as a hardware description language that defines hardware suitable for fabrication at any level in an integrated circuit, including as register transfer level (RTL) code, as a high-level circuit representation such as Verilog or VHDL, and as a low-level circuit representation such as OASIS (RTM) and GDSII. Higher-level representations that logically define hardware suitable for fabrication in an integrated circuit, such as RTL, may be processed at a computer system configured to generate a manufacturing definition of the integrated circuit in the context of a software environment that includes definitions of circuit elements and rules for combining those elements to generate a manufacturing definition of the integrated circuit so defined by the representation. As is typically the case with software executed at a computer system to define a machine, one or more intermediate user steps (e.g., providing commands, variables, etc.) may be required to configure the computer system to generate a manufacturing definition of the integrated circuit to execute code that defines the integrated circuit to generate the manufacturing definition of the integrated circuit.
[0224] Now refer to Fig.17An example of processing an integrated circuit definition data set at an integrated circuit manufacturing system to configure the system into a manufacturing processing system is described.
[0225] Fig.17 An example of an integrated circuit (IC) manufacturing system 1702 is shown, which is configured to manufacture a processing system as described in any of the examples herein. Specifically, the IC manufacturing system 1702 includes a layout processing system 1704 and an integrated circuit generation system 1706. The IC manufacturing system 1702 is configured to receive an IC definition data set (e.g., defining a processing system as described in any of the examples herein), process the IC definition data set, and generate an IC (e.g., which embodies a processing system as described in any of the examples herein) based on the IC definition data set. The processing of the IC definition data set configures the IC manufacturing system 1702 to manufacture an integrated circuit embodying a processing system as described in any of the examples herein.
[0226] The layout processing system 1704 is configured to receive and process an IC definition data set to determine a circuit layout. Methods for determining a circuit layout based on an IC definition data set are known in the art and may, for example, involve synthesizing RTL code to determine a gate-level representation of a circuit to be generated, for example, in terms of logic components (e.g., NAND, NOR, AND, OR, MUX, and FLIP-FLOP components). By determining location information for the logic components, the circuit layout may be determined by the gate-level representation of the circuit. This may be done automatically or with user involvement in order to optimize the circuit layout. When the layout processing system 1704 has determined the circuit layout, it may output the circuit layout definition to the IC generation system 1706. The circuit layout definition may be, for example, a circuit layout description.
[0227] As is known in the art, IC generation system 1706 generates an IC according to a circuit layout definition. For example, IC generation system 1706 may implement a semiconductor device manufacturing process for generating an IC, which may involve a multi-step sequence of photolithography and chemical processing steps during which electronic circuits are gradually formed on a wafer made of semiconductor material. The circuit layout definition may be in the form of a mask that may be used in a photolithography process for generating an IC according to the circuit definition. Alternatively, the circuit layout definition provided to IC generation system 1706 may be in the form of a computer readable code that IC generation system 1706 may use to form a suitable mask for generating an IC.
[0228] The different processes performed by IC manufacturing system 1702 may all be implemented in one location, for example, by one party. Alternatively, IC manufacturing system 1702 may be a distributed system such that some processes may be performed at different locations and may be performed by different parties. For example, some of the following stages may be performed at different locations and / or by different parties: (i) synthesizing RTL code representing an IC definition data set to form a gate-level representation of a circuit to be produced; (ii) generating a circuit layout based on the gate-level representation; (iii) forming a mask based on the circuit layout; and (iv) using the mask to manufacture the integrated circuit.
[0229] In other examples, processing of an integrated circuit definition data set at an integrated circuit manufacturing system may configure the system to manufacture a processing system without processing the IC definition data set to determine a circuit layout. For example, the integrated circuit definition data set may define a configuration of a reconfigurable processor such as an FPGA, and processing of the data set may configure the IC manufacturing system to produce a reconfigurable processor having the defined configuration (e.g., by loading the configuration data into the FPGA).
[0230] In some embodiments, when processed in an integrated circuit manufacturing system, the integrated circuit manufacturing definition data set can cause the integrated circuit manufacturing system to generate an apparatus as described herein. For example, by using the integrated circuit manufacturing definition data set as described above with reference to Fig.17 The processing system described herein can be manufactured by configuring an integrated circuit manufacturing system in the manner described.
[0231] In some examples, the integrated circuit definition data set may include software that runs on, or in combination with, hardware defined at the data set. Fig.17 In the example shown, the IC generation system can be further configured by the integrated circuit definition dataset to load firmware onto the integrated circuit according to the program code defined at the integrated circuit definition dataset when manufacturing the integrated circuit, or otherwise provide the integrated circuit with program code for use with the integrated circuit.
[0232] Compared with known specific implementations, the specific implementation of the concepts set forth in this application in devices, equipment, modules and / or systems (and in the methods implemented herein) can improve performance. Performance improvements can include one or more of improved computing performance, reduced latency, increased throughput and / or reduced power consumption. During the manufacture of such devices, devices, modules and systems (e.g., in integrated circuits), a trade-off can be made between performance improvements and physical implementations to improve manufacturing methods. For example, a trade-off can be made between performance improvements and layout area to match the performance of known implementations, but using less silicon. For example, this can be accomplished by reusing functional blocks in a serial manner or sharing functional blocks between elements of devices, devices, modules and / or systems. In contrast, the concepts set forth in this application that cause improvements in the physical implementations of devices, devices, modules and systems (e.g., reduced silicon area) can be weighed against performance improvements. This can be accomplished, for example, by manufacturing multiple instances of a module within a predefined area budget.
[0233] Applicants hereby independently disclose each individual feature described herein and any combination of two or more such features, to the extent that such feature or combination can be implemented based on the specification as a whole in view of the common general knowledge of a person skilled in the art, regardless of whether such feature or combination of features solves any problem disclosed herein. In view of the foregoing description, it will be clear to a person skilled in the art that various modifications can be made within the scope of the invention.
Claims
1. A computer-implemented method for compressing a neural network, the method include: Receiving a neural network comprising a plurality of layers; forming a graph representing a flow of data through the plurality of layers of the neural network, the graph comprising: a plurality of vertices, each vertex in the plurality of vertices representing an output channel of a layer in the plurality of layers of the neural network; and one or more edges, each edge of the one or more edges representing a potential non-zero data flow between corresponding output channels represented by a corresponding pair of vertices; identifying one or more redundant channels comprised by the plurality of layers of the neural network by traversing the graph; and A compressed neural network is output, wherein the identified one or more redundant channels are not present in the compressed neural network.
2. The method according to claim 1, in: A redundant channel is a channel included by a layer of the plurality of layers of the neural network that can be removed from the neural network without changing an output of the neural network.
3. A method as claimed in claim 1 or 2, comprising traversing the graph in reverse topological sort order and / or topological sort order.
4. The method according to claim 1 or 2, wherein the include: a plurality of vertex subsets, each vertex subset in the plurality of vertex subsets representing a corresponding layer in the plurality of layers of the neural network, each vertex subset in the plurality of vertex subsets comprising one or more vertices, each vertex in the one or more vertices representing an output channel of the corresponding layer of the neural network; as well as One or more edges, each of the one or more edges: connecting two vertices, the two vertices being included in different subsets of vertices of the graph; and Representing potential non-zero data flow between corresponding channels of the corresponding layer of the neural network represented by these vertices.
5. The method of claim 4, wherein a vertex subset of the plurality of vertex subsets represents a fully connected layer of the plurality of layers of the neural network, each vertex of the one or more vertices included in the vertex subset represents a corresponding output channel of the fully connected layer, and wherein the graph is formed include: Determining a matrix representing a set of coefficients of the fully connected layer, the matrix comprising one or more elements representing non-zero coefficients and one or more elements representing zero coefficients; For each element of the one or more elements representing a non-zero coefficient: Logo: output channels of the fully connected layer including the non-zero coefficients; the input channels of the fully connected layer including the non-zero coefficients; as well as an output channel of a previous layer of the plurality of layers of the neural network, the output channel corresponding to the identified input channel of the fully connected layer; as well as Vertices in the subset of vertices representing the identified output channels of the fully-connected layer are connected using edges to vertices in a different subset of vertices in the plurality of subsets of vertices representing the identified output channels of the previous layer.
6. The method of claim 4, wherein a vertex subset in the plurality of vertex subsets represents a convolutional layer in the plurality of layers of the neural network, each vertex in the one or more vertices included in the vertex subset represents a corresponding output channel of the convolutional layer, and forming the graph include: Determining a matrix representing a set of coefficients of the convolutional layer, the matrix comprising one or more elements representing non-zero values and one or more elements representing zero values; For each of the one or more elements representing a non-zero value: Logo: The output channel of the convolutional layer; The input channel of the convolutional layer; as well as an output channel of a previous layer of the plurality of layers of the neural network, the output channel corresponding to the identified input channel of the convolutional layer; as well as Vertices in the subset of vertices representing the identified output channels of the convolutional layer are connected using edges to vertices in a different subset of vertices in the plurality of subsets of vertices representing the identified output channels of the previous layer.
7. The method of claim 6, wherein the convolutional layer comprises a set of coefficients arranged in one or more filters, each of the one or more filters being arranged in one or more channels, each channel of each filter comprising a respective subset of the set of coefficients of the convolutional layer, and wherein determining the matrix include: For each channel of each filter: determining whether the channel of the filter includes non-zero coefficients; and In response to determining that the channel of the filter includes at least one non-zero coefficient, representing the channel of the filter using elements of the matrix representing non-zero values; or In response to determining that the channel of the filter includes only zero coefficients, the channel of the filter is represented using elements in the matrix representing zero values.
8. The method of claim 4, wherein the subset of vertices in the plurality of subsets of vertices further comprises a bias vertex, the bias vertex representing one or more biases of a layer in the plurality of layers of the neural network that is subsequent to the layer represented by the subset of vertices in the plurality of layers of the neural network, the bias vertex being connected to one or more vertices in the subset of vertices representing the subsequent layer by one or more edges, each of the edges representing a non-zero bias in the one or more biases represented by the bias vertex, the bias vertex being associated with a corresponding output channel in the one or more output channels represented by the subset of vertices representing the subsequent layer.
9. The method according to claim 4, in: a vertex subset of the plurality of vertex subsets representing an addition layer of the plurality of layers of the neural network, the vertex subset including a number of vertices equal to the number of channels of each activation data set in a plurality of activation data sets that the addition layer is configured to sum, each activation data set in the plurality of activation data sets having the same number of channels, each vertex included by the vertex subset representing a respective summation operation performed between respective sets of channels of the plurality of activation data sets, such that each vertex included by the vertex subset represents a respective output channel of the addition layer; and Each vertex included by the vertex subset is connected to a vertex in a different vertex subset by a corresponding edge, the vertex representing an output channel of a previous layer of the plurality of layers of the neural network, the output channel corresponding to a channel in the corresponding set of channels of the plurality of activation data sets, and the sum operation represented by the vertex is performed between the sets of the corresponding channels.
10. The method according to claim 4, in: a vertex subset of the plurality of vertex subsets representing a flattening layer of the plurality of layers of the neural network, the vertex subset comprising n vertex groups, n being equal to the number of channels of data of the activation data set on which the flattening layer is configured to perform a flattening operation, each vertex group comprising m vertices, m being equal to the number of values in each channel of data of the activation data set, each vertex included by the vertex subset representing a corresponding output channel of the flattening layer; as well as Each vertex included by each vertex group in the vertex subsets is connected to vertices in a different vertex subset by a corresponding edge, the vertex representing an output channel of a previous layer of the plurality of layers of the neural network, the output channel corresponding to a channel of the activation data set on which a portion of the flattening operation represented by the vertex group is performed.
11. The method of claim 4, wherein the plurality of subsets of vertices are arranged in a sequence representing a sequence in which the plurality of layers of the neural network are arranged, and wherein the one or more redundant channels are identified include: assigning a first state to each of the incoming edges of the vertices in the subset of vertices representing a sequentially last layer of the plurality of layers of the neural network; traversing the sequence of vertex subsets from a vertex subset representing a sequentially penultimate layer of the plurality of layers of the neural network to a vertex subset representing a sequentially first layer of the plurality of layers of the neural network, thereby evaluating each vertex of the one or more vertices in each vertex subset to determine whether the vertex has at least one outgoing edge assigned the first state, and: If the vertex has at least one outgoing edge assigned the first state, assigning the first state to each of the incoming edges of the vertex; and If the vertex does not have at least one outgoing edge assigned the first state, not assigning the first state to each of the incoming edges of the vertex; Subsequently, the sequence of vertex subsets is traversed from a vertex subset representing the sequentially first layer of the plurality of layers of the neural network to a vertex subset representing the sequentially second to last layer of the plurality of layers of the neural network, thereby evaluating each vertex of the one or more vertices in each vertex subset to determine whether the vertex has at least one incoming edge assigned the first state, and: If the vertex has at least one incoming edge assigned the first state, assigning the first state to each of the outgoing edges of the vertex; and If the vertex does not have at least one incoming edge assigned the first state, causing each of the outgoing edges of the vertex to not be assigned the first state; and Subsequently, one or more vertices that do not have any outgoing edges assigned the first state are identified, the one or more identified vertices representing the one or more redundant channels included by the plurality of layers of the neural network.
12. The method of claim 4, wherein the plurality of subsets of vertices are arranged in a sequence representing a sequence in which the plurality of layers of the neural network are arranged, and wherein the one or more redundant channels are identified include: assigning a first state to each of the incoming edges of the one or more vertices comprised by the subset of vertices representing a sequentially first layer of the plurality of layers of the neural network; traversing the sequence of vertex subsets from a vertex subset representing the sequentially first layer of the plurality of layers of the neural network to a vertex subset representing the sequentially penultimate layer of the plurality of layers of the neural network, thereby evaluating each vertex of the one or more vertices in each vertex subset to determine whether the vertex has at least one incoming edge assigned the first state, and: If the vertex has at least one incoming edge assigned the first state, assigning the first state to each of the outgoing edges of the vertex; and If the vertex does not have at least one outgoing edge assigned the first state, not assigning the first state to each of the outgoing edges of the vertex; Subsequently, the sequence of vertex subsets is traversed from a vertex subset representing the sequentially second-to-last layer of the plurality of layers of the neural network to a vertex subset representing the sequentially first layer of the plurality of layers of the neural network, thereby evaluating each vertex of the one or more vertices in each vertex subset to determine whether the vertex has at least one outgoing edge assigned the first state, and: If the vertex has at least one outgoing edge assigned the first state, assigning the first state to each of the incoming edges of the vertex; If the vertex does not have at least one outgoing edge assigned the first state, causing each of the incoming edges of the vertex to not be assigned the first state; and Subsequently, one or more vertices that do not have any outgoing edges assigned a first state are identified, the one or more identified vertices representing the one or more redundant channels included by the plurality of layers of the neural network.
13. The method of claim 4, wherein the plurality of subsets of vertices are arranged in a sequence representing a sequence in which the plurality of layers of the neural network are arranged, and wherein the one or more redundant channels are identified include: assigning a first state to each of the incoming edges of the vertices in the subset of vertices representing a sequentially last layer of the plurality of layers of the neural network; traversing the sequence of vertex subsets from a vertex subset representing a sequentially penultimate layer of the plurality of layers of the neural network to a vertex subset representing a sequentially first layer of the plurality of layers of the neural network, thereby evaluating each vertex of the one or more vertices in each vertex subset to determine whether the vertex has at least one outgoing edge assigned the first state, and: If the vertex has at least one outgoing edge assigned the first state, assigning the first state to each of the incoming edges of the vertex; and If the vertex does not have at least one outgoing edge assigned the first state, not assigning the first state to each of the incoming edges of the vertex; and Subsequently, one or more vertices that do not have any outgoing edges assigned the first state are identified, the one or more identified vertices representing the one or more redundant channels included by the plurality of layers of the neural network.
14. The method of claim 4, wherein the plurality of subsets of vertices are arranged in a sequence representing a sequence in which the plurality of layers of the neural network are arranged, and wherein the one or more redundant channels are identified include: assigning a first state to each of the incoming edges of the one or more vertices comprised by the subset of vertices representing a sequentially first layer of the plurality of layers of the neural network; traversing the sequence of vertex subsets from a vertex subset representing the sequentially first layer of the plurality of layers of the neural network to a vertex subset representing the sequentially penultimate layer of the plurality of layers of the neural network, thereby evaluating each vertex of the one or more vertices in each vertex subset to determine whether the vertex has at least one incoming edge assigned the first state, and: If the vertex has at least one incoming edge assigned the first state, assigning the first state to each of the outgoing edges of the vertex; and If the vertex does not have at least one outgoing edge assigned the first state, not assigning the first state to each of the outgoing edges of the vertex; and Subsequently, one or more vertices that do not have any outgoing edges assigned the first state are identified, the one or more identified vertices representing the one or more redundant channels included by the plurality of layers of the neural network.
15. The method according to claim 11, in: The edge can be an outgoing edge of the first vertex and / or an incoming edge of the second vertex; An incoming edge of a vertex represents a potential non-zero flow of data into an output channel represented by the vertex; and The outgoing edges of a vertex represent the potential non-zero data flow from the output channel represented by the vertex.
16. The method of claim 1 or 2, further comprising storing the compressed neural network for subsequent implementation.
17. The method of claim 1 or 2, further comprising outputting a computer-readable description of the compressed neural network, the computer-readable description causing the compressed neural network to be executed when implemented at a system for implementing a neural network.
18. The method of claim 1 or 2, further comprising configuring hardware logic to implement the compressed neural network, wherein the hardware logic comprises a neural network accelerator.
19. A processing system for compressing a neural network, the processing system comprising at least one processor, the at least one processor being configured to: Receiving a neural network comprising a plurality of layers; forming a graph representing a flow of data through the plurality of layers of the neural network, the graph comprising: a plurality of vertices, each vertex in the plurality of vertices representing an output channel of a layer in the plurality of layers of the neural network; and one or more edges, each edge of the one or more edges representing a potential non-zero data flow between corresponding output channels represented by a corresponding pair of vertices; identifying one or more redundant channels comprised by the plurality of layers of the neural network by traversing the graph; and A compressed neural network is output, wherein the identified one or more redundant channels are not present in the compressed neural network.
20. A non-transitory computer-readable storage medium having stored thereon computer-readable instructions which, when executed at a computer system, cause the computer system to perform a computer-implemented method of compressing a neural network, the method include: Receiving a neural network comprising a plurality of layers; forming a graph representing a flow of data through the plurality of layers of the neural network, the graph comprising: a plurality of vertices, each vertex in the plurality of vertices representing an output channel of a layer in the plurality of layers of the neural network; and one or more edges, each edge of the one or more edges representing a potential non-zero data flow between corresponding output channels represented by a corresponding pair of vertices; identifying one or more redundant channels comprised by the plurality of layers of the neural network by traversing the graph; and A compressed neural network is output, wherein the identified one or more redundant channels are not present in the compressed neural network.
