An apparatus and method for implementing a tensor chain decomposition operation

Abstract

The present application relates to a device for implementing tensor chain decomposition operation on a convolution layer of a convolutional neural network. The device receives input data comprising a first number of channels, and performs a 1x1 convolution on the input data to obtain a plurality of data groups, wherein the plurality of data groups comprises a second number of channels. The device further performs a grouped convolution on the plurality of data groups to obtain intermediate data comprising a third number of channels. In addition, the device performs a 1x1 convolution on the intermediate data to obtain output data comprising a fourth number of channels.

Description

TECHNICAL FIELD

[0001] The present disclosure relates generally to the field of data processing, and more specifically to convolutional neural networks. An apparatus and method for implementing tensor chain decomposition operations on a convolution layer of a convolutional neural network are disclosed. For example, the apparatus and method can perform hardware-friendly tensor chain decomposition operations, which can speed up the operation of the convolutional neural network. BACKGROUND

[0002] Generally, deep learning is a machine learning technique that trains a neural network to perform a task. The neural network can be a convolutional neural network. For example, a convolutional neural network can learn how to perform a task, such as a classification task related to computer vision, natural language processing, speech recognition, etc.

[0003] Conventional convolutional neural networks achieve different accuracies. In addition, it is desirable to find a convolutional neural network that can achieve a certain accuracy to solve a particular problem. However, when using deeper convolutional neural networks, for example, to further improve accuracy, these convolutional neural networks can become slower in terms of floating point operations (FLOP) and can become even slower when operating in a user device. For example, for a convolutional neural network that includes a convolution layer with 512 feature maps, the computation can require as many as 115 MFLOP operations, so these convolution layers can significantly reduce the inference time.

[0004] Tensor decomposition is proposed as a technique to reduce the computational cost. Tensor decomposition techniques are a class of methods that represent a high-dimensional tensor as a series of low-cost operations in order to reduce the number of tensor parameters and compress data.

[0005] Conventional tensor decomposition methods can be based on tensor chain decomposition, which is used for data compression, i.e., reducing the ratio of the original tensor size to the compressed size.

[0006] However, when conventional tensor chain decomposition is applied to the convolution layer of a convolutional neural network, all the above problems cannot still be overcome. SUMMARY

[0007] In view of the above problems and disadvantages, embodiments of the present disclosure aim to improve the application of tensor chain decomposition operations to the convolution layer of a convolutional neural network (CNN).

[0008] Embodiments can reduce the computational complexity of a CNN. In addition, embodiments facilitate hardware-friendly tensor chain decomposition of the convolution layer.

[0009] The implementation can select one or more convolutional layers of the CNN for decomposition, for example, determining the optimal decomposition order in the CNN.

[0010] Therefore, the goal is to provide a device and method that can efficiently perform tensor chain decomposition operations on the convolutional layers of CNNs.

[0011] This objective is achieved through embodiments of the invention described in the appended independent claims. Advantageous implementations of the embodiments of the invention are further defined in the dependent claims.

[0012] A first aspect of the present invention provides an apparatus for performing tensor chain decomposition operations on convolutional layers of a CNN. The apparatus is configured to: receive input data comprising a first number of channels; perform a 1x1 convolution on the input data to obtain multiple data groups, wherein the multiple data groups comprise a second number of channels; perform a grouped convolution on the multiple data groups to obtain intermediate data comprising a third number of channels; and perform a 1x1 convolution on the intermediate data to obtain output data comprising a fourth number of channels.

[0013] The device can be an electronic device or can be contained within an electronic device, such as a personal computer, server computer, client computer, laptop and notebook computer, tablet device, mobile phone, smartphone, surveillance camera, etc.

[0014] This device can be used to perform tensor chaining operations on the convolutional layers of a CNN. For example, the device can replace the convolutional layers of a CNN with tensor chaining operations. This operation can include tensor compression algorithms.

[0015] Typically, a tensor can be a multidimensional array containing multiple elements. For example, tensor A can be represented as follows:

[0016] A = (A, i1, i2, ..., i...) d ]), i k ∈{1, 2, ..., n k}

[0017] In addition, tensors are typically... The rank r = (r0, r1, ..., r) d The tensor chain decomposition (TT) of a matrix can be a representation where each tensor element is a matrix product, for example:

[0018]

[0019] Where, r0 = r d =1. Here, the word "chain" can be used to emphasize the similarity to a train car sequence.

[0020] A CNN is a deep learning neural network in which one or more building blocks are based on convolution operations.

[0021] In particular, the device can receive input data (e.g., an input tensor) comprising a first number of channels. The input data can be related to any type of data, e.g., image data, text data, speech data, etc. Further, the device can perform a 1x1 convolution on the input data, such that a plurality of data groups can be obtained.

[0022] For example, the device can perform a convolution operation, which can be, for example, an operation that converts an input feature map having a first number of channels to an output feature map having a second number of channels, in particular by convolving the input feature map with a convolution kernel. Without limiting the present invention to this particular example, an example of a convolution operation can convert an input feature map (C is the input channel) to an output feature map (S is the output channel) by convolving the input feature map

[0023]

[0024] The device of the first aspect can implement a tensor chain decomposition for a three-dimensional convolution tensor, in which a plurality of kernel size dimensions are combined together. For example, the tensor chain decomposition can be applied as follows:

[0025]

[0026] Further, the tensor chain convolution layer can be as follows:

[0027]

[0028] Further, the device can obtain a plurality of data groups comprising a second number of channels, intermediate data comprising a third number of channels, and output data comprising a fourth number of channels.

[0029] Compared to conventional decomposition methods, the decomposition of a convolution layer performed by the device can substantially reduce the computational cost. In particular, the decomposition performed by the device improves the speed on real hardware. Further, the implementation of the device of the first aspect can consider which convolution layer or layers are beneficial for decomposition and can further consider the order of decomposition of these layers.

[0030] In one implementation of the first aspect, the grouped convolution is performed based on a shared kernel shared between the plurality of data groups.

[0031] ​In particular, the device can perform grouped convolutions with shared kernels between groups. Moreover, performing grouped convolutions based on shared kernels shared between the plurality of data groups can enable additional speedup of tensor chainwise convolutions, e.g., by adding underlying operations (e.g., kernel fusion).

[0032] In a further implementation form of the first aspect, the third number of channels is determined based on a number of data groups of the plurality of data groups.

[0033] In a further implementation form of the first aspect, the third number of channels is further determined based on one or more hardware characteristics of the device.

[0034] For example, the implementation of the tensor chainwise decomposition operation is hardware friendly and does not require expensive data migration operations, and can significantly speed up the inference phase of CNNs. In particular, the device can obtain an optimal rank of the convolutional layers such that data migration related to reshaping operations, permutation operations, etc. can be avoided, and the speed of the processing hardware can be further boosted.

[0035] In a further implementation form of the first aspect, each data group comprises a fifth number of channels, and the second number of channels is determined based on the third number of channels and the fifth number of channels.

[0036] In a further implementation form of the first aspect, the device is further configured to obtain a CNN comprising a first number of convolutional layers, wherein each convolutional layer is associated with a respective first ordering number; provide, based on a training of the CNN, a decomposed CNN comprising a second number of convolutional layers and a third number of decomposed convolutional layers, wherein the first number is equal to a sum of the second number and the third number, and each decomposed convolutional layer is associated with a respective second ordering number.

[0037] For example, the device can obtain an optimal order of highly optimized convolutions with lower rank tensor representation and layer decomposition.

[0038] In a further implementation form of the first aspect, the device is further configured to determine, for a convolutional layer of the CNN, a weighted pair calculated based on: a weighted convolutional layer, wherein the weighted convolutional layer is obtained by assigning a first weighted trainable parameter to the convolutional layer; and a weighted decomposed convolutional layer, wherein the weighted decomposed convolutional layer is obtained by assigning a second weighted trainable parameter to a decomposed convolutional layer determined for the convolutional layer.

[0039] For example, the weighted pair can be op(x, a). Further, the first weighted trainable parameter can be "a" and the second weighted trainable parameter can be "1-a". The first and / or second weighted trainable parameter is trainable, i.e. these parameters can change during training.

[0040] Further, the device can determine a weighted pair op(x, a) of the convolutional layer Conv(x) such that

[0041] op(x, a) = a * Conv(x) + (1-a) * DConv(x), where a can be in the range [0, 1].

[0042] In other words, the convolutional layer can be weighted according to a first weighted trainable parameter "a" and the decomposed convolutional layer according to a second weighted trainable parameter "1-a".

[0043] In a further implementation form of the first aspect, the device is further configured to perform an initial training iteration of the CNN based on the at least one weighted pair.

[0044] In a further implementation form of the first aspect, the device is further configured to determine at least one convolutional layer having a minimum first weighted trainable parameter after performing the initial training iteration.

[0045] In a further implementation form of the first aspect, the device is further configured to perform an additional training iteration of the CNN by replacing the weighted pair of the convolutional layer having the minimum first weighted trainable parameter by its decomposed convolutional layers and based on the remaining part of the at least one weighted pair from the last iteration.

[0046] In a further implementation form of the first aspect, the device is further configured to iteratively perform the following steps: determining a convolutional layer having a minimum first weighted trainable parameter; replacing the weighted pair of the convolutional layer having the minimum first weighted trainable parameter by its decomposed convolutional layers; performing a next training iteration until a determined number of convolutional layers is replaced by its respective decomposed convolutional layers.

[0047] In a further implementation form of the first aspect, the device comprises an artificial intelligence accelerator suitable for tensor processing operations of the CNN.

[0048] The second aspect of the present application provides a method for implementing tensor chain decomposition operation on a convolution layer of a convolutional neural network, wherein the method comprises: receiving input data comprising a first number of channels; performing 1x1 convolution on the input data to obtain a plurality of data groups, wherein the plurality of data groups comprises a second number of channels; performing grouped convolution on the plurality of data groups to obtain intermediate data comprising a third number of channels; performing 1x1 convolution on the intermediate data to obtain output data comprising a fourth number of channels.

[0049] In an implementation form of the second aspect, the grouped convolution is performed based on shared kernels shared between the plurality of data groups.

[0050] In another implementation form of the second aspect, the third number of channels is determined based on a number of data groups in the plurality of data groups.

[0051] In another implementation form of the second aspect, the third number of channels is further determined based on one or more hardware characteristics of the device.

[0052] In another implementation form of the second aspect, each data group comprises a fifth number of channels, and the second number of channels is determined based on the third number of channels and the fifth number of channels.

[0053] In another implementation form of the second aspect, the method further comprises: obtaining a CNN comprising a first number of convolution layers, wherein each convolution layer is associated with a corresponding first ranking number; based on training of the CNN, providing a decomposed CNN comprising a second number of convolution layers and a third number of decomposed convolution layers, wherein the first number is equal to a sum of the second number and the third number, and each decomposed convolution layer is associated with a corresponding second ranking number.

[0054] In another implementation form of the second aspect, the method further comprises, for a convolution layer of the CNN, determining a weighted pair calculated based on: a weighted convolution layer, wherein the weighted convolution layer is obtained by assigning a first weighted trainable parameter to the convolution layer; and a weighted decomposed convolution layer, wherein the weighted decomposed convolution layer is obtained by assigning a second weighted trainable parameter to a decomposed convolution layer determined for the convolution layer.

[0055] In another implementation form of the second aspect, the method further comprises performing an initial training iteration of the CNN based on at least one weighted pair.

[0056] In another implementation form of the second aspect, the method further comprises, after performing the initial training iteration, determining at least one convolution layer having a minimum first weighted trainable parameter.

[0057] In a further implementation form of the second aspect, the method further comprises performing an additional training iteration of the CNN by replacing the pair of weights of the convolutional layer having the smallest first trainable parameter with its decomposed convolutional layer and based on the remaining part of the at least one pair of weights from the last iteration.

[0058] In a further implementation form of the second aspect, the method further comprises iteratively performing the following steps: determining a convolutional layer having a smallest first trainable parameter; replacing the pair of weights of the convolutional layer having the smallest first trainable parameter with its decomposed convolutional layer; performing a next training iteration until a determined number of convolutional layers is replaced with its respective decomposed convolutional layer.

[0059] In a further implementation form of the second aspect, the method is for a device comprising an artificial intelligence accelerator adapted for tensor processing operations of a CNN.

[0060] The method of the third aspect achieves the advantages and effects described for the transmitter device of the first aspect.

[0061] The third aspect of the application provides a computer program comprising program code for performing the method according to the second aspect or any implementation form thereof.

[0062] The fourth aspect of the application provides a non-transitory storage medium storing executable program code which, when executed by a processor, performs the method according to the second aspect or any implementation form thereof.

[0063] It has to be noted that all devices, elements, units and means described in the present application can be implemented in software or hardware elements or any combination thereof. All steps which are described in the present application to be performed by the various entities or functional units have to be seen as examples potentially only and not as a limitation to the present application. Functional units or elements of the present application can be implemented by one or more hardware components or software components such as e.g. a processor. It is further noted that the different steps of the methods described in the present application can be performed in any order or simultaneously. BRIEF DESCRIPTION OF DRAWINGS

[0064] The above described aspects and their implementation forms can be better understood with the following description of specific embodiments in conjunction with the attached drawings, in which:

[0065] Figure 1 A device for implementing tensor chain decomposition operations for convolutional layers of a CNN according to embodiments of the application is described;

[0066] Figure 2 Tensor chain decomposition of a three-dimensional convolution tensor is described;

[0067] Figure 3 Performing a 1x1 convolution is described;

[0068] Figure 4 A flowchart of a method for tensor chain decomposition operation is described;

[0069] Figure 5 A flowchart of a method for obtaining a decomposed CNN based on CNN training is described;

[0070] Figure 6 Replacing a convolution layer with a weighted convolution is described;

[0071] Figure 7 Replacing a pair of weighted convolutions of a convolution layer with its decomposed convolution layer is described;

[0072] Figure 8 Changing a set of weighted pairs and its corresponding convolution layer is described;

[0073] Figure 9 A flowchart of a method for implementing a tensor chain decomposition operation on a convolution layer of a convolutional neural network according to an embodiment of the present invention is described. DETAILED DESCRIPTION

[0074] Figure 1 An apparatus 100 for implementing a tensor chain decomposition operation on a convolution layer of a CNN according to an embodiment of the present invention is shown.

[0075] The apparatus 100 can be a computer, a personal computer, a smartphone, a surveillance camera, or the like electronic device.

[0076] The apparatus 100 is configured to receive input data 110 comprising a first number of channels.

[0077] The apparatus 100 is further configured to perform a 1x1 convolution on the input data 110 to obtain a plurality of data groups 120. The plurality of data groups 120 comprises a second number of channels.

[0078] The apparatus 100 is further configured to perform a grouped convolution on the plurality of data groups 120 to obtain intermediate data 130. The intermediate data 130 comprises a third number of channels.

[0079] The apparatus 100 is further configured to perform a 1x1 convolution on the intermediate data 130 to obtain output data 140. The output data 140 comprises a fourth number of channels.

[0080] The apparatus 100 can implement a tensor chain convolution operation on a convolution layer of a CNN.

[0081] Device 100 can be tuned more accurately and can be speed-upped on real hardware, for example, by not using different ranks for the tensor chain kernel.

[0082] For example, device 100 can perform a sequence of 1x1 convolutions, grouped convolutions with shared weights, and another 1x1 convolution to achieve hardware-friendly tensor chain decomposition. Furthermore, by using weight sharing in the grouped convolutions, device 100 can improve speed on real hardware due to weight reuse and reduced data transfer, and can avoid time-consuming permutation and reshaping operations.

[0083] Device 100 may include processing circuitry. Figure 1 (Not shown in the image), the processing circuitry is used to perform, conduct, or initiate various operations of the device 100 described herein. The processing circuitry may include hardware and software. The hardware may include analog or digital circuitry, or both. The digital circuitry may include components such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), or multi-purpose processors. In one embodiment, the processing circuitry system includes one or more processors and non-transitory memory connected to the one or more processors. The non-transitory memory may carry executable program code that, when executed by the one or more processors, causes the device 100 to perform, conduct, or initiate the operations or methods described herein.

[0084] Figure 2 The process of performing tensor chaining on a three-dimensional convolutional tensor is illustrated schematically. For example, device 100 can perform the tensor chaining shown on a three-dimensional convolutional tensor.

[0085] Specifically, device 100 can receive input data 110 including C channels (a first number of channels).

[0086] Device 100 can also perform 1x1 convolutions from the C channel to the R1R2 channels. For example, device 100 can perform a 1x1 convolution on input data 110 to obtain multiple data sets 120 including a second number of channels. Figure 2 In the diagram, the second number of channels is R1R2.

[0087] The device 100 can also perform a lxl grouped convolution on the plurality of data groups 120 having R1R2 channels to obtain intermediate data 130 having R2 channels (a third number of channels). For example, the device 100 can perform a grouped convolution that shares kernel weights. In Figure 2 In the diagram 200 of FIG. 2, the plurality of data groups 120 includes three data groups 221, 222, 223, and the grouped convolution is performed based on shared kernels that are shared among the data groups 221, 222, 223.

[0088] The device 100 can also perform a lxl convolution from the R2 channels to the S channels. For example, the device 100 can perform a lxl convolution on the intermediate data 130 to obtain output data 140 including S channels (a fourth number of channels).

[0089] In Figure 2 In the diagram 200 of FIG. 2, the tensor chain decomposition operation is represented as three convolutions, where the second convolution is a grouped convolution that shares kernel weights.

[0090] Figure 3 A process of performing a lxl convolution is schematically illustrated.

[0091] Figure 3 The diagram 300 is an exemplary illustration in which the device 100 can perform a first lxl convolution on the input data 110 including C number of channels to obtain data groups 320 including R channels (a second number of channels).

[0092] The device 100 can also perform a second lxl convolution on the data groups 320 to obtain output data including S channels (a fourth number of channels).

[0093] An example of the tensor chain decomposition operation can be as follows:

[0094]

[0095] Figure 4 A flowchart of a method 400 for the tensor chain decomposition operation is illustrated. The method 400 can be performed by the device 100, as described above.

[0096] At step 401, the device 100 can obtain input data 110. The input data 110 can include a series of image filters

[0097] At step 402, the device 100 can perform a lxl convolution on the input data 110. For example, the device 100 can convolve X with a kernel G0, where, The device 100 can also obtain X0 = Conv(X, G0), where,

[0098]

[0099] In step 403, device 100 may perform grouped convolution. For example, device 100 may perform grouped convolution X0 with kernel G1, where... G1 is shared on group R2. Device 100 can also obtain X1 as follows:

[0100] X1 = SharedGroupConv(X0, G1, R2), where,

[0101] In step 404, device 100 can convolve with kernel G2 X1, where, Device 100 can also obtain Y = Conv(X1, G2), where,

[0102] In step 405, device 100 can obtain output data 140. Output data 140 can be a series of output filters, wherein...

[0103] Now for reference Figure 5 The figure illustrates a flowchart of a method 500 for obtaining decomposed convolutional layers of a CNN. Method 500 can be executed by device 100, as described above.

[0104] In step 501, the device may obtain a CNN comprising a first number (L) of convolutional layers. For example, device 100 may receive an input architecture A with L convolutional layers from a dataset D.

[0105] In step 502, device 100 can convert each convolutional layer into a Conv l (x l Replace with weighted pair Op l (x l α l Device 100 can also initialize each α with a value of 0.5. l .

[0106] exist Figure 6 Figure 600 illustrates an exemplary illustration of replacing convolutional layers with weighted convolutions. Figure 6 Figure 600 shows that, for example, device 100 can replace all L-convolutional layers with weighted convolutions.

[0107] In step 503, device 100 can cycle C for k=1 to k=K.

[0108] At step 504, the device 100 can train the CNN with the op(x) instead of the regular convolution for m rounds. For example, the device 100 can perform an initial training iteration of the CNN A based on the at least one weighted pair op(x, a) and the at least one weighted convolution layer a*Conv(x).

[0109] At step 505, the device 100 can determine the convolution layer Conv(x) with the smallest weighting parameter a after performing the initial training iteration. For example, the device 100 can find the convolution layer with the smallest weight a according to the following condition: l

[0110]

[0111] At step 506, the device 100 can determine whether In addition, when the device 100 determines “yes”, the device 100 enters step 507, and when determines “no”, the device 100 returns to step 509.

[0112] At step 507, the device 100 can replace the weighted pair op(x, a) of the convolution layer Conv(x) with the smallest weighting parameter a with its decomposed convolution layer DConv(x).

[0113] Figure 7 FIG. 700 illustrates an example illustration of replacing the weighted pair of the convolution layer with its decomposed convolution. Figure 7 FIG. 700 illustrates, for example, the device 100 replaces to the corresponding

[0114] At step 508, the device 100 can increase k by 1, and can further return to step 503 for K times. (For example, K = 10)

[0115] At step 509, the device 100 can change the remaining L-k weighted pairs l (x l , a l ) to the corresponding convolution layers Conv l (x l ).

[0116] Figure 8 FIG. 700 illustrates an example illustration of replacing the weighted pair of the convolution layer with its decomposed convolution.

[0117]

[0118] where net(x) is the output of the neural network, and D is the data of the training examples (x, y). ​

[0119] At step 510, the device 100 can train the model M for m rounds. For example, the device 100 can perform an additional training iteration of the CNN A by replacing the weighted pair op(x, a) of the convolutional layer Conv(x) with the smallest weight parameter a with its decomposed convolutional layer DConv(x), based on the remaining part of the at least one weighted pair op(x, a) and the remaining part of the at least one weighted convolutional layer a*Conv(x) from the last iteration.

[0120] At step 511, the device 100 can evaluate the model M on test data.

[0121] At step 512, the device 100 can return the trained model M with k decomposed layers. For example, the device 100 can obtain a decomposed CNN M comprising a second number of convolutional layers and a third number k of decomposed convolutional layers.

[0122] In the following, an example of performing the device 100 is discussed, but the present application is not limited to this particular example.

[0123] First, the device 100 selects the ranks R1, R2 for the 3x3 convolutional layers and the rank R for the 1x1 convolution.

[0124] The device 100 can perform matrix multiplication operations. For example, the device 100 can split a large matrix into parts of a predetermined size (e.g. 16, but any device-specific number can be used) and can also perform the multiplication operation part by part. Furthermore, if the number of channels cannot be divided by 16, the channels can be padded with zeros until their number can be divided by 16.

[0125] The device 100 can also use R2 = 16, because the last convolution in the tensor chainwise convolution operates with this number of channels, where R1 = S / (4*R2). Thus, the device 100 can use the following conditions:

[0126] For example, if C = 512, S = 512, l = 3, then:

[0127] • The first convolution is a mapping from 512 channels to 128 channels.

[0128] • The second convolution is a 3x3 grouped convolution from 128 channels to 16 channels, where the number of groups is 16. Thus, in this convolution, the device 100 shares 3x3x8x1 shaped weights between 16 groups.

[0129] • The last convolution is a mapping from 16 channels to 512 channels.

[0130] Moreover, without limiting the present application, a comparison of the total number of floating point operations obtained by the device 100 and some conventional devices is presented, respectively. Thus, the following notation is used: N is the batch size, C is the number of input channels, S is the number of output channels, l is the kernel size, R1,..., R d are the original tensor train decomposition operation (TTConv) ranks, R1, R2 are related to the TTConv ranks obtained by the device 100, R is the TRConv (tensor train convolution) rank obtained by a conventional device.

[0131]

[0132]

[0133] Some examples (for convenience, N = 1, l = 3):

[0134] Next, the results obtained by the device 100 (based on performing the tensor train decomposition operation TTConv) are compared with previous implementations of the object detection task. Using a YOLO-based model, the last three layers of the decomposition process are as follows:

[0135] • The last three convolutional layers are converted from the related model to TTConv using the TT-SVD algorithm with fixed ranks. One of the convolutions has C = 256 and S = 512 channels, respectively, and the other two convolutions have C = 512 and S = 512 channels, respectively.

[0136] • The model is trained using the three TTConv layers.

[0137] • The device 100 measures the inference time.

[0138] The results show that using the device 100 (implementing the tensor train decomposition operation or TTConv) is more reasonable than the original operation.

[0139]

[0140] Next, the inference improvement of a single layer is calculated using the device 100. This layer is part of the ResNet50 backbone model.

[0141] In addition, the original convolutional layer is compared with the results obtained by the device 100.

[0142] C, S l, step size (s) Regular Conv inference TTConv inference 256、512 l＝1,s＝2 0.046 ms 0.033 ms (-28%) 512、512 l＝3,s＝1 0.059 ms 0.03 ms (-47%) 512、512 l＝3,s＝2 0.058 ms 0.32 ms (-45%) 512、2048 l＝1,s＝1 0.042 ms 0.023 ms (-45%) 1024、2048 l＝1,s＝2 0.056 ms 0.023 ms (-59%)

[0143] The results show that using TTConv can speed up a single convolutional layer in a real device. Therefore, it can be concluded that the TTConv performed by the device 100 is hardware friendly.

[0144] Moreover, the training operation performed by the device 100 can also improve the model quality. For example, ResNet34 is chosen as a model with good quality on the ImageNet dataset. The ResNet model comprises four 4-levels, wherein the number of channels increases with the level, in the case of ResNet34, the fourth level comprises only 512 channels convolutions.

[0145]

[0146] As ResNet34_stage, the device 100 uses: a model wherein all the convolutions in these levels are replaced with TTConv; ResNet34_auto - a model wherein all the convolutions in these levels are replaced with op(x, a) and trained with our training procedure.

[0147] Moreover, it can be concluded that using the proposed TTConv improves the model inference, for example, this model inference can be derived from the data shown on the last column. Moreover, it can be concluded that using the training performed by the device 100, the optimal layer can be determined.

[0148] Figure 9 A method 900 for implementing a tensor chain decomposition operation on a convolutional layer of a convolutional neural network according to an embodiment of the present application is shown. As mentioned above, the method 900 can be performed by the device 100.

[0149] The method 900 comprises a step 901 for receiving input data 110 comprising a first number of channels.

[0150] The method 900 further comprises a step 902 for performing a 1x1 convolution on the input data 110 to obtain a plurality of data groups 120, wherein the plurality of data groups 120 comprises a second number of channels.

[0151] The method 900 further comprises a step 903 for performing a grouped convolution on the plurality of data groups 120 to obtain intermediate data 130 comprising a third number of channels.

[0152] The method 900 further comprises a step 904 for performing a 1x1 convolution on the intermediate data 130 to obtain output data 140 comprising a fourth number of channels.

[0153] The application has been described in connection with the various embodiments presented herein as well as those presented in the following claims as examples and implementations. However, changes can be made in light of the above description and the attached claims, and in the knowledge of those skilled in the art. In the claims and the specification, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single element or other unit can fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

1. A computer (100) for implementing a tensor chain decomposition operation for a convolution layer of a convolutional neural network, CNN, for processing image data, text data or speech data, characterized in that, The computer (100) is configured to: receive input data (110) comprising a first number of channels, the input data (110) being image data, text data or speech data; perform a 1x1 convolution on the input data (110) to obtain a plurality of data groups (120), wherein the plurality of data groups (120) comprises a second number of channels; perform a grouped convolution based on shared kernels shared between the plurality of data groups (120) to obtain intermediate data (130) comprising a third number of channels; perform a 1x1 convolution on the intermediate data (130) to obtain output data (140) comprising a fourth number of channels, the output data (140) being processed image data, text data or speech data.

2. The computer (100) according to claim 1, wherein the third number of channels is determined based on a number of data groups in the plurality of data groups (120).

3. The computer (100) according to claim 2, wherein the third number of channels is further determined based on one or more hardware characteristics of the computer (100).

4. The computer (100) according to any one of claims 1-3, wherein each data group (221, 222, 223) comprises a fifth number of channels, the second number of channels is determined based on the third number of channels and the fifth number of channels.

5. The computer (100) according to any one of claims 1 to 3, characterized in that is further configured to: obtain a CNN comprising a first number of convolutional layers, wherein each convolutional layer is associated with a respective first rank number; provide, based on a training of the CNN, a factorized CNN comprising a second number of convolutional layers and a third number of factorized convolutional layers; wherein the first number is equal to a sum of the second number and the third number, and each factorized convolutional layer is associated with a respective second rank number.

6. The computer (100) of claim 5, characterized by is further configured to, for a convolutional layer of the CNN, determine a pair of weights computed based on: a weighted convolutional layer, wherein the weighted convolutional layer is obtained by assigning a first weight trainable parameter to the convolutional layer; a weighted factorized convolutional layer, wherein the weighted factorized convolutional layer is obtained by assigning a second weight trainable parameter to a factorized convolutional layer determined for the convolutional layer.

7. The computer (100) of claim 6, characterized by is further configured to: perform an initial training iteration of the CNN based on at least one pair of weights.

8. The computer (100) of claim 7, characterized by is further configured to: after performing the initial training iteration, determine at least one convolutional layer having a smallest first weight trainable parameter.

9. The computer (100) of claim 8, characterized by is further configured to: perform an additional training iteration of the CNN by replacing the pair of weights of the convolutional layer having the smallest first weight trainable parameter with its factorized convolutional layer and based on a remaining part of the at least one pair of weights from a previous iteration.

10. The computer (100) of claim 9, characterized by is further configured to: iteratively perform the steps of determining a convolutional layer having a smallest first weight trainable parameter; replacing the pair of weights of the convolutional layer having the smallest first weight trainable parameter with its factorized convolutional layer; perform a next training iteration until a determined number of convolutional layers is replaced with its respective factorized convolutional layer.

11. The computer (100) according to claim 10, characterized in that the computer (100) comprises an artificial intelligence accelerator suitable for tensor processing operations of CNNs.

12. A method (900) for implementing a tensor chain decomposition operation for a convolution layer of a convolutional neural network, CNN, for processing image data, text data or speech data, characterized in that, the method (900) comprises: receiving (901) input data (110) comprising a first number of channels, the input data (110) being image data, text data or speech data; performing (902) a 1x1 convolution on the input data (110) to obtain a plurality of data groups (120) comprising a second number of channels; performing (903) a grouped convolution based on shared kernels shared between the plurality of data groups (120) to obtain intermediate data (130) comprising a third number of channels; performing (904) a 1x1 convolution on the intermediate data (130) to obtain output data (140) comprising a fourth number of channels, the output data (140) being processed image data, text data or speech data.

13. A computer program product, characterised in that, a program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the steps of the method (900) according to claim 12.

Images