Systems, computer implementation methods, computer programs (efficient convolution in environments that force tiling)

Abstract

To provide efficient convolution in an environment that enforces tiles.SOLUTION: A computer-implemented method comprises: receiving an input tensor having a shape defined by [n1, ..., nk], where k is equal to the number of dimensions that characterize the input tensor; receiving tile tensor metadata comprising a tile tensor shape defined by [t1, ..., tk], and information indicative of an interleaving stride to be applied to each dimension of the tile tensor; constructing an output tensor comprising a plurality of the tile tensors, by applying a packing algorithm which maps each element of the input tensor to at least one slot location of one of the plurality of tile tensors, based on the tile tensor shape and the interleaving stride.SELECTED DRAWING: Figure 1C

Claims

1. At least one hardware processor, A non-temporary computer-readable storage medium for storing program instructions, wherein the program instructions are stored in the at least one hardware processor. A procedure for receiving an input tensor, wherein the shape of the input tensor is [n １ , ..., n ｋ The procedure is defined by ], where k is equal to the number of dimensions characterizing the input tensor, At least (i) [t １ , ..., t ｋ A procedure for receiving tile tensor metadata that includes information indicating the tile tensor shape defined by (ii) and the interleaved stride applied to each dimension of the tile tensor, A procedure for constructing an output tensor including a plurality of tile tensors by applying a packing algorithm that maps each element of the input tensor to at least one slot position of one of a plurality of tile tensors, at least in part, based on the tile tensor shape and the interleaved stride, Each of the tile tensors includes a subset of the input tensor's elements, spaced apart within the input tensor according to the interleaved stride, such that the interleaved stride results in a discontinuous mapping of the input tensor's elements. It is executable to make it perform the action. Non-temporary computer-readable storage media and A system that includes these features.

2. The system according to claim 1, wherein the program instruction is further executable to cause a procedure for storing the output tensor and the tile tensor metadata.

3. The system according to claim 1, wherein the program instruction is further executable to cause a procedure to unpack the input tensor from the stored output tensor based on the tile tensor metadata.

4. The system according to claim 1, wherein the tile tensor metadata further includes replication parameters, and the packing algorithm is configured to perform replication of each element of the input tensor based on the replication parameters, so that each of the elements of the input tensor is mapped to a plurality of slot positions along one dimension of the tile tensor.

5. The aforementioned program instruction is, A procedure for receiving a filter related to a convolution operation on the input tensor, A procedure for calculating the convolution by applying a multiplication operator that multiplies the filter element by element to each of the tile tensors of the output tensor, A procedure for applying a summation algorithm to the results of the aforementioned multiplications, As a result of the aforementioned convolution, a procedure is performed to output the result of the procedure applied to the summation algorithm. The system according to claim 1, further executable to perform the following.

6. The system according to claim 5, wherein the convolution is part of neural network inference.

7. The system according to any one of claims 1 to 6, wherein the tile tensor is a homomorphic encrypted ciphertext.

8. A method performed by a computer, The step of receiving an input tensor, wherein the shape of the input tensor is [n １ , ..., n ｋ Defined by ], where k is equal to the number of dimensions characterizing the input tensor, and consists of steps, At least (i) [t １ , ..., t ｋ (ii) receiving tile tensor metadata which includes information indicating the tile tensor shape defined by ] and the interleaved stride applied to each dimension of the tile tensor, A step of constructing an output tensor including the plurality of tile tensors by applying a packing algorithm that maps each element of the input tensor to at least one slot position of one of the plurality of tile tensors, at least in part, based on the tile tensor shape and the interleaved stride, Each of the tile tensors includes a subset of the input tensor's elements, spaced apart within the input tensor according to the interleaved stride, such that the interleaved stride results in a discontinuous mapping of the input tensor's elements in stages and A method performed by a computer, comprising the following:

9. The method performed by a computer according to claim 8, further comprising the step of storing the output tensor and the tile tensor metadata.

10. The method performed by a computer according to claim 8, further comprising the step of unpacking the input tensor from the stored output tensor based on the tile tensor metadata.

11. The method performed by a computer according to claim 8, wherein the tile tensor metadata further includes replication parameters, and the packing algorithm is configured to perform replication of each element of the input tensor based on the replication parameters, so that each of the elements of the input tensor is mapped to a plurality of slot positions along one dimension of the tile tensor.

12. The steps include receiving a filter related to the convolution operation on the input tensor, The step of calculating the convolution by applying a multiplication operator that multiplies the filter element by element to each of the tile tensors of the output tensor, The step of applying a summation algorithm to the results of the aforementioned multiplication, As a result of the aforementioned convolution, the process outputs the result of the step in which the summation algorithm is applied. The computer-based method according to claim 8, further comprising the following:

13. The method performed by a computer according to claim 12, wherein the convolution is part of neural network inference.

14. The method performed by a computer according to any one of claims 8 to 13, wherein the tile tensor is a homomorphic encrypted ciphertext.

15. A computer program in which program instructions are embodied, wherein the program instructions are implemented in at least one hardware processor. A procedure for receiving an input tensor, wherein the shape of the input tensor is defined by [n １ , …, n ｋ , and k is equal to the number of dimensions characterizing the input tensor, and a procedure, At least (i) [t １ , ..., t ｋ A procedure for receiving tile tensor metadata that includes information indicating the tile tensor shape defined by (ii) and the interleaved stride applied to each dimension of the tile tensor, A procedure for constructing an output tensor including a plurality of tile tensors by applying a packing algorithm that maps each element of the input tensor to at least one slot position of one of a plurality of tile tensors, at least in part, based on the tile tensor shape and the interleaved stride, Each of the tile tensors includes a subset of the input tensor's elements, spaced apart within the input tensor according to the interleaved stride, such that the interleaved stride results in a discontinuous mapping of the input tensor's elements. It is executable to make it perform the action. Computer program.

16. The computer program according to claim 15, wherein the program instruction is further executable to cause a procedure for storing the output tensor and the tile tensor metadata.

17. The computer program according to claim 15, wherein the program instruction is further executable to cause a procedure to unpack the input tensor from the stored output tensor based on the tile tensor metadata.

18. The computer program according to claim 15, wherein the tile tensor metadata further includes replication parameters, and the packing algorithm is configured to perform replication of each element of the input tensor based on the replication parameters, so that each of the elements of the input tensor is mapped to a plurality of slot positions along one dimension of the tile tensor.

19. The aforementioned program instruction is, A procedure for receiving a filter related to a convolution operation on the input tensor, A procedure for calculating the convolution by applying a multiplication operator that multiplies the filter element by element to each of the tile tensors of the output tensor, A procedure for applying a summation algorithm to the results of the aforementioned multiplications, As a result of the aforementioned convolution, a procedure is performed to output the result of the procedure applied to the summation algorithm. A computer program according to any one of claims 15 to 18, further executable to cause the execution of

20. The computer program according to claim 19, wherein the convolution is part of neural network inference.

Images