Fault detection and mitigation in sparsity calculations in deep neural networks

Abstract

Computation in processing elements (PEs) for running a deep neural network (DNN) can be accelerated based on sparsity. Compressed activation operands and compressed weight operands can be stored. A compressed activation operand contains one or more non-zero activations within the activation operand. A compressed weight operand contains one or more non-zero weights within the weight operand. A sparsity module associated with a PE can generate a bitmap based on the activation sparsity vector of the activation operand and the weight sparsity vector of the weight operand. Based on the bitmap, the sparsity module identifies non-zero activations (or non-zero weights) from the compressed activation operand (or compressed weight operand). The sparsity module can detect failures when identifying non-zero activations or non-zero weights based on the number of one or more non-zero elements in the bitmap. The sparsity module can further mitigate failures.

Claims

1. A method for deep learning: A step of storing a compressed activation operand and a compressed weight operand, wherein the compressed activation operand includes one or more non-zeroed activations in the activation operand of a deep learning operation, and the compressed weight operand includes one or more non-zeroed weights in the weight operand of the deep learning operation; A step of generating a bitmap based on an activation sparsity vector and a weight sparsity vector, wherein the activation sparsity vector indicates one or more positions of the one or more non-zeroed activations in the activation operand, and the weight sparsity vector indicates one or more positions of the one or more non-zeroed weights in the weight operand; A step of identifying non-zero activations in the compressed activation operand, or non-zero weights in the compressed weight operand, based on the bitmap; and A step of determining whether there is an obstacle in identifying the non-zero activation or the non-zero weight, based on the number of one or more non-zero elements in the bitmap; A method that includes this.

2. The method according to claim 1, wherein the bitmap is generated based on a preceding bitmap, and another non-zeroed activation in the compressed activation operand, or another non-zeroed weight in the compressed weight operand, is identified based on the preceding bitmap, and the step of determining whether there is an obstacle in identifying the non-zeroed activation or the non-zeroed weight is: A step of determining the number of one or more non-zero elements in the preceding bitmap; and A step of determining whether the number of one or more non-zero elements in the bitmap is not equal to the sum of the number of one or more non-zero elements in the preceding bitmap plus one; Methods that include...

3. The method according to claim 2, the step of identifying non-zero-valued activations in the compressed activation operand is: The steps of generating a new bitmap based on the preceding bitmap after determining that there is a problem in identifying the non-zeroed activation or the non-zeroed weight; and A step of identifying non-zero activations in the compressed activation operand, or non-zero weights in the compressed weight operand, based on the new bitmap; Methods that include...

4. In the method according to claim 3, the step of generating a new bitmap based on the preceding bitmap is: A step of replacing non-zero elements in the preceding bitmap with zeros; Methods that include...

5. The method according to claim 1, the step of determining whether there is an obstacle in identifying the non-zeroed activation or the non-zeroed weight is: A step of determining whether the number of one or more non-zero elements in the bitmap is greater than the number of one or more non-zero-valued activations in the activation operand; Methods that include...

6. The method according to claim 1, the step of determining whether there is an obstacle in identifying the non-zeroed activation or the non-zeroed weight is: A step of determining whether the number of one or more non-zero elements in the bitmap is greater than the number of one or more non-zero weights in the weight operand; Methods that include...

7. The method according to claim 1, the step of identifying non-zero-valued activations in the compressed activation operand is: Steps include: determining that a failure exists when identifying the non-zeroed activations, and then generating a first bitmap and a second bitmap based on the activation sparsity vector and the weight sparsity vector; A step of determining the location of the non-zeroed activation in the compressed activation operand based on the bitmap; A step of determining a first position of a non-zeroed activation in the compressed activation operand based on the first bitmap; A step of determining a second position of the non-zero-valued activation in the compressed activation operand based on the second bitmap; and A step of identifying non-zero-valued activations in the compressed activation operand based on the aforementioned position, the first position, and the second position; Methods that include...

8. The method according to claim 1, the step of identifying the non-zero weights in the compressed weight operand is: The steps include: determining that there is a problem in identifying the non-zero weights, and then generating a first bitmap and a second bitmap based on the activation sparsity vector and the weight sparsity vector; A step of determining the position of the non-zero weights in the compressed weight operand based on the bitmap; A step of determining a first position of the non-zero weight in the compressed weight operand based on the first bitmap; A step of determining a second position of the non-zero weight in the compressed weight operand based on the second bitmap; and A step of determining the non-zero weights in the compressed weight operand based on the aforementioned position, the first position, and the second position; Methods that include...

9. The method according to claim 1, the step of identifying non-zero-valued activations in the compressed activation operand is: A method comprising the step of identifying another non-zero activation in the compressed activation operand after determining that there is a problem in identifying the non-zero activation, wherein the other non-zero activation follows the previously identified non-zero activation in the compressed activation operand.

10. The method according to any one of claims 1 to 9, wherein the step of identifying a non-zeroed activation in the compressed activation operand or a non-zeroed weight in the compressed weight operand is: A step of identifying the location of the non-zeroed activation in the compressed activation operand; and A step of identifying the location of the non-zero weights in the compressed weight operand; A method comprising the non-zero activation being multiplied by the non-zero weight in the deep learning operation.

11. A computer program that includes instructions causing a computer to perform operations for network computing, wherein the operations are: A step of storing a compressed activation operand and a compressed weight operand, wherein the compressed activation operand includes one or more non-zeroed activations in the activation operand of a deep learning operation, and the compressed weight operand includes one or more non-zeroed weights in the weight operand of the deep learning operation; A step of generating a bitmap based on an activation sparsity vector and a weight sparsity vector, wherein the activation sparsity vector indicates one or more positions of the one or more non-zeroed activations in the activation operand, and the weight sparsity vector indicates one or more positions of the one or more non-zeroed weights in the weight operand; A step of identifying non-zero activations in the compressed activation operand, or non-zero weights in the compressed weight operand, based on the bitmap; and A step of determining whether there is an obstacle in identifying the non-zero activation or the non-zero weight, based on the number of one or more non-zero elements in the bitmap; A computer program that includes [this].

12. The computer program according to claim 11, wherein the bitmap is generated based on a preceding bitmap, and another non-zero activation in the compressed activation operand, or another non-zero weight in the compressed weight operand, is identified based on the preceding bitmap, and the step of determining whether there is an obstacle in identifying the non-zero activation or the non-zero weight is: A step of determining the number of one or more non-zero elements in the preceding bitmap; and A step of determining whether the number of one or more non-zero elements in the bitmap is not equal to the sum of the number of one or more non-zero elements in the preceding bitmap plus one; A computer program that includes [this].

13. In the computer program according to claim 12, the step of identifying non-zero-valued activations in the compressed activation operand is: The steps of generating a new bitmap based on the preceding bitmap after determining that there is a problem in identifying the non-zeroed activation or the non-zeroed weight; and A step of identifying non-zero activations in the compressed activation operand, or non-zero weights in the compressed weight operand, based on the new bitmap; A computer program that includes [this].

14. In the computer program according to claim 13, the step of generating a new bitmap based on the preceding bitmap is: A step of replacing non-zero elements in the preceding bitmap with zeros; A computer program that includes [this].

15. The computer program according to claim 11, the step of determining whether there is a problem when identifying the non-zeroed activation or the non-zeroed weight is: A step of determining whether the number of one or more non-zero elements in the bitmap is greater than the number of one or more non-zero-valued activations in the activation operand; A computer program that includes [this].

16. The computer program according to claim 11, the step of determining whether there is a problem when identifying the non-zeroed activation or the non-zeroed weight is: A step of determining whether the number of one or more non-zero elements in the bitmap is greater than the number of one or more non-zero weights in the weight operand; A computer program that includes [this].

17. In the computer program according to claim 11, the step of identifying non-zero-valued activations in the compressed activation operand is: Steps include: determining that a failure exists when identifying the non-zeroed activations, and then generating a first bitmap and a second bitmap based on the activation sparsity vector and the weight sparsity vector; A step of determining the location of the non-zeroed activation in the compressed activation operand based on the bitmap; A step of determining a first position of a non-zeroed activation in the compressed activation operand based on the first bitmap; A step of determining a second position of the non-zero-valued activation in the compressed activation operand based on the second bitmap; and A step of identifying non-zero-valued activations in the compressed activation operand based on the aforementioned position, the first position, and the second position; A computer program that includes [this].

18. In the computer program according to claim 11, the step of identifying the non-zero weights in the compressed weight operand is: The steps include: determining that there is a problem in identifying the non-zero weights, and then generating a first bitmap and a second bitmap based on the activation sparsity vector and the weight sparsity vector; A step of determining the position of the non-zero weights in the compressed weight operand based on the bitmap; A step of determining a first position of the non-zero weight in the compressed weight operand based on the first bitmap; A step of determining a second position of the non-zero weight in the compressed weight operand based on the second bitmap; and A step of determining the non-zero weights in the compressed weight operand based on the aforementioned position, the first position, and the second position; A computer program that includes [this].

19. In the computer program according to claim 11, the step of identifying non-zero-valued activations in the compressed activation operand is: A computer program comprising the step of identifying another non-zero activation in the compressed activation operand after determining that a failure exists in identifying the non-zero activation, wherein the other non-zero activation follows the previously identified non-zero activation in the compressed activation operand.

20. The computer program according to any one of claims 11 to 19, the step of identifying a non-zero activation in the compressed activation operand, or a non-zero weight in the compressed weight operand, is: A step of identifying the location of the non-zeroed activation in the compressed activation operand; A step of identifying the location of the non-zero weights in the compressed weight operand; A computer program comprising the non-zero activations being multiplied by the non-zero weights in the deep learning operation.

21. A computer processor that executes computer program instructions; and Non-temporary computer-readable memory for storing the aforementioned computer program instructions; A device including, wherein the computer program instructions are: A step of storing a compressed activation operand and a compressed weight operand, wherein the compressed activation operand includes one or more non-zeroed activations in the activation operand of a deep learning operation, and the compressed weight operand includes one or more non-zeroed weights in the weight operand of the deep learning operation; A step of generating a bitmap based on an activation sparsity vector and a weight sparsity vector, wherein the activation sparsity vector indicates one or more positions of the one or more non-zeroed activations in the activation operand, and the weight sparsity vector indicates one or more positions of the one or more non-zeroed weights in the weight operand; A step of identifying non-zero activations in the compressed activation operand, or non-zero weights in the compressed weight operand, based on the bitmap; and A step of determining whether there is an obstacle in identifying the non-zero activation or the non-zero weight, based on the number of one or more non-zero elements in the bitmap; A device that causes the computer processor to perform an operation including the above.

22. In the apparatus according to claim 21, the bitmap is generated based on a preceding bitmap, and another non-zero activation in the compressed activation operand, or another non-zero weight in the compressed weight operand, is identified based on the preceding bitmap, and the step of determining whether there is an obstacle in identifying the non-zero activation or the non-zero weight is: A step of determining the number of one or more non-zero elements in the preceding bitmap; and A step of determining whether the number of one or more non-zero elements in the bitmap is not equal to the sum of the number of one or more non-zero elements in the preceding bitmap plus one; A device including a device.

23. In the apparatus according to claim 21, the step of determining whether there is a problem when identifying the non-zeroed activation or the non-zeroed weight is: A step of determining whether the number of one or more non-zero elements in the bitmap is greater than the number of one or more non-zero-valued activations in the activation operand; A device including a device.

24. In the apparatus according to claim 21, the step of identifying the non-zero-valued activation in the compressed activation operand is: Steps include: determining that a failure exists when identifying the non-zeroed activations, and then generating a first bitmap and a second bitmap based on the activation sparsity vector and the weight sparsity vector; A step of determining the location of the non-zeroed activation in the compressed activation operand based on the bitmap; A step of determining a first position of a non-zeroed activation in the compressed activation operand based on the first bitmap; A step of determining a second position of the non-zero-valued activation in the compressed activation operand based on the second bitmap; and A step of identifying non-zero-valued activations in the compressed activation operand based on the aforementioned position, the first position, and the second position; A device including a device.

25. In the apparatus according to any one of claims 21 to 24, the step of identifying a non-zero activation in the compressed activation operand, or a non-zero weight in the compressed weight operand, is: A step of identifying the location of the non-zeroed activation in the compressed activation operand; A step of identifying the location of the non-zero weights in the compressed weight operand; A device comprising the non-zero activation being multiplied by the non-zero weight in the deep learning operation.

Images