System and method for INT9 quantization

Abstract

To provide a method for converting data stored in a memory into a second format from a first format.SOLUTION: A method includes the steps of: extending the number of bits in data stored in a Double Data Rate (DDR) memory by one bit to form extended data; determining whether the data stored in the DDR memory is signed or unsigned data; responsive to determining that the data is signed, adding a sign value to the most significant bit of the extended data; copying the data to lower order bits of the extended data; responsive to determining that the data is unsigned, copying the data to lower order bits of the extended data and setting the most significant bit to an unsigned value; and storing the extended data in an On-Chip Memory (OCM) of a processing tile of a machine learning computer array.SELECTED DRAWING: Figure 3A

Claims

1. A method for converting data stored in a storage unit from a first format to a second format, wherein the method is: The steps include: expanding the data stored in the storage unit by one bit to form expanded data; A step of determining whether the data stored in the storage unit is signed or unsigned data, In response to determining that the data is signed, the step is to add a sign value to the extended 1-bit signed bit of the data, In response to determining that the data is unsigned, the step of setting the signed bit of the extended single bit of the data to an unsigned value, The steps include storing the expanded data in the on-chip memory (OCM) of the processing tile of the computing unit, and A method for providing this.

2. The method according to claim 1, wherein the data is an unsigned integer.

3. The method according to claim 1, wherein the data is a signed integer.

4. The method according to any one of claims 1 to 3, wherein the data is 8 bits and the extended data is 9 bits.

5. The method according to any one of claims 1 to 4, wherein the extended data is int9 data.

6. A step of tracking whether the data stored in the storage unit is signed or unsigned, A step of scheduling an appropriate instruction for the extended data based on whether the data is signed or unsigned. The method according to any one of claims 1 to 5, further comprising:

7. The method according to claim 6, further comprising the step of performing an arithmetic logic unit (ALU) operation on the extended data as an operand.

8. The method according to claim 7, further comprising the step of storing the result of the calculation in the OCM of the processing tile of the computing unit.

9. The method according to claim 8, further comprising the step of storing the results stored in the OCM in the storage unit.

10. The method according to claim 9, further comprising the steps of, before the step of storing the results in the storage unit, adjusting the value of the results to the maximum value if the value of the results exceeds the maximum value of the range for the data, and adjusting the value of the results to the minimum value of the range for the data if the value of the results is lower than the minimum value of the range for the data.

11. The method according to claim 9, further comprising the step of dropping the extended bit of the result before the step of storing the result from the OCM to the storage unit.

12. The method according to any one of claims 1 to 11, wherein the data stored in the storage unit is an integer representation of the data relating to floating-point data.

13. The method according to claim 12, wherein the floating-point data is scaled and quantized to form the data in the first format.

14. The method according to claim 13, wherein a first scaling value is used to convert the floating-point data to int8 format, and a second scaling value is used to convert the floating-point data to uint8 format.

15. A storage unit configured to store integer data in a first format, A processing unit having multiple processing tiles Each processing tile is equipped with, A system having on-chip memory (OCM) configured to accept and maintain extended data converted from the integer data in a first format from the storage unit for various calculations, wherein the extended data includes at least one or more additional bits compared to the integer data in a first format, and if the integer data in a first format is signed, the signed bits of the extended data are signed, and if the integer data in a first format is unsigned, the signed bits of the extended data are set to unsigned values.

16. The system according to claim 15, wherein the integer data in the first format is either int8 or uint8.

17. The system according to claim 15 or 16, wherein the extended data is int9.

18. The system according to any one of claims 15 to 17, wherein the system tracks whether the integer data in the first format stored in the storage unit is signed or unsigned, and an appropriate instruction is scheduled depending on whether the integer data in the first format is signed or unsigned.

19. The system according to any one of claims 15 to 18, wherein the extended data is an operand for calculation.

20. The system according to claim 19, wherein the result of the calculation is stored in the OCM.

21. The system according to claim 20, wherein the result of the calculation stored in the OCM is further stored in the storage unit.

22. The system according to claim 21, wherein if the value of the result exceeds the maximum value, the value of the result is adjusted to the maximum value of the range for the integer data in the first format, and if the value of the result is lower than the minimum value of the range for the data, the value of the result is adjusted to the minimum value of the range for the integer data in the first format before storing the result in the storage unit.

23. The system according to claim 21 or 22, wherein the signed bits of the result are dropped before the result is stored in the storage unit.

24. The system according to any one of claims 15 to 23, wherein the integer data in the first format is an integer representation of floating-point data, and the floating-point data is scaled and quantized to form the integer data in the first format.

25. The system according to claim 24, wherein a first scaling value is used to convert the floating-point data to int8 format, and a second scaling value is used to convert the floating-point data to uint8 format.

26. The process involves expanding the data stored in the storage unit by one bit to form the expanded data, The step of determining whether the aforementioned data is signed or unsigned data, If the data is signed, the step is to add the sign value to the extended 1-bit signed bit of the data, If the data is unsigned, the step of setting the signed bit of the extended single bit of the data to an unsigned value, The step of storing the expanded data A method that includes [a certain feature].

27. The method according to claim 26, wherein the data is an unsigned integer.

28. The method according to claim 26, wherein the data is a signed integer.

29. The method according to any one of claims 26 to 28, wherein the data is 8 bits and the extended data is 9 bits.

30. The method according to any one of claims 26 to 29, wherein the extended data is int9 data.

31. A step of tracking whether the data stored in the storage unit is signed or unsigned, A step of scheduling an appropriate instruction for the extended data based on whether the data is signed or unsigned. The method according to any one of claims 26 to 30, further comprising:

32. The method according to claim 31, further comprising the step of performing an arithmetic logic unit (ALU) operation on the extended data as an operand.

33. The method according to claim 32, further comprising the step of storing the result of the calculation in the on-chip memory (OCM) of the processing tile of the computing unit.

34. The method according to claim 33, further comprising the step of storing the results stored in the OCM in the storage unit.

35. The method according to claim 34, further comprising the steps of, before the step of storing the result in the storage unit, adjusting the value of the result to the maximum value if the value of the result exceeds the maximum value of the range for the data, and adjusting the value of the result to the minimum value of the range for the data if the value of the result is lower than the minimum value of the range for the data.

36. The method according to claim 34 or 35, further comprising the step of dropping the extended bit of the result before the step of storing the result from the OCM to the storage unit.

37. The method according to any one of claims 26 to 36, wherein the data stored in the storage unit is an integer representation of the data relating to floating-point data.

38. The method according to claim 37, wherein the floating-point data is scaled and quantized to form the data in a first format.

39. The method according to claim 38, wherein a first scaling value is used to convert the floating-point data to int8 format, and a second scaling value is used to convert the floating-point data to uint8 format.

Images