Matrix calculation method and device and related equipment

Abstract

The invention provides a matrix calculation method and device and related equipment, and relates to the field of digital circuits. The matrix calculation method is applied to a processing unit. The matrix calculation method comprises the following steps: acquiring a target bit width region of a target element in matrix calculation; the target element is an element which needs to be subjected to bit value adjustment in the matrix calculation; the target bit width region is a bit width region which can still enable the target element to meet the calculation precision requirement corresponding to the target element after the value on each bit is modified; adjusting the value of a target bit width area of the target element to be 0; and performing matrix calculation based on the adjusted target element. Through the method, the power consumption of matrix calculation can be reduced on the premise that the matrix calculation meets the precision requirement.

Description

Technical Field

[0001] The present application relates to the field of digital circuits, and in particular to a matrix calculation method, device and related equipment. Background Art

[0002] The processing unit can perform matrix calculations, which are implemented by multipliers and adders. When the data precision is high, the power consumption of the processing unit to perform matrix calculations is high, especially in a single data format scenario. Since the data format specifies the bit width of each element involved in the matrix calculation, it is impossible to reduce the calculation complexity by quantizing the elements and reducing the bit width of the elements, thereby reducing power consumption.

[0003] Currently, the power consumption of the processing unit during matrix calculation is usually reduced by improving the hardware performance of the processing unit by simplifying the circuit structure design of the processing unit, using devices with lower power consumption, etc., while enabling the processing unit to perform the required work.

[0004] However, based on certain process conditions, the design of the current processing unit has been optimized to a high level and is difficult to be significantly improved. Therefore, it is necessary to further reduce the power consumption of the processing unit for matrix calculation of elements in a single data format under the condition that the hardware resources of the processing unit are limited. Summary of the invention

[0005] In view of this, the present application aims to provide a matrix calculation method, apparatus and related equipment to reduce the power consumption of performing matrix calculations on elements of a single data format.

[0006] In a first aspect, an embodiment of the present application provides a matrix calculation method, which is applied to a processing unit, and the matrix calculation method includes: obtaining a target bit width area of ​​a target element in a matrix calculation; the target element is an element whose bit value needs to be adjusted in the matrix calculation; the target bit width area is a bit width area in which the value on each bit is modified so that the target element still meets the calculation accuracy requirement corresponding to the target element; adjusting the value of the target bit width area of ​​the target element to 0; and performing matrix calculation based on the adjusted target element.

[0007] Since the elements of the matrix calculation are in the same data format, it is no longer possible to use quantization to reduce the bit width to reduce the power consumption of the matrix calculation. In the matrix calculation, each element has certain calculation accuracy requirements. In the calculation accuracy requirements, the value of the element on some low-bit bits can be ignored when the element is calculated and used. Even if the value on the low-bit bits of this part is modified to 0, it does not affect the final calculation result and subsequent use. The bit width area composed of this part of the bits is also the target bit width area. Therefore, in the embodiment of the present application, the target bit width area can be determined and the value can be adjusted to 0, so that the calculation result value of the target bit width area of ​​the target element is 0 when performing matrix calculation, reducing the complexity of the calculation and the flip rate of the transistor, thereby reducing the power consumption of the matrix calculation under the premise that the calculation accuracy meets the requirements.

[0008] In one embodiment, the matrix calculation includes multiplication calculation and addition calculation, the element of the multiplication calculation is a multiplier; the addition calculation accumulates the multiplication result of the multiplication calculation and the original value of the accumulator through an accumulator; the target element includes at least one of the multiplier, the multiplication result, and the original value of the accumulator.

[0009] In the embodiments of the present application, matrix calculation includes two parts: multiplication and addition, and the calculation order is multiplication first and then addition. The elements involved in the multiplication are multipliers, and the elements involved in the addition operation are multiplication results. The values ​​of the target bit width area of ​​the elements involved in the two calculations are adjusted to 0, which can achieve the effect of reducing the calculation complexity and thus reducing power consumption.

[0010] In one embodiment, adjusting the value of the target bit width region of the target element to 0 includes: generating a mask according to the target bit width region; and adjusting the value of the target bit width region of the target element to 0 using the mask.

[0011] In an embodiment of the present application, the target bit width area is adjusted to 0 through a mask. The mask is generated according to the target bit width area and can be used to characterize the size of the target bit width area, which is equivalent to recording the target bit width area using a mask. Thus, the target bit width area can be accurately adjusted, reducing the possibility of affecting the final calculation result due to the adjustment of the target element, so that after reducing the calculation power consumption, the calculation result still has a certain accuracy.

[0012] In one embodiment, the target bit width area is an area composed of the 0th bit to the i-1th bit of the target element; i is an integer greater than or equal to 1, and the bit width of the target element is X; generating a mask based on the target bit width area includes: a first mask of a preset bit width X generated based on the target bit width area; the values ​​of the 0th bit to the i-1th bit of the first mask are 0, and the values ​​of the i-th bit to the X-1th bit are 1; accordingly, using the mask to adjust the value of the target bit width area of ​​the target element to 0 includes: performing an AND operation on the first mask and the target element to obtain the adjusted target element.

[0013] In the embodiment of the present application, the bit width of the first mask is X, the value of the 0th bit to the i-1th bit is 0, and the value of the ith bit to the X-1th bit is 1. The first mask with the same bit width as the target element is generated and used to perform an AND operation on the target element, and the target bit width area of ​​the target element can be adjusted to 0. The AND operation is simple to implement, which can reduce the difficulty of adjustment and the required power consumption, thereby reducing the possibility of excessive power consumption caused by adjusting the target bit width area.

[0014] In one embodiment, the target bit width area is an area consisting of the 0th bit to the i-1th bit of the target element; i is an integer greater than or equal to 1; generating a mask based on the target bit width area includes: a second mask of a preset bit width i generated based on the target bit width area; the values ​​of all bits of the second mask are 0; accordingly, using the mask to adjust the value of the target bit width area of ​​the target element to 0 includes: using the second mask to replace the target bit width area of ​​the target element to obtain the adjusted target element.

[0015] In the embodiment of the present application, the bit width of the second mask is i, and the values ​​of all bits are 0, then the second mask is consistent with the width of the target bit width area, and by replacing the target bit width area with the second mask, the target bit width area of ​​the target element can also be adjusted to 0. The bit width of the second mask required by this method is smaller, which is suitable for scenarios with low precision requirements and large target bit width areas.

[0016] In one embodiment, the target bit width region is a region formed by the 0th bit to the i-1th bit of the target element; i is an integer greater than or equal to 1, the bit width of the controllable bits in the target element is K, K is greater than or equal to i, and less than or equal to the total bit width of the target element; generating a mask according to the target bit width region includes: generating an initial mask; the bit width of the initial mask is less than the total bit width of the mask; obtaining the granularity of the initial mask relative to the target element; the granularity represents the granularity of the target element The ratio P of the bit width K of the controllable bits in the element to the bit width of the initial mask, where P is a positive integer; the initial mask satisfies: the width is K / P, and the values ​​of the 0th bit to the (i / P-1)th bit of the initial mask are 0, and the values ​​of the i / Pth bit to the (K / P-1)th bit are 1; or, the initial mask satisfies: the width is K / P and the values ​​of all bits of the initial mask are 0; the K / P and the i / P are both positive integers; the initial mask is expanded based on the granularity to obtain the mask.

[0017] In different hardware structures, there are restrictions on the bit width of the mask. Therefore, in an embodiment of the present application, an initial mask with a smaller bit width can be stored, and when the target bit width area needs to be adjusted, the initial mask can be expanded. In this way, the method of adjusting the target bit width area using the mask can be applicable to scenarios with more hardware structures.

[0018] In one embodiment, there are multiple target elements whose target bit width areas have their values ​​adjusted to 0, and at least some of the target elements have different calculation accuracy requirements; generating a mask based on the target bit width area includes: for each of the target elements, generating a corresponding mask according to the calculation accuracy requirements of each target element; using the mask to adjust the value of the target bit width area of ​​the target element to 0 includes: using the mask corresponding to each of the target elements to adjust the value of the target bit width area of ​​each target element to 0.

[0019] During matrix calculation, the elements involved in the operation may come from different scenarios, and different scenarios have different requirements for the calculation accuracy of the elements. Therefore, in an embodiment of the present application, masks of corresponding widths can be configured for different elements according to different accuracy requirements, and different width masks can be used to adjust the target bit width areas of the corresponding target elements, so that different target elements can meet the calculation accuracy requirements. As a result, it can be applicable to matrix calculations of target elements with different calculation accuracy requirements, thereby expanding the scope of application.

[0020] In one embodiment, the processing unit includes multiple other operation units in addition to the matrix operation, and the multiple target elements include data from different operation units; the calculation accuracy requirements corresponding to at least some of the operation units are different; for any one of the target elements, the calculation accuracy requirement used to generate the mask corresponding to the target element is the calculation accuracy requirement of the operation unit that outputs the target element.

[0021] The same processing unit may use different computing units to perform different operations, and different computing units may have different precision requirements. Therefore, in the embodiment of the present application, the corresponding computing precision requirements can be determined by determining the computing unit corresponding to each target element, so as to accurately adjust the target bit width area and perform matrix calculations.

[0022] In a second aspect, an embodiment of the present application provides a matrix calculation device, which is applied to a processing unit, and the matrix calculation device includes: an acquisition module, which acquires a target bit width area of ​​a target element in a matrix calculation; the target element is an element whose bit value needs to be adjusted in the matrix calculation; the target bit width area is a bit width area that can still enable the target element to meet the calculation accuracy requirement corresponding to the target element after the value on each bit is modified; an adjustment module, which adjusts the value of the target bit width area of ​​the target element to 0; and a matrix calculation module, which is used to perform matrix calculation based on the adjusted target element.

[0023] In the third aspect, an embodiment of the present application provides a processing unit, including: a masking circuit, used to determine a target bit width region of a target element in a matrix calculation, and adjust the value of the target bit width region of the target element to 0; the target element is an element whose bit value needs to be adjusted in the matrix calculation; the target bit width region is a bit width region that can still enable the target element to meet the calculation accuracy requirement corresponding to the target element after the value on each bit is modified; a matrix calculation circuit, connected to the masking circuit, for performing matrix calculation based on the adjusted target element.

[0024] In one embodiment, the processing unit is a graphics processing unit (GPU).

[0025] In a fourth aspect, an embodiment of the present application further provides an electronic device, which includes the processing unit described in the third aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0027] Figure 1 A flowchart of a method provided in one embodiment of the present application;

[0028] Figure 2 A schematic diagram of matrix calculation provided in one embodiment of the present application;

[0029] Figure 3 A schematic diagram of a target bit width region provided in an embodiment of the present application;

[0030] Figure 4 A schematic diagram of a matrix computing device provided in one embodiment of the present application;

[0031] Figure 5 A schematic diagram of a processing unit provided in accordance with an embodiment of the present application.

[0032] Icons: matrix calculation device 400; acquisition module 410; adjustment module 240; matrix calculation module 430; processing unit 500; masking circuit 510; matrix calculation circuit 520. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0034] First, the present application provides a matrix calculation method, which can be applied to a processing unit, and the processing unit can be a GPU (Graphics Processing Unit), a CPU (Central Processing Unit), etc.

[0035] In one embodiment of the present application, the matrix calculation method can be implemented as a software program configured in the processing unit, or a dedicated circuit for implementing the matrix calculation method can be configured in the processing unit to implement the matrix calculation method provided in the embodiment of the present application through the dedicated circuit.

[0036] See also Figure 1 , Figure 1 This is a flowchart of a matrix calculation method provided in one embodiment of the present application. The matrix calculation method provided in the embodiment of the present application includes:

[0037] S110, obtaining a target bit width region of a target element in matrix calculation.

[0038] See also Figure 2 , Figure 2A schematic diagram of matrix calculation provided for an embodiment of the present application. Currently, a typical matrix calculation is implemented by a series of multiplication and addition operations. Multiplication calculations can be performed by multipliers, and addition calculations can be implemented by adders (or "accumulators"). The power consumption of matrix calculations is mainly caused by the multipliers and adders performing calculations. In digital integrated circuits, data is saved in the form of 0 and 1, and recorded through different states of transistors. When switching between 0 and 1, the transistor will flip the state, thereby generating power consumption.

[0039] like Figure 2 As shown, assuming that the multipliers are a with a bit width of M and b with a bit width of N, the multiplication of the two can obtain a multiplication result with a bit width of M+N. Similarly, assuming that n multiplication results are obtained, the n multiplication results are offset and added according to the requirements of matrix calculation, and finally the original value of the accumulator with a bit width of P configured in the accumulator is added to obtain the result of the matrix calculation. The above is only an example, and you can refer to the relevant content of the existing matrix calculation, which will not be expanded here.

[0040] Therefore, the elements of the matrix calculation include the multiplier of the multiplication calculation, the multiplication result of the addition calculation, and the original value of the accumulator.

[0041] In an embodiment of the present application, the target element is an element whose bit value needs to be adjusted in matrix calculation.

[0042] How to reduce power consumption when performing matrix calculations for elements in the same data format. Therefore, since the data format is fixed, it is impossible to reduce power consumption by reducing the amount of matrix calculation by reducing the bit width by means of truncation, quantization, etc. Therefore, in an embodiment of the present application, the values ​​of the elements participating in the matrix calculation can be adjusted on the low bit positions and modified to 0, thereby simplifying the calculation on the low bit positions and then reducing the power consumption of the calculation. For example, 1111 and 111 are added, and after the values ​​of the 0th and 0th bits are modified to 0, 1100 and 100 are added. Compared with the addition of 1111 and 111, the addition of 1100 and 100 does not need to be added on the low bit positions, which can effectively simplify the complexity of the calculation. Regardless of multiplication, 0 multiplied by any number is 0, and for addition, 0 added to 0 is 0. Therefore, after the value of the low-bit element of the matrix calculation becomes 0, the value of the low-bit element is not easy to change when participating in the matrix calculation, and its value is usually 0. Therefore, when using a transistor to record the state of data, it does not need to be flipped, so that no power consumption is generated due to the flipping.

[0043] In the embodiment of the present application, the target bit width region is a bit width region in which the value on each bit is modified so that the target element still meets the calculation accuracy requirement corresponding to the target element.

[0044] In the embodiments of the present application, the calculation accuracy requirement can characterize the area of ​​ignorable values ​​in the data. For example, the bit width of a certain type of data is 8, and the matrix calculation requires a calculation accuracy of 6 bits for this type, and the first 6 bits need to be accurately calculated. Therefore, the lower 2 bits of the bit are the area where the values ​​can be ignored. The higher the calculation accuracy requirement, the smaller the corresponding target bit width area, and vice versa, if the calculation accuracy requirement is lower, the corresponding target bit width area is larger.

[0045] In addition, in the embodiments of the present application, the calculation accuracy requirements are pre-configured and can be directly obtained, for example, configured according to the requirements of the hardware device, or configured for the accuracy requirements of the data, etc., and the acquisition of the calculation accuracy requirements will not be expanded here.

[0046] As mentioned above, the low-bit width area of ​​the element can be adjusted to 0 to reduce the power consumption of the calculation. When calculating the matrix, the result of the matrix calculation should not deviate too much from the actual result, otherwise the subsequent work may not be performed correctly. Therefore, the result of the matrix calculation needs to meet the calculation accuracy requirements. The result of the matrix calculation is calculated by the elements participating in the matrix calculation, so the elements participating in the matrix calculation should also meet the calculation accuracy requirements.

[0047] Therefore, in the embodiment of the present application, it can be determined that the value of the target element and the target bit width region of the target element are adjusted to 0, thereby reducing computing power consumption.

[0048] like Figure 2 As shown, matrix calculation includes multiplication and addition, and the elements therefore include multipliers, multiplication results, and original accumulator values. Then, in one embodiment of the present application, on the premise that each element meets the calculation accuracy requirements, the target element may include at least one of the multipliers, multiplication results, and original accumulator values.

[0049] Selecting the multiplier as the target element and adjusting the target width area can reduce the power consumption of the multiplier during calculation. Similarly, selecting the multiplication result and the original value of the accumulator as the target element and adjusting the target width area can reduce the power consumption of the adder during calculation. All three can reduce the power consumption of matrix calculation, and all three can be selected as target elements.

[0050] Among them, in the addition calculation of matrix calculation, the multiplication result and the original value of the accumulator need to be right-shifted according to the exponential deviation from the maximum number. In the embodiment of the present application, if the multiplication result or the original value of the accumulator is selected as the target element, the multiplication result or the original value of the accumulator after right shift should be selected as the target element.

[0051] S120, adjusting the value of the target bit width area of ​​the target element to 0.

[0052] In multiplication calculation, the result of multiplication with 0 is 0. In addition calculation, the value of any number added to 0 is the original value of the number. Compared with multiplication or addition with the effective value 1, the complexity of multiplication or addition with 0 will be lower. For example, the multiplication of 110 and 101 is 11110, and the calculation process is equivalent to adding 110, 0000 and 11000. If the last bit is adjusted to 0, it is equivalent to multiplying 110 and 100, directly shifting 110 left by two places to get 11000, and its complexity is reduced.

[0053] The above is only an example. The effect of reducing the computational complexity is more obvious when calculating the matrix of elements with a larger bit width. At the same time, when the target bit width area adjusted to 0 is larger, the effect of reducing the computational complexity is also more obvious.

[0054] In an embodiment of the present application, the total bit width of the target element is X, and the target bit width region is the region formed by the 0th bit to the i-1th bit of the target element; i is an integer greater than or equal to 1. Among them, the bit width X of different target elements may be different, and the width i of the target bit width region of different target elements may also be different. This will be used as an example to illustrate the embodiments of the present application.

[0055] In an embodiment of the present application, adjusting the value of the target bit width region of the target element to 0 may include: generating a mask according to the target bit width region; and adjusting the value of the target bit width region of the target element to 0 using the mask.

[0056] In the embodiment of the present application, the mask is used to adjust the value of the target bit width region to zero, and the mask may include information that the target bit width region is adjusted to zero, for example, the mask may be used to characterize the width of the target bit width region, for example, the mask may include i bits 0. Through the mask, the width of the target bit width region can be accurately recorded, so that when making adjustments, the problem of inaccurate calculation results caused by the width of the adjusted target bit width region being too large is reduced, and the problem of overly complex calculations and poor power consumption reduction effects caused by the width of the adjusted target bit width region being too small can be reduced.

[0057] In one embodiment of the present application, generating a mask based on a target bit width region may include: a first mask of a preset bit width X generated based on the target bit width region, wherein the values ​​of the 0th bit to the i-1th bit of the first mask are 0, the bit width of the bit with a value of 0 is consistent with the target bit width region, which is i, and the values ​​of the i-th bit to the X-1th bit are 1.

[0058] In this embodiment, the bit widths of the first mask and the target element are consistent, both being X, and the bits that need to be adjusted in the target element can be represented by 0, and the bits that do not need to be adjusted can be represented by 1. The calculation accuracy requirement can be made clear through the first mask, which facilitates determining whether the matrix calculation is accurate.

[0059] Exemplarily, if the bit width of the target element is 16 bits, then X=16, and the generated first mask is also 16 bits. For example, if the width of the target bit width region is 8, the first mask in this example may be 11111111000000000. Similarly, if the bit width of the target element is 8 bits, then X=8, and the generated first mask is also 8 bits. If the width of the target bit width region is 2, the first mask in this example may be 11111100.

[0060] Correspondingly, in this embodiment, using the mask to adjust the value of the target bit width region of the target element to 0 includes: performing an AND operation on the first mask and the target element to obtain the adjusted target element.

[0061] In this embodiment, since the bit widths of the mask and the target element are consistent, the two do not need to be aligned by extending and padding, and can be directly ANDed, thereby not causing large additional power consumption.

[0062] In the principle of AND operation, if there is 0 in the two input numbers, the output result is 0, and in the AND operation, the value on the same bit is ANDed. Therefore, the area composed of the 0th bit to the i-1th bit of the first mask will be ANDed with the area composed of the 0th bit to the i-1th bit of the target element. Since the value of the area composed of the 0th bit to the i-1th bit of the first mask is 0, the operation result will make the value of the area composed of the 0th bit to the i-1th bit of the target element 0. The area composed of the 0th bit to the i-1th bit of the target element is the target bit width area. Therefore, the AND operation result is equivalent to the result after adjusting the value of the target bit width area of ​​the target element to 0.

[0063] For example, if the target element is 1011, the generated first mask is 1100, and the result of the AND operation of the two is 1000. For another example, if the target element is 11011101, and the first mask is 11110000, the result of the AND operation of the two is 11010000.

[0064] In another embodiment of the present application, a second mask of a preset bit width i may be generated based on the target bit width region, and the values ​​of all bits of the second mask are 0. Accordingly, using the mask to adjust the value of the target bit width region of the target element to 0 includes: using the second mask to replace the target bit width region of the target element to obtain the adjusted target element.

[0065] In this embodiment, a second mask consisting of 0s and having the same width as the target bit width region may be directly generated. For example, if the bit width of the target bit width region is 2, the second mask may be 00, and for another example, if the bit width of the target bit width region is 4, the second mask may be 0000.

[0066] In this embodiment, since the bit width of the second mask is inconsistent with the bit width of the target element and is all 0, the AND operation cannot be used. Therefore, the target bit width region of the target element can be directly replaced with the second mask, and the purpose of adjusting the value of the target bit width region to 0 can also be achieved. In this embodiment, since the second mask does not need to make the value of the high bit 1, it can be all 0, so the second mask method can be suitable for scenes with low precision requirements and large target bit width regions.

[0067] In a processing unit, there may be multiple data formats, for example, 32-bit wide data, 16-bit wide data, 8-bit wide data, etc., and during hardware design, some hardware may only support 8-bit wide data. For example, a register unit for storing a mask may only support storing an 8-bit wide mask, and the target bit width area of ​​some elements may be greater than 8, which makes it impossible for the mask to adjust the value of the target element to 0 in the maximum range.

[0068] Therefore, in an embodiment of the present application, an initial mask with a smaller bit width may be stored, so as to generate the mask for adjusting the target bit width region through the initial mask.

[0069] In this embodiment, generating a mask according to the target bit width region may include: generating an initial mask; obtaining a granularity of the initial mask relative to the target element; and expanding the initial mask based on the granularity to obtain a mask.

[0070] In this embodiment, the bit width of the initial mask is smaller than the total bit width of the mask. For example, if the total bit width of the mask is 8, the bit width of the initial mask may be 4 or 2. For another example, if the total bit width of the mask is 32, the bit width of the initial mask may be 16, 8, 4 or 2.

[0071] The maximum bit width of the initial mask is determined by the hardware structure. For example, if the maximum bit width of a single instruction required by the storage unit storing the mask is 8, then the maximum bit width of the initial mask is 8.

[0072] In some embodiments of the present application, only some bits in the target element may be allowed to be controlled, for example, the width of the target element is 8, and the last 6 bits of the target element may be controlled. Different from the target bit width area, the area composed of the controllable bits is greater than or equal to the target bit width area, and the target bit width area is within the area composed of the controllable bits. Therefore, in an embodiment of the present application, the bit width of the controllable bits in the target element is expressed as K, K is greater than or equal to i, and less than or equal to the total bit width of the target element. K can be equal to the target bit width area, or it can be equal to the total bit width of the target element, and there is no limitation here.

[0073] In this embodiment, the granularity represents the ratio of the bit width K of the controllable bits in the target element to the bit width of the initial mask, which is denoted as P, where P is a positive integer. For example, if the bit width K of the controllable bits of the target element is 16 and the bit width of the initial mask is 4, the P value of the granularity is equal to 4. If the bit width of the initial mask is 2, the P value of the granularity is equal to 8.

[0074] In an embodiment of the present application, the initial mask is expanded using the granularity. Since the granularity and the bit width of the initial mask multiplied together can be equal to the bit width of the mask, the same value (P-1) can be added after the value of each bit. For example, taking K as an example, if the granularity P=2 and the initial mask is 1110, 1 is added after the first 1 to obtain 11. Similarly, 1 is added after the second and third 1s, and 0 is added after the last 0. The expanded mask is 11 11 11 00. For another example, if the granularity is 4 and the initial mask is 1100, the expanded mask is 1111 1111 00000000. For another example, if the mask is 00 and the granularity is 4, the expanded mask is 0000 0000, and so on.

[0075] In this embodiment, K / P and i / P are both positive integers, and the initial mask satisfies one of the following two conditions:

[0076] The first condition is that the width is K / P, and the values ​​of the 0th bit to the (i / P-1)th bit of the initial mask are 0, and the values ​​of the i / Pth bit to the (K / P-1)th bit are 1.

[0077] This condition applies to the case where the preset bit width of the mask is K, that is, it applies to the case where the target bit width area is adjusted using the AND operation. In the embodiment of the AND operation, the initial mask needs to be expanded to be consistent with the bit width of the target element. The initial mask is expanded based on the granularity. If the expanded mask is consistent with the bit width of the target element, the width of the initial mask needs to be K / P. Accordingly, the values ​​of the 0th bit to the (i / P-1)th bit of the initial mask need to be 0, and the values ​​of the i / Pth bit to the (K / P-1)th bit need to be 1.

[0078] The above can be directly applied to the case where K is equal to the total bit width X of the target element. When K is less than the total bit width of the target element, after expansion in the above manner, 1 can be added before the highest bit until the mask is consistent with the width of the target element, and then the AND operation is performed. For example, the target element is 11011110, and the bit width of the controllable initial mask is 6. If the initial mask is 100, the initial mask can be expanded first to obtain 11 00 00, and then 1 can be added before the highest bit to be consistent with the bit width of the target element, so 1111 0000 can be obtained.

[0079] The second condition is that the initial mask satisfies: the width is K / P and the values ​​of all bits of the initial mask are 0.

[0080] This condition applies to the case where the preset bit width of the mask is i and K=i, that is, it applies to the case where the target bit width area is replaced by the second mask. In the alternative embodiment, the initial mask needs to be expanded to be consistent with the target bit width area of ​​the target element. The initial mask is expanded based on the granularity. If the expanded mask is consistent with the target bit width area, the width of the initial mask needs to be i / P. Accordingly, the value of all bits of the initial mask needs to be 0.

[0081] As mentioned above, the target element may be a multiplier, a multiplication result, and an original value of an accumulator, so there may be multiple target elements to be adjusted. Among the target elements to be adjusted, the requirements for accuracy may be different.

[0082] See also Figure 3 , Figure 3 A schematic diagram of a target bit width region provided for an embodiment of the present application, wherein the shaded area represents the target bit width region. Taking matrix calculation as an example, the weights of influence of the multiplier, the multiplication result, and the original value of the accumulator on the final matrix calculation result are inconsistent, which makes the calculation accuracy requirements of the three different. In matrix calculation, the multiplier has the greatest impact on the final result, followed by the multiplication result, and finally the original value of the accumulator. This requires a higher calculation accuracy of the multiplier. Correspondingly, the calculation accuracy requirement of the multiplier is higher, and its corresponding target bit width region is smaller. The multiplication result has a smaller impact on the final result, and its corresponding target bit width region is larger than the multiplier. For example, in Figure 3 In the embodiment, the bit widths of the multipliers a and b are smaller than the target bit width region of the multiplication result. In other embodiments, if the original value of the accumulator has a greater impact on the final result, the target bit width region configured for the accumulator should be smaller to better meet the calculation accuracy requirements.

[0083] For another example, the processing unit may be configured with a neural network, and the computational accuracy requirements for different data of the neural network may be different. For example, the computational accuracy requirements for activation functions and weight functions may be different, and the activation function and weight function may participate in the same matrix calculation.

[0084] Therefore, in an embodiment of the present application, if the values ​​of the target bit width regions of multiple target elements are adjusted to 0, and at least some of the target elements have different calculation accuracy requirements, for each target element, a corresponding mask can be generated according to the calculation accuracy requirements of each target element. Accordingly, using the mask to adjust the value of the target bit width region of the target element to 0 includes: using the mask corresponding to each target element to adjust the value of the target bit width region of each target element to 0.

[0085] For different target elements, masks determined by their respective calculation accuracy requirements are used respectively. When adjusting the target bit width area, the corresponding masks are used for adjustment. In this way, each target element can meet the calculation accuracy requirements during calculation, thereby improving the accuracy of the final calculation result.

[0086] The processing unit may perform multiple operations, and the output results of the operation units of different operations may all be used as elements of matrix calculation. Among different operation units, some operation units may have different calculation precisions. For example, one operation may require precision to the upper 12 bits of the data, while another operation may require precision to the upper 10 bits of the data.

[0087] Therefore, in the embodiment of the present application, if the processing unit includes multiple other operation units in addition to matrix operation, and the calculation accuracy requirements corresponding to at least some of the operation units are different, and these operation units all output data as target elements for matrix calculation, for any target element, the calculation accuracy requirement of the operation unit that outputs the target element is used as the calculation accuracy requirement for generating the mask corresponding to the target element. That is, for any target element, the calculation accuracy requirement used to generate the mask corresponding to the target element is the calculation accuracy requirement of the operation unit that outputs the target element.

[0088] After the matrix is ​​calculated, in the embodiment of the present application, a corresponding mask can be configured for each type of target element. Figure 3 For example, the multipliers include multiplier a and multiplier b. Mask a1 is configured for multiplier a and mask b1 is configured for multiplier b. Then, the same mask c1 is configured for each multiplication result, and mask c2 is configured for the original value of the accumulator.

[0089] Masks a1, b1, c1, and c2 may be stored in registers. When adjusting a target bit width region, the type of the target element may be determined, and the corresponding mask may be called from the register to adjust the target bit width region of the target element.

[0090] In addition, in an embodiment of the present application, a mask with a uniform bit width may be used for all target elements. The above is only an example and should not be a limitation to the present application.

[0091] S130, performing matrix calculation based on the adjusted target elements.

[0092] In this embodiment, corresponding matrix calculations are performed based on the adjusted target elements. The number of multiplications and the number of additions in different matrix calculations are different and will not be expanded here.

[0093] Since the target matrix area on the low bit position of the adjusted target element is 0, the complexity of both multiplication and addition calculations is reduced compared to the calculation with a value of 1. It can be understood that in the circuit, the state of the transistor is used to represent 0 and 1. Therefore, after the value of the target bit width area is 0, the transistor corresponding to the target bit width area always maintains the state corresponding to 0 during calculation, reducing the number of flips, so that the flip rate will be reduced. Therefore, the computing power consumption on the target matrix area can be reduced.

[0094] Based on the same inventive concept, an embodiment of the present application further provides a matrix computing device, which can be applied to a processing unit, and each module can be a software module.

[0095] See also Figure 4 , Figure 4 This is a schematic diagram of a matrix calculation device provided in an embodiment of the present application. The matrix calculation device 400 includes: an acquisition module 410, an adjustment module 420 and a matrix calculation module 430.

[0096] The acquisition module 410 is used to obtain the target bit width region of the target element in the matrix calculation; the target element is the element whose bit value needs to be adjusted in the matrix calculation; the target bit width region is the bit width region in which the value on each bit is modified so that the target element can still meet the calculation accuracy requirement corresponding to the target element.

[0097] The adjustment module 420 is configured to adjust the value of the target bit width region of the target element to 0.

[0098] The matrix calculation module 430 is used to perform matrix calculation based on the adjusted target elements.

[0099] Matrix calculation includes multiplication calculation and addition calculation, the element of the multiplication calculation is the multiplier; the addition calculation accumulates the multiplication result of the multiplication calculation and the original value of the accumulator through an accumulator. The target element acquired by the acquisition module 410 includes at least one of the multiplier, the multiplication result, and the original value of the accumulator.

[0100] The adjustment module 420 is specifically configured to generate a mask according to the target bit width region; and use the mask to adjust the value of the target bit width region of the target element to 0.

[0101] The target bit width region is a region formed by the 0th bit to the i-1th bit of the target element; i is an integer greater than or equal to 1, and the bit width of the target element is X.

[0102] The adjustment module 420 is specifically used to generate a first mask of a preset bit width X based on the target bit width area; the values ​​of the 0th bit to the i-1th bit of the first mask are 0, and the values ​​of the i-th bit to the X-1th bit are 1; the first mask is ANDed with the target element to obtain the adjusted target element.

[0103] The adjustment module 420 is specifically used to generate a second mask of a preset bit width i based on the target bit width area; the value of all bits of the second mask is 0; and the target bit width area of ​​the target element is replaced by the second mask to obtain the adjusted target element.

[0104] The adjustment module 420 is specifically used to generate an initial mask; the bit width of the initial mask is smaller than the total bit width of the mask; the granularity of the initial mask relative to the target element is obtained; the granularity represents the ratio of the bit width K of the controllable bits in the target element to the bit width of the initial mask, denoted as P, where P is a positive integer; the initial mask satisfies: the width is K / P, and the values ​​of the 0th bit to the (i / P-1)th bit of the initial mask are 0, and the values ​​of the i / Pth bit to the (K / P-1)th bit are 1; or, the initial mask satisfies: the width is K / P and the values ​​of all bits of the initial mask are 0; both K / P and i / P are positive integers; the initial mask is expanded based on the granularity to obtain the mask.

[0105] There are multiple target elements whose target bit width areas are adjusted to 0, and at least some of the target elements have different calculation accuracy requirements; the adjustment module is specifically used to generate corresponding masks for each of the target elements according to the calculation accuracy requirements of each target element; and the values ​​of the target bit width areas of each target element are adjusted to 0 using the masks corresponding to each of the target elements.

[0106] The processing unit includes multiple other operation units in addition to the matrix operation, and the multiple target elements include data from different operation units; at least some of the operation units have different calculation accuracy requirements; the adjustment module 420 is specifically used to generate, for any one of the target operations, the calculation accuracy requirement used for the mask corresponding to the target element is the calculation accuracy requirement of the operation unit that outputs the target element.

[0107] It can be understood that the function implemented by the matrix calculation device 400 is similar to the content of the aforementioned matrix calculation method. The specific content can refer to the content of the aforementioned matrix calculation method, which will not be repeated here.

[0108] As mentioned above, the matrix calculation method can be implemented using a dedicated circuit in a processing unit. Therefore, based on the same inventive concept, the embodiment of the present application also embodies a processing unit, which includes corresponding circuits to implement the aforementioned matrix calculation method.

[0109] See also Figure 5 , Figure 5 A processing unit is provided in accordance with an embodiment of the present application. In the embodiment of the present application, the processing unit 500 includes: a masking circuit 510 and a matrix calculation circuit 520 .

[0110] The masking circuit 510 is used to determine the target bit width region of the target element in the matrix calculation, and adjust the value of the target bit width region of the target element to 0; the target element is the element whose bit value needs to be adjusted in the matrix calculation; the target bit width region is the bit width region that can still make the target element meet the calculation accuracy requirement corresponding to the target element after the value on each bit is modified.

[0111] The matrix calculation circuit 520 is connected to the masking circuit 510 and is used to perform matrix calculation based on the adjusted target elements.

[0112] In an embodiment of the present application, the matrix calculation circuit 520 includes a multiplier and an accumulator, both of which are connected to a masking circuit, and the masking circuit 510 is used to store at least one of the multiplier, the multiplication result and the original value of the accumulator of the multiplier and adjust the target bit width area.

[0113] In an embodiment of the present application, the masking circuit 510 may include multiple register units, the register units include registers, wherein at least one register unit can be used to store a target element of an input matrix calculation circuit, and at least one register unit stores a mask with a preconfigured number. The masking circuit 510 can adjust the target bit width area of ​​the target element through the values ​​in different calculation units.

[0114] In an embodiment of the present application, the masking circuit 510 stores a first mask of a preset bit width X generated based on the target bit width area; the values ​​of the 0th bit to the i-1th bit of the first mask are 0, and the values ​​of the i-th bit to the X-1th bit are 1; the masking circuit may also include an AND operation circuit, and the masking circuit performs an AND operation on the first mask and the target element through the AND operation circuit to obtain an adjusted target element.

[0115] In an embodiment of the present application, the masking circuit 510 can also store a second mask of a preset bit width i generated based on the target bit width area; the values ​​of all bits of the second mask are 0; the masking circuit is used to replace the target bit width area of ​​the target element in the second mask register to obtain the adjusted target element.

[0116] In one embodiment, a pre-generated initial mask is stored in the masking circuit 510; the bit width of the initial mask is smaller than the total bit width of the mask; and the granularity of the initial mask relative to the target element is stored; the granularity represents the ratio P of the bit width K of the controllable bits in the target element to the bit width of the initial mask, and P is a positive integer; the initial mask satisfies: the width is K / P, and the values ​​of the 0th bit to the (i / P-1)th bit of the initial mask are 0, and the values ​​of the i / Pth bit to the (K / P-1)th bit are 1; or, the initial mask satisfies: the width is i / P and the values ​​of all bits of the initial mask are 0; both K / P and i / P are positive integers; the masking circuit 510 is also used to expand the initial mask based on the granularity to obtain the mask.

[0117] In one embodiment, there are multiple target elements whose target bit width regions are adjusted to 0, and at least some of the target elements have different calculation accuracy requirements; the masking circuit 510 stores masks corresponding to the calculation accuracy requirements of each target element; the masking circuit 510 is used to adjust the value of the target bit width region of each target element to 0 using the masks corresponding to each of the target elements.

[0118] In one embodiment, the processing unit may also include other operation unit circuits in addition to the matrix calculation circuit 520. For any target element stored in the masking circuit 510, the calculation accuracy requirement used to generate the mask corresponding to the target element is the calculation accuracy requirement of the operation unit that outputs the target element.

[0119] In the embodiment of the present application, existing devices can be used to implement the functions of the above-mentioned masking circuit 510, and the matrix calculation circuit can refer to the existing technology. The processing unit can also include other structures, which will not be expanded here.

[0120] However, GPU has increasingly higher requirements on the power consumption of matrix calculations. Therefore, in one embodiment of the present application, the processing unit may be a GPU, which effectively reduces the power consumption of the GPU.

[0121] Based on the same inventive concept, an embodiment of the present application further provides an electronic device, which may include the processing unit provided in the above embodiment.

[0122] In the embodiments of the present application, the electronic device may be a computer, a server, a mobile terminal, etc., which is not limited here.

[0123] The technical features of the above embodiments can be freely combined without conflict, and the embodiments obtained by the combination are included in the protection scope of this application.

[0124] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0125] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

Claims

1. A matrix calculation method, characterized in that: Applied to a processing unit, the matrix calculation method includes: Obtain a target bit width region of a target element in a matrix calculation; the target element is an element whose bit value needs to be adjusted in the matrix calculation; the target bit width region is a bit width region in which the value on each bit can still make the target element meet the calculation accuracy requirement corresponding to the target element after the value on each bit is modified; Adjust the value of the target bit width area of ​​the target element to 0; Perform matrix calculations based on the adjusted target elements.

2. The matrix calculation method according to claim 1, characterized in that: The matrix calculation includes multiplication calculation and addition calculation, the element of the multiplication calculation is a multiplier; the addition calculation accumulates the multiplication result of the multiplication calculation and the original value of the accumulator through an accumulator; The target element includes at least one of the multiplier, the multiplication result, and the original value of the accumulator.

3. The matrix calculation method according to claim 1 or 2, characterized in that: The step of adjusting the value of the target bit width area of ​​the target element to 0 includes: generating a mask according to the target bit width region; The value of the target bit width area of ​​the target element is adjusted to 0 using the mask.

4. The matrix calculation method according to claim 3, characterized in that: The target bit width region is a region consisting of the 0th bit to the i-1th bit of the target element; The i is an integer greater than or equal to 1, and the bit width of the target element is X; Generating a mask according to the target bit width region includes: A first mask of a preset bit width X generated based on the target bit width region; the values ​​of the 0th bit to the i-1th bit of the first mask are 0, and the values ​​of the ith bit to the X-1th bit are 1; Accordingly, using the mask to adjust the value of the target bit width area of ​​the target element to 0 includes: An AND operation is performed on the first mask and the target element to obtain the adjusted target element.

5. The matrix calculation method according to claim 3, characterized in that: The target bit width region is a region consisting of the 0th bit to the i-1th bit of the target element; Said i is an integer greater than or equal to 1; Generating a mask according to the target bit width region includes: A second mask of preset bit width i generated based on the target bit width region; values ​​of all bits of the second mask are 0; Accordingly, using the mask to adjust the value of the target bit width area of ​​the target element to 0 includes: The target bit width region of the target element is replaced using the second mask to obtain the adjusted target element.

6. The matrix calculation method according to claim 3, characterized in that: The target bit width region is a region formed by the 0th bit to the i-1th bit of the target element; i is an integer greater than or equal to 1, the bit width of the controllable bits in the target element is K, and K is greater than or equal to i and less than or equal to the total bit width X of the target element; Generating a mask according to the target bit width region includes: Generate an initial mask; the bit width of the initial mask is smaller than the total bit width of the mask; Obtain the granularity of the initial mask relative to the target element; the granularity represents the ratio P of the bit width K of the controllable bits in the target element to the bit width of the initial mask, where P is a positive integer; the initial mask satisfies: the width is K / P, and the values ​​of the 0th bit to the (i / P-1)th bit of the initial mask are 0, and the values ​​of the i / Pth bit to the (K / P-1)th bit are 1; or, the initial mask satisfies: the width is K / P and the values ​​of all bits of the initial mask are 0; both K / P and i / P are positive integers; The initial mask is expanded based on the granularity to obtain the mask.

7. The matrix calculation method according to claim 3, characterized in that: The value of the target bit width region where there are multiple target elements is adjusted to 0, and at least some of the target elements have different calculation accuracy requirements; Generating a mask according to the target bit width region includes: For each of the target elements, a corresponding mask is generated according to the calculation accuracy requirements of each target element; The using the mask to adjust the value of the target bit width area of ​​the target element to 0 includes: The value of the target bit width area of ​​each target element is adjusted to 0 using the mask corresponding to each target element.

8. The matrix calculation method according to claim 7, characterized in that: The processing unit includes a plurality of other operation units in addition to the matrix operation, and the plurality of target elements include data from different operation units; at least some of the operation units have different calculation accuracy requirements; For any of the target elements, the calculation accuracy requirement used to generate the mask corresponding to the target element is the calculation accuracy requirement of the operation unit that outputs the target element.

9. A matrix computing device, characterized in that: Applied to a processing unit, the matrix calculation device comprises: An acquisition module is used to acquire a target bit width region of a target element in a matrix calculation; the target element is an element whose bit value needs to be adjusted in the matrix calculation; the target bit width region is a bit width region in which the value on each bit is modified to still enable the target element to meet the calculation accuracy requirement corresponding to the target element; An adjustment module, used for adjusting the value of the target bit width area of ​​the target element to 0; A matrix calculation module is used to perform matrix calculation based on the adjusted target elements.

10. A processing unit, characterized in that: include: A masking circuit, used for determining a target bit width region of a target element in a matrix calculation, and adjusting a value of the target bit width region of the target element to 0; The target element is an element whose bit value needs to be adjusted in the matrix calculation; The target bit width region is a bit width region in which the target element can still meet the calculation accuracy requirement corresponding to the target element after the value on each bit is modified; A matrix calculation circuit is connected to the masking circuit and is used to perform matrix calculation based on the adjusted target element.

11. The processing unit according to claim 10, characterized in that The processing unit is a graphics processing unit GPU.

12. An electronic device, characterized in that: include: A processing unit as claimed in claim 10 or 11.

Images