Quantitative calculation method, device, electronic device and storage medium

Abstract

The present disclosure provides a quantitative calculation method, device, electronic device and storage medium. The quantitative calculation method is used for a quantitative calculation device. The quantitative calculation device includes a conversion module, a calculation module and a quantization module. The method includes: the conversion module obtains a first input data with a sign, performs a sign conversion process on the first input data with a sign, and obtains a second input data without a sign; the calculation module obtains the second input data, performs an in-memory calculation on the second input data and the signed weight data stored in the calculation module, and obtains a first calculation result; the quantization module performs data quantization on the first calculation result based on the offset value corresponding to the sign conversion process, and obtains a quantized output result. Based on the quantitative calculation method disclosed in the present disclosure, data sparsity can be improved and quantization complexity can be reduced on a storage and computing integrated chip.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing, and in particular to a quantitative calculation method, device, electronic device and storage medium. Background Art

[0002] In the process of end-side reasoning, the data types are generally unified, that is, the data is either signed or unsigned. However, the quantization steps of signed data are usually simpler than those of unsigned data, and the sparsity of unsigned data is better than that of signed data. The main reason for this difference is that if signed quantization is used, the highest bit of a negative number, that is, the sign bit, is always 1, and its sparsity is very poor.

[0003] In the prior art, if a signed data type is selected, the quantization process is relatively simple, and the hardware design module is relatively simple, but its sparsity performance is not good, that is, power consumption control is not friendly, and the calculation delay is slightly longer than unsigned quantization. If an unsigned data type is selected, its sparsity performance is good, but additional hardware modules are required to support quantization calculations, and the circuit design complexity is high. Therefore, how to improve data sparsity and reduce quantization complexity on a storage and computing integrated chip has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present disclosure provide a quantitative calculation method, device, electronic device and storage medium, which can improve data sparsity and reduce quantization complexity on a storage and computing integrated chip.

[0005] According to a first aspect of the present disclosure, a quantization calculation method is provided for a quantization calculation device, the quantization calculation device comprising a conversion module, a calculation module and a quantization module, the method comprising: the conversion module obtains first input data with a sign, performs a sign conversion process on the first input data with a sign, and obtains second unsigned input data; the calculation module obtains the second input data, performs an in-memory calculation on the second input data and the signed weight data stored in the calculation module, and obtains a first calculation result; the quantization module performs data quantization on the first calculation result based on an offset value corresponding to the sign conversion process, and obtains a quantized output result.

[0006] In one embodiment, the conversion module is an adder, which obtains first input data with a sign, performs a sign conversion process on the first input data with a sign, and obtains second unsigned input data, including: the adder obtains the first input data with a sign; the adder adds the first input data and the offset value corresponding to the sign conversion process to obtain the second input data.

[0007] In one embodiment, the conversion module obtains first input data with a sign, performs sign conversion processing on the first input data with a sign, and obtains second unsigned input data, including: the conversion module obtains the first input data with a sign; the conversion module determines the sign bit and the data bit in the first input data with a sign; the conversion module performs sign conversion processing on the sign bit and the data bit according to a preset conversion rule to obtain the second unsigned input data.

[0008] In one embodiment, the conversion module determines the sign bit and the data bit in the first input data with a sign, including: the conversion module determines the first bit in the first input data with a sign as the sign bit, wherein when the value of the sign bit is zero, it represents a positive number, and when the value of the sign bit is one, it represents a negative number; the conversion module determines the other bits in the first input data with a sign except the first bit as the data bits.

[0009] In one embodiment, the conversion module performs sign conversion processing on the sign bit and the data bit according to a preset conversion rule to obtain the unsigned second input data, including: the conversion module determines the offset value corresponding to the sign conversion processing according to the preset conversion rule; the conversion module converts the sign bit and the data bit into the second input data according to the offset value.

[0010] In one embodiment, the conversion module determines the offset value corresponding to the symbol conversion processing according to a preset conversion rule, including: the conversion module determines a target mapping strategy for mapping the first input data with a sign to the second input data with no sign according to the preset conversion rule; the conversion module determines the offset value corresponding to the data mapping based on the target mapping strategy.

[0011] In one embodiment, the calculation module includes an in-memory calculation array module, an analog-to-digital conversion module and a shift-and-accumulate module. The calculation module obtains the second input data, performs in-memory calculation on the second input data and the weight data with signs stored in the calculation module, and obtains a first calculation result, including: the in-memory calculation array module obtains the second input data; the in-memory calculation array module performs corresponding multiplication on each bit of the second input data and each bit of the weight data with signs stored in the in-memory calculation array module to obtain multiple product sub-results; the analog-to-digital conversion module adds multiple product sub-results in the same column to obtain multiple accumulated sub-results; the shift-and-accumulate module performs shift-and-accumulate on multiple accumulated sub-results according to the data bit width of the second input data to obtain the first calculation result.

[0012] In one embodiment, the shift-and-accumulate module shifts and accumulates the multiple accumulated sub-results according to the data bit width of the second input data to obtain the first calculation result, including: the shift-and-accumulate module determines the data bit width of the second input data; the shift-and-accumulate module shifts and accumulates the multiple accumulated sub-results according to the data bit width to obtain the first calculation result, and in the process of the shift-and-accumulate, the sign bit of the negative number in the weight data is calculated by subtraction, and the second input data is calculated by addition.

[0013] In one embodiment, the calculation module obtains the second input data, performs in-memory calculations on the second input data and the signed weight data stored in the calculation module, and obtains a first calculation result, including: the calculation module obtains a first data matrix corresponding to the second input data and a second data matrix corresponding to the weight data; the calculation module determines a plurality of first quantization values ​​corresponding to the first data matrix and a plurality of second quantization values ​​corresponding to the second data matrix; the calculation module accumulates and sums the products of the first quantization value and the second quantization value to obtain the first calculation result.

[0014] In one embodiment, the quantization module performs data quantization on the first calculation result based on the offset value corresponding to the symbol conversion processing to obtain a quantized output result, including: the quantization module determines the target quantization formula adopted by the quantization module; the quantization module obtains the target quantization parameter corresponding to the target quantization formula; the quantization module performs data quantization on the first calculation result according to the target quantization parameter and the offset value to obtain a quantized output result.

[0015] In one embodiment, the target quantization parameter includes a first quantization parameter corresponding to the second input data, a second quantization parameter corresponding to the weight data, and an auxiliary quantization parameter, the first quantization parameter includes a first quantization value corresponding to the second input data, a first zero point value carried by the second input data quantization, and a first cumulative sum corresponding to the second input data, the second quantization parameter includes a second quantization value corresponding to the weight data, a second zero point value carried by the weight data quantization, and a second cumulative sum corresponding to the weight data, the auxiliary quantization parameter includes a third zero point value carried by the output quantization, a quantization value scale, and a vector length involved in the calculation, wherein the weight data is normally distributed and the second zero point value is zero.

[0016] In one embodiment, the quantization module performs data quantization on the first calculation result according to the target quantization parameter and the offset value to obtain a quantized output result, including: the quantization module performs data quantization on the first calculation result according to the first zero-point value, the second zero-point value, the third zero-point value, the quantization value scale, the vector length, the first quantization value, the second quantization value, the first cumulative sum, the second cumulative sum and the offset value to obtain a quantized output result, wherein the vector length, the first zero-point value and the second zero-point value are multiplied, and the second zero-point value is multiplied with the first cumulative sum.

[0017] In one embodiment, the quantization module performs data quantization on the first calculation result according to the target quantization parameter and the offset value to obtain a quantized output result, including: the quantization module multiplies the offset value and the second cumulative sum to obtain a first sub-result; the quantization module multiplies the first zero point value and the second cumulative sum to obtain a second sub-result; the quantization module subtracts the first sub-result and the second sub-result from the first calculation result to obtain a third sub-result; the quantization module multiplies the third sub-result by the quantization value scale to obtain a fourth sub-result; the quantization module adds the fourth sub-result and the third zero point value to obtain the quantized output result.

[0018] According to a second aspect of the present disclosure, a quantization calculation device is provided, which includes a conversion module, a calculation module and a quantization module, wherein the conversion module is used to obtain first input data with a sign, perform a sign conversion process on the first input data with a sign, and obtain unsigned second input data; the calculation module is used to obtain the second input data, perform in-memory calculation on the second input data and the signed weight data stored in the calculation module, and obtain a first calculation result; the quantization module is used to perform data quantization on the first calculation result based on the offset value corresponding to the sign conversion process, and obtain a quantized output result.

[0019] According to a third aspect of the present disclosure, an electronic device is provided, comprising a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein the program, when executed by the processor, realizes the quantitative calculation method as described in the first aspect.

[0020] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the quantitative calculation method as described in the first aspect.

[0021] The quantization calculation method, device, electronic device and storage medium provided by the present disclosure, the method obtains the first input data with a sign through the conversion module, performs a sign conversion process on the first input data with a sign, and obtains the second input data without a sign; the calculation module obtains the second input data, performs an in-memory calculation on the second input data and the weight data with a sign stored in the calculation module, and obtains the first calculation result; the quantization module performs data quantization on the first calculation result based on the offset value corresponding to the sign conversion process, and obtains the quantized output result. Based on this, the first input data with a sign obtained is first converted into the second input data without a sign to improve the data sparsity of the input data; as a calculation module of a storage and calculation integrated chip, it performs in-memory calculation on the second input data with a sign obtained and the weight data with a sign stored in itself, and the quantization module performs data quantization on the first calculation result based on the offset value corresponding to the sign conversion process, and obtains the quantized output result. Since the weight data with a sign is quantized and stored in the in-memory calculation array, the additional hardware module can be reduced when the calculation result is quantized after the calculation result is output later, and the designed hardware module also becomes relatively simple, so that the quantization process also becomes relatively simple, and the power consumption and delay can be reduced at the chip level. Based on this, the quantitative calculation method provided by the present invention only requires minor changes in the hardware, and the software level still performs quantization according to signed quantized data, so as to improve data sparsity and reduce quantization complexity on the storage and computing integrated chip, and at the chip level, it can reduce power consumption, reduce calculation delay, and reduce the area of ​​the computing module.

[0022] Other features and advantages of the present disclosure will be described in the following description, and partly become apparent from the description, or understood by practicing the present disclosure. The purpose and other advantages of the present disclosure can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are used to provide further understanding of the technical solution of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solution of the present disclosure and do not constitute a limitation on the technical solution of the present disclosure.

[0024] Figure 1 This is the architecture diagram of in-memory computing;

[0025] Figure 2 is a main flow chart of a quantitative calculation method provided by an embodiment of the present disclosure;

[0026] Figure 3 These are the five components of a target quantification formula provided by an embodiment of the present disclosure;

[0027] Figure 4 is a sub-flow chart of step S101 provided by an embodiment of the present disclosure;

[0028] Figure 5 is a sub-flow chart of step S101 provided in another embodiment of the present disclosure;

[0029] Figure 6 is a sub-flowchart of step S502 provided by an embodiment of the present disclosure;

[0030] Figure 7 is a sub-flowchart of step S503 provided by an embodiment of the present disclosure;

[0031] Figure 8 is a sub-flowchart of step S701 provided by an embodiment of the present disclosure;

[0032] Fig. 9 is a sub-flow chart of step S102 provided by an embodiment of the present disclosure;

[0033] Fig.10 is a sub-flowchart of step S904 provided by another embodiment of the present disclosure;

[0034] Fig.11 is a sub-flow chart of step S102 provided by an embodiment of the present disclosure;

[0035] Fig.12 is a sub-flow chart of step S103 provided by an embodiment of the present disclosure;

[0036] Fig.13 is a sub-flow chart of step S103 provided in another embodiment of the present disclosure;

[0037] Fig.14 It is a schematic diagram of the structure of the quantitative calculation device of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] The present disclosure is further described below in conjunction with the accompanying drawings and specific embodiments. The described embodiments should not be regarded as limiting the present disclosure, and all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0039] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0040] Unless otherwise defined, all technical and scientific terms used in this disclosure have the same meaning as those commonly understood by those skilled in the art to which this disclosure belongs. The terms used in this disclosure are only for the purpose of describing the embodiments of this disclosure and are not intended to limit this disclosure.

[0041] In related technologies, in-memory computing represents a further evolution of storage and computing integration, and its core lies in the complete integration of storage units and computing units. Unlike traditional computing models, in-memory computing does not rely on independent computing units. Instead, it embeds algorithms directly on memory particles and uses the memory cells inside the memory chip to perform calculations directly. Typical in-memory computing technologies include storage and computing integration based on static random access memory SRAM and memristor ReRAM.

[0042] like Figure 1 As shown, Figure 1 The architecture diagram of in-memory computing shows a storage unit that can store n*m ​​bits and its computing unit. It can be considered as an SRAM with a size of n*m bits. nm It is represented as a single bit of weight data, that is, either 0 or 1. IN[m] is represented as a single bit of feature map data. The architecture includes an in-memory computing CIM array module, an analog-to-digital conversion module, and a shift-and-accumulate module. The CIM array module is used to complete the multiplication of a 1-bit matrix, the analog-to-digital conversion module is used to complete the addition of a column of bits, and the shift-and-accumulate module is used to complete the shift-and-addition according to the data bit width to obtain the final result. Specifically, the corresponding data of the weight and feature map are first multiplied by the CIM array module, and then the accumulated value is obtained by the analog-to-digital conversion module, and then the final result is obtained by the shift-and-accumulate module. The calculation rules are as follows:

[0043] Acc=IN[0]*W 00 +IN[1]* W 01 +IN[2]* W 02 +…+IN[m]* W 0m

[0044] Therefore, the final Acc value can be obtained to be less than the sum of the weight bit columns of the column, that is:

[0045] Acc<=SUM(W 00 +W 01 +W 02 +…+W 0m )

[0046] Among them, W nmWhether the value is 0 indicates its sparsity. If the number of 0s is large, it is generally considered that the sparsity is good, otherwise, it is considered that the sparsity is poor. The better the sparsity, the shorter the computing delay for the chip and the lower the power consumption.

[0047] In the process of end-side reasoning, the data types are generally unified, that is, the data is either signed or unsigned. However, the quantization steps of signed data are usually simpler than those of unsigned data, and the sparsity of unsigned data is better than that of signed data. The main reason for this difference is that if signed quantization is used, the highest bit of a negative number, that is, the sign bit, is always 1, and its sparsity is very poor.

[0048] In the prior art, if a signed data type is selected, the quantization process is relatively simple, and the hardware design module is relatively simple, but its sparsity performance is not good, that is, power consumption control is not friendly, and the calculation delay is slightly longer than unsigned quantization. If an unsigned data type is selected, its sparsity performance is good, but additional hardware modules are required to support quantization calculations, and the circuit design complexity is high. Therefore, how to improve data sparsity and reduce quantization complexity on a storage and computing integrated chip has become a technical problem that needs to be solved urgently.

[0049] Based on this, the present disclosure provides a quantitative calculation method and related devices, first converting the acquired signed first input data into unsigned second input data to improve the data sparsity of the input data; as a computing module of a storage-computing integrated chip, it performs in-memory calculation on the acquired unsigned second input data and the signed weight data stored in itself, and the quantization module performs data quantization on the first calculation result based on the offset value corresponding to the sign conversion process to obtain a quantized output result. Since the signed weight data is quantized and stored in the in-memory calculation array, additional hardware modules can be reduced when the calculation results are quantized after the calculation results are subsequently output, and the designed hardware modules also become relatively simple, so that the quantization process also becomes relatively simple, and power consumption and delay can be reduced at the chip level. Based on this, the quantitative calculation method provided by the present disclosure only needs to implement less changes in hardware, and the software level is still quantized according to the signed quantized data, so as to improve data sparsity and reduce quantization complexity on the storage-computing integrated chip, and reduce power consumption, calculation delay, and area of ​​the calculation module at the chip level.

[0050] It should be noted that in-memory computing includes analog in-memory computing and digital in-memory computing. The quantitative computing method provided in the present invention is mainly used for analog in-memory computing. Analog in-memory computing involves analog-to-digital converters. The analog-to-digital converters are dependent on the sparsity of data. The worse the sparsity of the data, the longer the calculation cycle of the analog-to-digital converter, which affects the latency and computing power of the in-memory computing.

[0051] like Figure 2 As shown, the present disclosure provides a quantitative calculation method, which is applied to a quantitative calculation device, wherein the quantitative calculation device may include a conversion module, a calculation module and a quantization module. The quantitative calculation method includes:

[0052] Step S101: a conversion module obtains first input data with a sign, performs a sign conversion process on the first input data with a sign, and obtains second input data without a sign.

[0053] Step S102: the calculation module obtains the second input data, performs in-memory calculation on the second input data and the weight data with signs stored in the calculation module, and obtains a first calculation result.

[0054] Step S103: The quantization module performs data quantization on the first calculation result based on the offset value corresponding to the sign conversion process to obtain a quantized output result.

[0055] The following is an overview of steps S101 to S103.

[0056] In step S101, the input data with a sign acquired by the conversion module is quantized integer data, and the input data includes but is not limited to image data, text data, voice data, etc., which is applicable to all neural network models. With the reasoning of the neural network model, the data distribution of the input data will be that the data located on the negative half axis is significantly more than the data located on the positive half axis. The data width of the input data includes but is not limited to 4-bit data width, 8-bit data width and 16-bit data width. Taking the 8-bit data width as an example of the first input data with a sign, the range of signed data that can be represented by 8-bit binary is -128 to 127. By performing a sign conversion process on the first input data with a sign, the corresponding unsigned second input data represents a data range of 0 to 255. For example, -128 corresponds to 0, and 127 corresponds to 255. It can be seen that the offset value corresponding to its sign conversion process is 128.

[0057] It is understandable that the first input data with a sign is generally data with nonlinear distribution, and the data located on the negative semi-axis is significantly more than the data located on the positive semi-axis, that is, there are more data located between the data range of -128 and 0, that is, there are more negative numbers. Since the sign bit of negative numbers is 1, the sparsity of the first input data with a sign is poor. Based on this, in order to improve the sparsity of the input data, the present disclosure performs a sign conversion process on the first input data with a sign through a conversion module, and the corresponding unsigned second input data can be obtained. After the sign conversion process, the data in the first input data between the data range of -128 to 0 is converted into the unsigned second input data, and the data range represented is 0 to 128. The highest bit of the unsigned data between 0 and 128 (including 0 but not including 128) is 0, and the data between 0 and 128 is far more than the unsigned data between 128 and 255, that is, the unsigned data with the highest bit of 0 is far more than the unsigned data with the highest bit of 1. Therefore, compared with the signed first input data, the unsigned second input data has better sparsity, thereby significantly improving the sparsity of the input data.

[0058] It should be noted that, on the one hand, the conversion module can use a simple hardware module to implement the above-mentioned symbol conversion processing function, including but not limited to an adder, and its additional cost is relatively small. On the other hand, the conversion module can also be implemented in the form of a software module, for example, by using a programming language to implement the above-mentioned symbol conversion processing function without increasing the hardware cost.

[0059] In step S102, the computing module, as a storage and computing integrated chip, stores signed weight data. The weight data is used to represent the relative importance of a variable in the whole, which can be represented in the form of a data matrix. The signed weight data is stored in the computing module, and is quantized according to signed. For signed data types, the quantization process is relatively simple, and the hardware design module is also relatively simple.

[0060] It should be pointed out that the signed weight data is generally normally distributed data, and the data on the negative semi-axis is basically equal to the data on the positive semi-axis, that is, the data between the data range -128 to 0 is basically the same as the data between the data range 0 to 127, and the closer to 0, the more data there is. It can be seen that even if the signed weight data is converted to unsigned weight data, since the highest bit of the unsigned data between 0 and 128 (including 0 but not including 128) is 0, and the highest bit of the unsigned data between 128 and 255 is 1, and the unsigned data in the two data ranges is basically the same, its sparsity will not be significantly improved. Therefore, there is no need to perform sign conversion processing on the signed weight data, so that the signed weight data is still stored in the calculation module. And the quantization process of the signed weight data is relatively simple, and the module of hardware design is also relatively simple.

[0061] The calculation module performs in-memory calculation on the acquired unsigned second input data and the signed weight data to obtain a first calculation result. As a storage-computation integrated chip, the calculation module can obtain the first data matrix corresponding to the second input data and the second data matrix corresponding to the weight data, determine the multiple first quantization values ​​corresponding to the first data matrix and the multiple second quantization values ​​corresponding to the second data matrix, and accumulate and sum the products of the first quantization values ​​and the second quantization values ​​to obtain the first calculation result. For the calculation module, it can also include an in-memory calculation array module, an analog-to-digital conversion module, and a shift accumulation module. For example, in the in-memory calculation process, the in-memory calculation array module obtains the second input data, and multiplies each bit of the second input data and each bit of the signed weight data stored in the in-memory calculation array module to obtain multiple product sub-results; the analog-to-digital conversion module adds multiple product sub-results in the same column to obtain multiple accumulated sub-results; the shift accumulation module shifts and accumulates multiple accumulated sub-results according to the data bit width of the second input data to obtain the first calculation result. The in-memory computing array module can use charge-based memory (such as SRAM, DRAM, Flash) or new non-volatile memory (such as RRAM, MRAM, PCM). The analog-to-digital conversion module is dependent on the sparsity of the data. The worse the data sparsity, the longer the calculation cycle of the analog-to-digital converter, which affects the latency and computing power of the in-memory computing.

[0062] In step S103, the quantization module performs data quantization on the first calculation result based on the offset value corresponding to the sign conversion process to obtain a quantized output result. Since in step S101, the first input data with a sign is converted into the second input data without a sign through the sign conversion process, the sign conversion process causes the offset value corresponding to the data offset, so it needs to be subtracted during the calculation process. It should be noted that subtracting this part of the offset value can be achieved by a simple algorithm, so no additional hardware module is required.

[0063] In the quantization process, the target quantization formula can be used for quantization. For example, Figure 3 The target quantization formula shown in the figure includes multiple target quantization parameters. Specifically, q1 represents the quantization value of the input data, and its common data bit width can be 4 bits, 8 bits and 16 bits. q2 represents the quantization value of the weight data. q3 represents the quantization value of the calculated output. Z1, Z2, and Z3 represent the zero point value ZeroPoint carried by the input data, weight data, and output quantization, respectively. M is a quantization value scale. a1 represents the cumulative sum of the input data involved in the calculation, and a2 represents the cumulative sum of the weight data.

[0064] The target quantization formula consists of 5 parts. The first part of the quantization formula is the matrix multiplication of integers, which is completed by the calculation module; the second part of the quantization formula is to calculate N*Z1*Z2, where N is the length of the vector involved in the calculation; the third part of the quantization formula is to calculate Z1*a2; the fourth part of the quantization formula is to calculate Z2*a1; the fifth part of the quantization formula is to calculate the result of the previous part multiplied by M and then added to Z3.

[0065] It should be noted that for unsigned quantized data, q3 needs to be calculated, and the specific calculation steps are as follows: the first part of the quantization formula is completed by the calculation module, and the second to fifth parts require additional hardware modules to implement. Therefore, if unsigned quantized data is used for quantization, the hardware circuit design complexity is high, resulting in high quantization complexity.

[0066] If signed quantization data is used for quantization, such as quantizing the signed weight data in the present disclosure, because the signed weight data is normally distributed data, its ZeroPoint is 0, that is, Z2=0. Based on this, the calculation result of the second part N*Z1*Z2 in the quantization formula is 0, and the calculation result of the fourth part Z2*a1 in the quantization formula is 0. Therefore, the second and fourth parts of the quantization formula can be omitted and do not need to be calculated, and the corresponding two hardware modules do not need to be added additionally. In other words, only the first, third and fifth parts of the above quantization formula need to be calculated, that is, the output value q3 can be calculated, thereby reducing the complexity of hardware circuit design and further reducing the complexity of quantization.

[0067] Based on this, the quantitative calculation method provided by the present invention only requires minor changes in the hardware, and the software level still performs quantization according to signed quantized data, so as to improve data sparsity and reduce quantization complexity on the storage and computing integrated chip, and at the chip level, it can reduce power consumption, reduce calculation delay, and reduce the area of ​​the computing module.

[0068] It should be noted that the quantitative calculation method disclosed in the present invention can be used for the design of integrated computing and storage chips including but not limited to static random access memory SRAM, memristor ReRAM, etc.

[0069] In some embodiments, the conversion module may be an adder, such as Figure 4 As shown, step S101 may include the following steps:

[0070] Step S401: the adder obtains the first input data with a sign;

[0071] Step S402: the adder adds the first input data and the offset value corresponding to the sign conversion process to obtain the second input data.

[0072] It is understandable that the conversion module can use a simple hardware module to implement the above-mentioned symbol conversion processing function, which can be an adder or other simple devices that can realize the symbol conversion processing function. The additional cost is small, and the present disclosure does not specifically limit this. In this embodiment, the conversion module is taken as an adder as an example. The adder adds the acquired first input data with a sign and the offset value corresponding to the sign conversion processing, and the unsigned second input data can be converted. For example, the first input data with a sign takes an 8-bit data width as an example. The range of signed data that can be represented by 8-bit binary is -128 to 127, and the offset value corresponding to the sign conversion processing is 128. The first input data and the offset value 128 are added by the adder, for example, -128+128=0, -127+128=1, and so on. 127+128=255 can be converted to obtain the unsigned second input data with a data range of 0 to 255. After the adder sign conversion processing, the data in the first input data between the data range of -128 to 0 is converted into the unsigned second input data, and the data range represented is 0 to 128. The highest bit of the unsigned data between 0 and 128 (including 0 but not including 128) is 0, and the data between 0 and 128 is far more than the data between 128 and 255, that is, the unsigned data with the highest bit of 0 is far more than the unsigned data with the highest bit of 1. Therefore, compared with the signed first input data, the unsigned second input data has better sparsity, thereby significantly improving the sparsity of the input data.

[0073] In some embodiments, Figure 5 As shown, step S101 may also include the following steps:

[0074] Step S501: The conversion module obtains the first input data with a sign;

[0075] Step S502: The conversion module determines the sign bit and the data bit in the first input data with a sign;

[0076] Step S503: the conversion module performs sign conversion processing on the sign bit and the data bit according to a preset conversion rule to obtain unsigned second input data.

[0077] It is understandable that the conversion module can be implemented in the form of a software module, for example, the above-mentioned symbol conversion processing function can be implemented by a programming language without increasing the hardware cost. The conversion module obtains the first input data with a sign. Since the first input data with a sign has a sign bit and a data bit, it is necessary to determine the sign bit and the data bit in the first input data. For example, the first input data with a sign takes an 8-bit data bit width as an example. Generally speaking, its highest bit is the sign bit and the last 7 bits are the data bits. When the sign bit is "0", it means that the data is a positive number; and when the sign bit is "1", it means that the data is a negative number. The last 7 bits are data bits used to indicate the size of the value. Therefore, the range of signed data that can be represented by 8-bit binary is -128 to 127. After determining the sign bit and the data bit in the first input data, the conversion module can perform a sign conversion process on the sign bit and the data bit according to the preset conversion rule, thereby obtaining the unsigned second input data. Among them, at the software level, the preset conversion rule can be implemented according to the programming language pre-written by the programmer. Therefore, the conversion module only needs to directly execute the preset conversion rule to convert the first input data with a sign into the second input data without a sign. This method does not need to increase additional hardware costs, and the hardware cost can be further reduced.

[0078] In some embodiments, Figure 6 As shown, step S502 may include the following steps:

[0079] Step S601: The conversion module determines the first bit in the first input data with a sign as a sign bit, wherein when the value of the sign bit is zero, it indicates a positive number, and when the value of the sign bit is one, it indicates a negative number;

[0080] Step S602: The conversion module determines the other bits except the first bit in the first input data with a sign as data bits.

[0081] It can be understood that, since the first input data with a sign has a sign bit and a data bit, it is necessary to determine the sign bit and the data bit in the first input data. In this embodiment, the first bit in the first input data can be used as a sign bit, and the other bits in the first input data except the first bit can be used as data bits. For example, the first input data with a sign takes an 8-bit data bit width as an example, and its first bit (i.e., the highest bit) is used as a sign bit, and the last 7 bits are used as data bits. And it is agreed that when the sign bit is "0", it means that the data is a positive number; and when the sign bit is "1", it means that the data is a negative number. The last 7 bits are data bits used to indicate the size of the value. Therefore, the range of signed data that can be represented by 8-bit binary is -128 to 127. After determining the sign bit and data bit in the first input data, the conversion module can perform sign conversion processing on the sign bit and the data bit according to the preset conversion rule, thereby obtaining the unsigned second input data.

[0082] In some embodiments, Figure 7 As shown, step S503 may include the following steps:

[0083] Step S701: The conversion module determines an offset value corresponding to the symbol conversion process according to a preset conversion rule;

[0084] Step S702: The conversion module converts the sign bit and the data bit into second input data according to the offset value.

[0085] It can be understood that based on the preset conversion rules, the conversion module determines the offset value corresponding to the sign conversion processing. For example, taking the first input data with a sign with an 8-bit data width as an example, the corresponding preset conversion rule is to convert the first input data with a data range of -128 to 127 into the second input data with a data range of 0 to 255, that is, -128 is converted to 0, -127 is converted to 1, and so on, 127 is converted to 255, thereby determining that the offset value corresponding to its sign conversion processing is 128. Based on this, the conversion module converts the first input data with a sign bit and a data bit into the unsigned second input data according to the offset value 128. It should be noted that at the software level, the preset conversion rules can be implemented according to the programming language pre-written by the programmer, and the corresponding conversion rules are formulated according to the technical goals that the programmer wants to achieve.

[0086] In some embodiments, Figure 8 As shown, step S701 may include the following steps:

[0087] Step S801: The conversion module determines a target mapping strategy for mapping signed first input data to unsigned second input data according to a preset conversion rule;

[0088] Step S802: The conversion module determines an offset value corresponding to the data mapping based on the target mapping strategy.

[0089] It can be understood that, in the case where the preset conversion rule is to adopt the method of data mapping, the conversion module can determine the target mapping strategy of mapping the first input data with a sign to the second input data without a sign according to the preset conversion rule, and then determine the offset value corresponding to the data mapping based on the target mapping strategy. For example, for the first input data with a sign of 8-bit data width, its target mapping strategy is to map the first feature map with a data range of -128 to 127 one by one to the second input data with a data range of 0 to 255, that is, -128 is mapped to 0, -127 is mapped to 1, and so on, 127 is mapped to 255, thereby determining that the offset value corresponding to its data mapping is 128. Based on this, the conversion module can convert the first input data with a sign into the second input data without a sign by executing the preset conversion rule. This method does not require additional hardware costs, and the hardware cost can be further reduced.

[0090] In some embodiments, Fig. 9 As shown, the calculation module includes an in-memory calculation array module, an analog-to-digital conversion module, and a shift-accumulation module. Step S102 may include the following steps:

[0091] Step S901: the in-memory computing array module obtains second input data;

[0092] Step S902: the in-memory calculation array module performs corresponding multiplication on each bit of the second input data and each bit of the weight data with a sign stored in the in-memory calculation array module to obtain a plurality of product sub-results;

[0093] Step S903: the analog-to-digital conversion module adds multiple product sub-results in the same column to obtain multiple accumulated sub-results;

[0094] Step S904: the shift accumulation module performs shift accumulation on a plurality of accumulation sub-results according to the data bit width of the second input data to obtain a first calculation result.

[0095] It can be understood that the computing module, as a storage-computing integrated chip, may include an in-memory computing array module, an analog-to-digital conversion module, and a shift-and-accumulate module. During the in-memory computing process, the in-memory computing array module obtains the second input data, and then multiplies each bit of the second input data with each bit of the signed weight data stored in the in-memory computing array module to obtain a plurality of product sub-results. Then, the analog-to-digital conversion module adds the multiple product sub-results in the same column to obtain a plurality of accumulated sub-results. Finally, the shift-and-accumulate module shifts and accumulates the multiple accumulated sub-results according to the data bit width of the second input data to obtain the first calculation result.

[0096] In some embodiments, Fig.10As shown, step S904 may include the following steps:

[0097] Step S1001: The shift-and-accumulate module determines the data bit width of the second input data;

[0098] Step S1002: The shift accumulation module performs shift accumulation on multiple accumulation sub-results according to the data bit width to obtain a first calculation result. During the shift accumulation process, the sign bit of the negative number in the weight data is calculated by subtraction, and the second input data is calculated by addition.

[0099] It can be understood that since the sign bit of the positive number in the weight data is "0" and the sign bit of the negative number is "1", and the second input data after the sign conversion processing is unsigned data, therefore, in the process of shift accumulation, the sign bit of the negative number in the weight data is calculated by subtraction, and the second input data is calculated by addition.

[0100] In some embodiments, Fig.11 As shown, step S102 may include the following steps:

[0101] Step S1101: The calculation module obtains a first data matrix corresponding to the second input data and a second data matrix corresponding to the weight data;

[0102] Step S1102: The calculation module determines a plurality of first quantization values ​​corresponding to the first data matrix and a plurality of second quantization values ​​corresponding to the second data matrix;

[0103] Step S1103: the calculation module accumulates and sums the product of the first quantization value and the second quantization value to obtain a first calculation result.

[0104] It can be understood that the second input data and the weight data can both be represented in the form of a data matrix. For example, the second input data can be represented as a first data matrix with i rows and j columns, and the weight data can be represented as a second data matrix with j rows and k columns. The first data matrix and the second data matrix are both quantization value matrices, wherein the first data matrix is ​​a data matrix composed of first quantization values, and the first quantization value is the quantization value of the unsigned second input data obtained after conversion. The elements of the matrix can be represented as +128; the second data matrix is ​​a data matrix composed of second quantized values, the second quantized values ​​are quantized values ​​of weight data, and the elements of the matrix can be expressed as The calculation module multiplies each first quantization value and each second quantization value correspondingly, and then accumulates and sums them to obtain a first calculation result, which can be expressed as Σ(q1(i,j)+128)*q2(j,k).

[0105] In some embodiments, Fig.12 As shown, step S103 may include the following steps:

[0106] Step S1201: The quantization module determines a target quantization formula used by the quantization module;

[0107] Step S1202: the quantization module obtains a target quantization parameter corresponding to the target quantization formula;

[0108] Step S1203: the quantization module performs data quantization on the first calculation result according to the target quantization parameter and the offset value to obtain a quantized output result.

[0109] It is understandable that the purpose of quantization calculation is to reduce the memory usage of the model, improve the calculation speed, reduce the hardware power consumption, and reduce the deployment and inference costs. In the quantization process, first determine the target quantization formula used by the quantization module, and obtain the target quantization parameters corresponding to the target quantization formula. For example, you can use Figure 3 The target quantization formula is shown. The quantization module performs data quantization on the first calculation result according to the target quantization parameter and the offset value to obtain a quantized output result. It should be noted that in the quantization calculation process, since in step S101, the first input data with a sign is processed into the second input data without a sign through a sign conversion process, the sign conversion process causes an offset value corresponding to the data offset, so it needs to be subtracted in the quantization calculation process.

[0110] As Figure 3 Taking the target quantization formula shown as an example, its target quantization parameters may include the first quantization parameter corresponding to the second input data, the second quantization parameter corresponding to the weight data, and the auxiliary quantization parameter. The first quantization parameter includes the first quantization value corresponding to the second input data, the first zero point value carried by the second input data quantization, and the first cumulative sum corresponding to the second input data. The second quantization parameter includes the second quantization value corresponding to the weight data, the second zero point value carried by the weight data quantization, and the second cumulative sum corresponding to the weight data. The auxiliary quantization parameter includes the third zero point value carried by the output quantization, the quantization value scale, and the vector length involved in the calculation. Specifically, in the above quantization formula, q1 represents the first quantization value corresponding to the second input data, and its common data bit width can be 4 bits, 8 bits, and 16 bits; q2 represents the second quantization value corresponding to the weight data, q3 represents the quantization value of the calculated output, Z1, Z2, and Z3 represent the zero point value ZeroPoint carried by the second input data, the weight data, and the output quantization, respectively, M is a quantization value scale, a1 represents the cumulative sum of the second input data involved in the calculation, and a2 represents the cumulative sum of the weight data. It should be noted that, since the weight data is normally distributed, the second zero point value carried by its quantization is zero, that is, Z2=0.

[0111] During the quantization process, the quantization module performs data quantization on the first calculation result according to the first zero point value, the second zero point value, the third zero point value, the quantization value scale, the vector length, the first quantization value, the second quantization value, the first cumulative sum, the second cumulative sum and the offset value to obtain a quantized output result, wherein the vector length, the first zero point value and the second zero point value are multiplied, and the second zero point value is multiplied by the first cumulative sum.

[0112] Since the signed quantized data is used for quantization, the signed weight data is quantized. Since the signed weight data is normally distributed data, its ZeroPoint is 0, that is, Z2=0. Therefore, the vector length, the first zero point value and the second zero point value are multiplied, that is, N*Z1*Z2, and the result is zero; similarly, the second zero point value is multiplied with the first cumulative sum, that is, Z2*a1, and the result is zero. In other words, if Figure 3 The calculation result of the second part N*Z1*Z2 in the target quantization formula shown is 0, and the calculation result of the fourth part Z2*a1 in the quantization formula is 0. Therefore, the second and fourth parts in the quantization formula can be omitted and do not need to be calculated, and the corresponding two hardware modules do not need to be added additionally. In other words, only the first, third and fifth parts of the above quantization formula need to be calculated, and the output value q3 can be calculated, thereby reducing the complexity of hardware circuit design and further reducing the quantization complexity.

[0113] In some embodiments, Fig.13 As shown, step S103 may include the following steps:

[0114] Step S1301: The quantization module multiplies the offset value and the second accumulated sum to obtain a first sub-result;

[0115] Step S1302: The quantization module multiplies the first zero point value and the second accumulated sum to obtain a second sub-result;

[0116] Step S1303: The quantization module subtracts the first sub-result and the second sub-result from the first calculation result to obtain a third sub-result;

[0117] Step S1304: the quantization module multiplies the third sub-result by the quantization value scale to obtain a fourth sub-result;

[0118] Step S1305: The quantization module adds the fourth sub-result and the third zero point value to obtain a quantized output result.

[0119] It can be understood that, taking 8-bit signed quantized data as an example, the quantization module multiplies the offset value and the second accumulated sum to obtain a first sub-result, expressed as 𝟏𝟐𝟖∗Σ𝒒𝟐(𝒋,𝒌). The quantization module multiplies the first zero point value and the second accumulated sum to obtain a second sub-result, expressed as 𝒛𝟏*𝒂𝟐(𝒌). The quantization module subtracts the first sub-result and the second sub-result from the first calculation result to obtain the third sub-result, expressed as Σ(𝒒𝟏(𝒊,𝒋)+𝟏𝟐𝟖)∗𝒒𝟐(𝒋,𝒌)−𝟏𝟐𝟖∗Σ𝒒𝟐(𝒋,𝒌)–𝒛𝟏*𝒂𝟐(𝒌). The quantization module multiplies the third sub-result by the quantization value scale to obtain the fourth sub-result, expressed as M*(Σ(𝒒𝟏(𝒊,𝒋)+𝟏𝟐𝟖)∗𝒒𝟐(𝒋,𝒌)−𝟏𝟐𝟖∗Σ𝒒𝟐(𝒋,𝒌)–𝒛𝟏*𝒂𝟐(𝒌)); The quantization module adds the fourth sub-result and the third zero point value to obtain the quantized output result 𝒒𝟑=𝒛𝟑+M*(Σ(𝒒𝟏(𝒊,𝒋)+𝟏𝟐𝟖)∗𝒒𝟐(𝒋,𝒌)−𝟏𝟐𝟖∗Σ𝒒𝟐(𝒋,𝒌)–𝒛𝟏*𝒂𝟐(𝒌)).

[0120] In one embodiment, the specific quantization data calculation formula for 8-bit signed quantization is as follows:

[0121] 𝒒𝟑 = 𝒛𝟑+ M*(Σ(𝒒𝟏(𝒊,𝒋)+𝟏𝟐𝟖)*𝒒𝟐(𝒋,𝒌 )−𝟏𝟐𝟖∗Σ𝒒𝟐(𝒋,𝒌)–𝒛𝟏*𝒂𝟐(𝒌))

[0122] The formula can be broken down step by step to calculate as follows:

[0123] Step 1: r_𝒒𝟏(𝒊,𝒋)=𝒒𝟏(𝒊,𝒋)+𝟏𝟐𝟖;

[0124] Step 2: S = Σr_𝒒𝟏(𝒊,𝒋)* 𝒒𝟐(𝒋,𝒌);

[0125] Step 3: S = S-𝟏𝟐𝟖∗Σ𝒒𝟐(𝒋,𝒌)–𝒛𝟏*𝒂𝟐(𝒌), where 𝒂𝟐=Σ𝒒𝟐(𝒋,𝒌);

[0126] Step 4: 𝒒𝟑=S*M;

[0127] Step 5: 𝒒𝟑= 𝒒𝟑+ 𝒛𝟑.

[0128] It can be understood that the first step is to convert the first input data with a sign into the second input data without a sign, which can be achieved by a conversion module. For the conversion module, on the one hand, the conversion module can use a simple hardware module to implement the above-mentioned symbol conversion processing function, including but not limited to an adder, and its additional cost is small. On the other hand, the conversion module can also be implemented in the form of a software module, for example, by using a programming language to implement the above-mentioned symbol conversion processing function without increasing the hardware cost.

[0129] It can be understood that the second step is to implement the multiplication of the unsigned second input data matrix and the signed weight data matrix through the calculation module and then perform a cumulative summation operation.

[0130] It can be understood that after the first and second steps are calculated, in the above third step, it is necessary to subtract the offset value 𝟏𝟐𝟖∗Σ𝒒𝟐(𝒋,𝒌) corresponding to the sign conversion processing in the first step. Subtracting this part of the offset value can be achieved by a simple algorithm, so no additional hardware module is required.

[0131] Based on this, the quantitative calculation method provided by the present disclosure, by first converting the acquired signed first input data into unsigned second input data, to improve the data sparsity of the input data; as a computing module of the storage and computing integrated chip, it performs in-memory calculation on the acquired unsigned second input data and the signed weight data stored by itself, and the quantization module performs data quantization on the first calculation result based on the offset value corresponding to the sign conversion process to obtain a quantized output result. Since the signed weight data is quantized and stored in the in-memory calculation array, the additional hardware modules can be reduced when the calculation results are quantized after the subsequent output calculation results, and the designed hardware modules also become relatively simple, so that the quantization process also becomes relatively simple, and the power consumption and delay can be reduced at the chip level. Based on this, the quantitative calculation method provided by the present disclosure only needs to implement less changes in hardware, and the software level is still quantized according to the signed quantized data, so as to improve the data sparsity and reduce the quantization complexity on the storage and computing integrated chip, and reduce the power consumption, reduce the calculation delay, and reduce the area of ​​the calculation module at the chip level.

[0132] like Fig.14 As shown in FIG. 1 , it is a structural block diagram of a quantitative calculation device. The quantitative calculation device 100 may include a conversion module 110, a calculation module 120 and a quantization module 130. It should be noted that: Fig.14 The structure is only for illustration and may also include Fig.14 More or fewer components as shown, or with Fig.14 Different configurations are shown. Fig.14Each component shown in the figure may be implemented by hardware and / or software.

[0133] The conversion module 110 may be used to obtain first input data with a sign, perform a sign conversion process on the first input data with a sign, and obtain second input data without a sign.

[0134] It can be understood that the signed input data acquired by the conversion module 110 is quantized integer data, and the input data includes but is not limited to image data, text data, voice data, etc., which is applicable to all neural network models. With the reasoning of the neural network model, the data distribution of the input data will be that the data located on the negative half axis is significantly more than the data located on the positive half axis. The data width of the input data includes but is not limited to 4-bit data width, 8-bit data width and 16-bit data width. Taking the 8-bit data width as an example of the first signed input data, the range of signed data that can be represented by 8-bit binary is -128 to 127. By performing a sign conversion process on the first signed input data, the corresponding unsigned second input data represents a data range of 0 to 255. For example, -128 corresponds to 0, and 127 corresponds to 255. It can be seen that the offset value corresponding to its sign conversion process is 128.

[0135] It is understandable that the first input data with a sign is generally data with nonlinear distribution, and the data located on the negative semi-axis is significantly more than the data located on the positive semi-axis, that is, there are more data located between the data range of -128 and 0, that is, there are more negative numbers. Since the sign bit of negative numbers is 1, the sparsity of the first input data with a sign is poor. Based on this, in order to improve the sparsity of the input data, the present disclosure performs a sign conversion process on the first input data with a sign through the conversion module 110, and the corresponding unsigned second input data can be obtained. After the sign conversion process, the data in the first input data between the data range of -128 to 0 is converted into the unsigned second input data, and the data range represented is 0 to 128. The highest bit of the unsigned data between 0 and 128 (including 0 but not including 128) is 0, and the data between 0 and 128 is far more than the unsigned data between 128 and 255, that is, the unsigned data with the highest bit of 0 is far more than the unsigned data with the highest bit of 1. Therefore, compared with the signed first input data, the unsigned second input data has better sparsity, thereby significantly improving the sparsity of the input data.

[0136] It should be noted that, on the one hand, the conversion module 110 can use a simple hardware module to implement the above-mentioned symbol conversion processing function, including but not limited to an adder, and its additional cost is relatively small. On the other hand, the conversion module 110 can also be implemented in the form of a software module, for example, by using a programming language to implement the above-mentioned symbol conversion processing function without increasing the hardware cost.

[0137] In one embodiment, the conversion module 110 can use a simple hardware module to implement the above-mentioned symbol conversion processing function, which can be an adder or other simple devices that can realize the symbol conversion processing function. The additional cost is small, and the present disclosure does not specifically limit this. In this embodiment, the conversion module 110 is taken as an adder as an example. The adder adds the acquired first input data with a sign and the offset value corresponding to the symbol conversion processing, and then converts it to obtain the unsigned second input data. For example, the first input data with a sign takes an 8-bit data width as an example. The range of signed data that can be represented by 8-bit binary is -128 to 127, and the offset value corresponding to the sign conversion processing is 128. The first input data and the offset value 128 are added by the adder, for example, -128+128=0, -127+128=1, and so on. 127+128=255 can be converted to obtain the unsigned second input data with a data range of 0 to 255. After the adder sign conversion processing, the data in the first input data between the data range of -128 to 0 is converted into the unsigned second input data, and the data range represented is 0 to 128. The highest bit of the unsigned data between 0 and 128 (including 0 but not including 128) is 0, and the data between 0 and 128 is far more than the data between 128 and 255, that is, the unsigned data with the highest bit of 0 is far more than the unsigned data with the highest bit of 1. Therefore, compared with the signed first input data, the unsigned second input data has better sparsity, thereby significantly improving the sparsity of the input data.

[0138] In one embodiment, the conversion module 110 can be implemented in the form of a software module, for example, the above-mentioned symbol conversion processing function can be implemented by a programming language without increasing the hardware cost. The conversion module 110 obtains the first input data with a sign. Since the first input data with a sign has a sign bit and a data bit, it is necessary to determine the sign bit and the data bit in the first input data. For example, the first input data with a sign takes an 8-bit data bit width as an example. Generally speaking, its highest bit is the sign bit and the last 7 bits are the data bits. When the sign bit is "0", it means that the data is a positive number; and when the sign bit is "1", it means that the data is a negative number. The last 7 bits are data bits used to indicate the size of the value. Therefore, the range of signed data that can be represented by 8-bit binary is -128 to 127. After determining the sign bit and the data bit in the first input data, the conversion module 110 can perform a sign conversion process on the sign bit and the data bit according to a preset conversion rule, thereby obtaining an unsigned second input data. Among them, at the software level, the preset conversion rule can be implemented according to a programming language pre-written by a programmer. Therefore, the conversion module 110 only needs to directly execute the preset conversion rule to convert the first input data with a sign into the second input data without a sign. This method does not require additional hardware cost, and the hardware cost can be further reduced.

[0139] The calculation module 120 may be used to obtain second input data, perform in-memory calculation on the second input data and weight data with a sign stored in the calculation module, and obtain a first calculation result.

[0140] It is understandable that the computing module 120, as a storage and computing integrated chip, stores signed weight data. The weight data is used to represent the relative importance of a variable in the whole, which can be represented in the form of a data matrix. The signed weight data is stored in the computing module 120, and it is quantized according to signed. For signed data types, the quantization process is relatively simple, and the hardware design module is also relatively simple.

[0141] It should be pointed out that the weight data with a sign is generally normally distributed data, and the data located on the negative semi-axis is basically equal to the data located on the positive semi-axis, that is, the data located between the data range of -128 to 0 is basically the same as the data located between the data range of 0 to 127, and the data closer to 0 is the most. It can be seen that even if the weight data with a sign is converted to unsigned weight data, since the highest bit of the data between the unsigned data 0 to 128 (including 0 but not including 128) is 0, and the highest bit of the data between the unsigned data 128 to 255 is 1, and the unsigned data in the two data ranges is basically the same, its sparsity will not be significantly improved. Therefore, there is no need to perform a sign conversion process on the weight data with a sign, so that the weight data with a sign is still stored in the calculation module 120. And the weight data with a sign is quantized, and its quantization process is relatively simple, and the module of the hardware design is also relatively simple.

[0142] The calculation module 120 performs in-memory calculation on the acquired unsigned second input data and the signed weight data to obtain a first calculation result. As a storage-computation integrated chip, the calculation module 120 can obtain a first data matrix corresponding to the second input data and a second data matrix corresponding to the weight data, determine a plurality of first quantization values ​​corresponding to the first data matrix and a plurality of second quantization values ​​corresponding to the second data matrix, and accumulate and sum the products of the first quantization values ​​and the second quantization values ​​to obtain a first calculation result. For the calculation module 120, it can also include an in-memory calculation array module, an analog-to-digital conversion module, and a shift accumulation module. For example, in the in-memory calculation process, the in-memory calculation array module obtains the second input data, and multiplies each bit of the second input data and each bit of the signed weight data stored in the in-memory calculation array module to obtain a plurality of product sub-results; the analog-to-digital conversion module adds a plurality of product sub-results in the same column to obtain a plurality of accumulated sub-results; the shift accumulation module performs shift accumulation on a plurality of accumulated sub-results according to the data bit width of the second input data to obtain a first calculation result.

[0143] The quantization module 130 may be configured to perform data quantization on the first calculation result based on the offset value corresponding to the sign conversion process to obtain a quantized output result.

[0144] It is understandable that, since in step S101, the first input data with a sign is converted into the second input data without a sign through the sign conversion process, the sign conversion process causes an offset value corresponding to the data offset, and therefore, it is necessary for the quantization module 130 to subtract it during the calculation process. It should be noted that subtracting this part of the offset value can be implemented by a simple algorithm, and therefore, no additional hardware module is required.

[0145] During the quantization process, the quantization module 130 may use a target quantization formula for quantization. For example, Figure 3 The target quantization formula shown in the figure includes multiple target quantization parameters. Specifically, q1 represents the quantization value of the input data, and its common data bit width can be 4 bits, 8 bits and 16 bits. q2 represents the quantization value of the weight data. q3 represents the quantization value of the calculated output. Z1, Z2, and Z3 represent the zero point value ZeroPoint carried by the input data, weight data, and output quantization, respectively. M is a quantization value scale. a1 represents the cumulative sum of the input data involved in the calculation, and a2 represents the cumulative sum of the weight data.

[0146] The target quantization formula consists of 5 parts. The first part of the quantization formula is the matrix multiplication of integers, which is completed by the calculation module; the second part of the quantization formula is to calculate N*Z1*Z2, where N is the length of the vector involved in the calculation; the third part of the quantization formula is to calculate Z1*a2; the fourth part of the quantization formula is to calculate Z2*a1; the fifth part of the quantization formula is to calculate the result of the previous part multiplied by M and then added to Z3.

[0147] It should be noted that for unsigned quantized data, q3 needs to be calculated, and the specific calculation steps are as follows: the first part of the quantization formula is completed by the calculation module, and the second to fifth parts require additional hardware modules to implement. Therefore, if unsigned quantized data is used for quantization, the hardware circuit design complexity is high, resulting in high quantization complexity.

[0148] If signed quantization data is used for quantization, such as quantizing the signed weight data in the present disclosure, because the signed weight data is normally distributed data, its ZeroPoint is 0, that is, Z2=0. Based on this, the calculation result of the second part N*Z1*Z2 in the quantization formula is 0, and the calculation result of the fourth part Z2*a1 in the quantization formula is 0. Therefore, the second and fourth parts of the quantization formula can be omitted and do not need to be calculated, and the corresponding two hardware modules do not need to be added additionally. In other words, only the first, third and fifth parts of the above quantization formula need to be calculated, that is, the output value q3 can be calculated, thereby reducing the complexity of hardware circuit design and further reducing the complexity of quantization.

[0149] It is understandable that the purpose of quantization calculation is to reduce the memory usage of the model, improve the calculation speed, reduce the hardware power consumption, and reduce the deployment and inference costs. In the quantization process, the target quantization formula used by the quantization module 130 is first determined, and the target quantization parameters corresponding to the target quantization formula are obtained. For example, the following can be used: Figure 3The target quantization formula is shown. The quantization module 130 performs data quantization on the first calculation result according to the target quantization parameter and the offset value to obtain a quantized output result. It should be noted that in the quantization calculation process, since in step S101, the first input data with a sign is processed into the second input data without a sign through a sign conversion process, the sign conversion process causes an offset value corresponding to the data offset, so it needs to be subtracted in the quantization calculation process.

[0150] As Figure 3 Taking the target quantization formula shown as an example, its target quantization parameters may include the first quantization parameter corresponding to the second input data, the second quantization parameter corresponding to the weight data, and the auxiliary quantization parameter. The first quantization parameter includes the first quantization value corresponding to the second input data, the first zero point value carried by the second input data quantization, and the first cumulative sum corresponding to the second input data. The second quantization parameter includes the second quantization value corresponding to the weight data, the second zero point value carried by the weight data quantization, and the second cumulative sum corresponding to the weight data. The auxiliary quantization parameter includes the third zero point value carried by the output quantization, the quantization value scale, and the vector length involved in the calculation. Specifically, in the above quantization formula, q1 represents the first quantization value corresponding to the second input data, and its common data bit width can be 4 bits, 8 bits, and 16 bits; q2 represents the second quantization value corresponding to the weight data, q3 represents the quantization value of the calculated output, Z1, Z2, and Z3 represent the zero point value ZeroPoint carried by the second input data, the weight data, and the output quantization, respectively, M is a quantization value scale, a1 represents the cumulative sum of the second input data involved in the calculation, and a2 represents the cumulative sum of the weight data. It should be noted that, since the weight data is normally distributed, the second zero point value carried by its quantization is zero, that is, Z2=0.

[0151] During the quantization process, the quantization module 130 performs data quantization on the first calculation result according to the first zero point value, the second zero point value, the third zero point value, the quantization value scale, the vector length, the first quantization value, the second quantization value, the first cumulative sum, the second cumulative sum and the offset value to obtain a quantized output result, wherein the vector length, the first zero point value and the second zero point value are multiplied, and the second zero point value is multiplied by the first cumulative sum.

[0152] Since the signed quantized data is used for quantization, the signed weight data is quantized. Since the signed weight data is normally distributed data, its ZeroPoint is 0, that is, Z2=0. Therefore, the vector length, the first zero point value and the second zero point value are multiplied, that is, N*Z1*Z2, and the result is zero; similarly, the second zero point value is multiplied with the first cumulative sum, that is, Z2*a1, and the result is zero. In other words, if Figure 3The calculation result of the second part N*Z1*Z2 in the target quantization formula shown is 0, and the calculation result of the fourth part Z2*a1 in the quantization formula is 0. Therefore, the second and fourth parts in the quantization formula can be omitted and do not need to be calculated, and the corresponding two hardware modules do not need to be added additionally. In other words, only the first, third and fifth parts of the above quantization formula need to be calculated, and the output value q3 can be calculated, thereby reducing the complexity of hardware circuit design and further reducing the quantization complexity.

[0153] It can be understood that, taking 8-bit signed quantized data as an example, the quantization module 130 multiplies the offset value and the second accumulated sum to obtain a first sub-result, expressed as 𝟏𝟐𝟖∗Σ𝒒𝟐(𝒋,𝒌). The quantization module 130 multiplies the first zero point value and the second accumulated sum to obtain a second sub-result, expressed as 𝒛𝟏*𝒂𝟐(𝒌). The quantization module 130 subtracts the first sub-result and the second sub-result from the first calculation result to obtain a third sub-result, expressed as Σ(𝒒𝟏(𝒊,𝒋)+𝟏𝟐𝟖)∗𝒒𝟐(𝒋,𝒌)−𝟏𝟐𝟖∗Σ𝒒𝟐(𝒋,𝒌)–𝒛𝟏*𝒂𝟐(𝒌). The quantization module 130 multiplies the third sub-result by the quantization value scale to obtain a fourth sub-result, expressed as M*(Σ(𝒒𝟏(𝒊,𝒋)+𝟏𝟐𝟖)∗𝒒𝟐(𝒋,𝒌)−𝟏𝟐𝟖∗Σ𝒒𝟐(𝒋,𝒌)–𝒛𝟏*𝒂𝟐(𝒌)); The quantization module 130 adds the fourth sub-result and the third zero point value to obtain a quantized output result 𝒒𝟑=𝒛𝟑+M*(Σ(𝒒𝟏(𝒊,𝒋)+𝟏𝟐𝟖)∗𝒒𝟐(𝒋,𝒌)−𝟏𝟐𝟖∗Σ𝒒𝟐(𝒋,𝒌)–𝒛𝟏*𝒂𝟐(𝒌)).

[0154] In one embodiment, the specific quantization data calculation formula for 8-bit signed quantization is as follows:

[0155] 𝒒𝟑 = 𝒛𝟑+ M*(Σ(𝒒𝟏(𝒊,𝒋)+𝟏𝟐𝟖)*𝒒𝟐(𝒋,𝒌 )−𝟏𝟐𝟖∗Σ𝒒𝟐(𝒋,𝒌)–𝒛𝟏*𝒂𝟐(𝒌))

[0156] The formula can be broken down step by step to calculate as follows:

[0157] Step 1: r_𝒒𝟏(𝒊,𝒋)=𝒒𝟏(𝒊,𝒋)+𝟏𝟐𝟖;

[0158] Step 2: S = Σr_𝒒𝟏(𝒊,𝒋)* 𝒒𝟐(𝒋,𝒌);

[0159] Step 3: S = S-𝟏𝟐𝟖∗Σ𝒒𝟐(𝒋,𝒌)–𝒛𝟏*𝒂𝟐(𝒌), where 𝒂𝟐=Σ𝒒𝟐(𝒋,𝒌);

[0160] Step 4: 𝒒𝟑=S*M;

[0161] Step 5: 𝒒𝟑= 𝒒𝟑+ 𝒛𝟑.

[0162] It can be understood that the first step is to convert the first input data with a sign into the second input data without a sign, which can be achieved by a conversion module. For the conversion module, on the one hand, the conversion module can use a simple hardware module to implement the above-mentioned symbol conversion processing function, including but not limited to an adder, and its additional cost is small. On the other hand, the conversion module can also be implemented in the form of a software module, for example, by using a programming language to implement the above-mentioned symbol conversion processing function without increasing the hardware cost.

[0163] It can be understood that the second step is to implement the multiplication of the unsigned second input data matrix and the signed weight data matrix through the calculation module and then perform a cumulative summation operation.

[0164] It can be understood that after the first and second steps are calculated, in the above third step, it is necessary to subtract the offset value 𝟏𝟐𝟖∗Σ𝒒𝟐(𝒋,𝒌) corresponding to the sign conversion processing in the first step. Subtracting this part of the offset value can be achieved by a simple algorithm, so no additional hardware module is required.

[0165] Based on this, the quantization calculation device provided by the present disclosure converts the acquired signed first input data into unsigned second input data through the conversion module 110 to improve the data sparsity of the input data; as the calculation module 120 of the storage and computing integrated chip, it performs in-memory calculation on the acquired unsigned second input data and the signed weight data stored by itself, and the quantization module 130 performs data quantization on the first calculation result based on the offset value corresponding to the sign conversion process to obtain a quantized output result. Since the signed weight data is quantized and stored in the in-memory calculation array, the additional hardware modules can be reduced when the calculation results are quantized after the calculation results are subsequently output, and the designed hardware modules also become relatively simple, so that the quantization process also becomes relatively simple, and the power consumption and delay can be reduced at the chip level. Based on this, the quantization calculation device provided by the present disclosure only needs to implement less changes in hardware, and the software level is still quantized according to the signed quantized data, so as to improve the data sparsity and reduce the quantization complexity on the storage and computing integrated chip, and reduce the power consumption, reduce the calculation delay, and reduce the area of ​​the calculation module at the chip level.

[0166] In addition, an embodiment of the present disclosure further discloses an electronic device, including: at least one processor; at least one memory for storing at least one program; when the at least one program is executed by the at least one processor, the quantitative calculation method in any of the previous embodiments is implemented.

[0167] In addition, an embodiment of the present disclosure further discloses a computer-readable storage medium, in which computer-executable instructions are stored. The computer-executable instructions are used to execute the quantitative calculation method in any of the previous embodiments.

[0168] The system architecture and application scenarios described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure, and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those skilled in the art will appreciate that with the evolution of the system architecture and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are equally applicable to similar technical problems.

[0169] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "comprises" and "comprising" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0170] It should be understood that in the present disclosure, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0171] It should be understood that in the description of the embodiments of the present disclosure, the meaning of multiple (or multiple items) is more than two, greater than, less than, exceed, etc. are understood as not including the number, and above, below, within, etc. are understood as including the number.

[0172] In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0173] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0174] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0175] It should also be understood that the various implementations provided in the embodiments of the present disclosure can be combined arbitrarily to achieve different technical effects.

[0176] The above is a specific description of the implementation methods of the present disclosure, but the present disclosure is not limited to the above implementation methods. Technical personnel familiar with the art can also make various equivalent modifications or substitutions without violating the spirit of the present disclosure. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present disclosure.

Claims

1. A quantitative calculation method, characterized in that: Used in a quantitative calculation device, the quantitative calculation device includes a conversion module, a calculation module and a quantization module, and the method includes: The conversion module acquires the first input data with a sign, performs a sign conversion process on the first input data with a sign, and obtains the second input data without a sign; The calculation module obtains the second input data, performs in-memory calculation on the second input data and the weight data with a sign stored in the calculation module, and obtains a first calculation result; The quantization module performs data quantization on the first calculation result based on the offset value corresponding to the sign conversion process to obtain a quantized output result.

2. The quantitative calculation method according to claim 1, characterized in that: The conversion module is an adder, and the conversion module obtains first input data with a sign, performs a sign conversion process on the first input data with a sign, and obtains unsigned second input data, including: The adder obtains the first input data with a sign; The adder adds the first input data and the offset value corresponding to the sign conversion process to obtain the second input data.

3. The quantitative calculation method according to claim 1, characterized in that: The conversion module acquires first input data with a sign, performs a sign conversion process on the first input data with a sign, and obtains second input data without a sign, including: The conversion module obtains the first input data with a sign; The conversion module determines a sign bit and a data bit in the first input data with a sign; The conversion module performs sign conversion processing on the sign bit and the data bit according to a preset conversion rule to obtain the unsigned second input data.

4. The quantitative calculation method according to claim 3, characterized in that: The conversion module performs sign conversion processing on the sign bit and the data bit according to a preset conversion rule to obtain the unsigned second input data, including: The conversion module determines the offset value corresponding to the symbol conversion process according to a preset conversion rule; The conversion module converts the sign bit and the data bit into the second input data according to the offset value.

5. The quantitative calculation method according to claim 1, characterized in that: The calculation module includes an in-memory calculation array module, an analog-to-digital conversion module, and a shift-accumulation module. The calculation module obtains the second input data, performs in-memory calculation on the second input data and the weight data with a sign stored in the calculation module, and obtains a first calculation result, including: The in-memory computing array module obtains the second input data; The in-memory calculation array module performs corresponding multiplication on each bit of the second input data and each bit of the weight data with a sign stored in the in-memory calculation array module to obtain a plurality of product sub-results; The analog-to-digital conversion module adds the multiple product sub-results in the same column to obtain multiple accumulation sub-results; The shift-and-accumulate module performs shift-and-accumulate operations on the plurality of accumulated sub-results according to a data bit width of the second input data to obtain the first calculation result.

6. The quantitative calculation method according to claim 5, characterized in that: The shift accumulation module performs shift accumulation on the plurality of accumulated sub-results according to the data bit width of the second input data to obtain the first calculation result, including: The shift-and-accumulate module determines the data bit width of the second input data; The shift-and-accumulate module shifts and accumulates the multiple accumulated sub-results according to the data bit width to obtain the first calculation result. During the shift-and-accumulate process, the sign bit of the negative number in the weight data is calculated by subtraction, and the second input data is calculated by addition.

7. The quantitative calculation method according to claim 1, characterized in that: The calculation module obtains the second input data, performs in-memory calculation on the second input data and the weight data with a sign stored in the calculation module, and obtains a first calculation result, including: The calculation module obtains a first data matrix corresponding to the second input data and a second data matrix corresponding to the weight data; The calculation module determines a plurality of first quantization values ​​corresponding to the first data matrix and a plurality of second quantization values ​​corresponding to the second data matrix; The calculation module accumulates and sums the product of the first quantization value and the second quantization value to obtain the first calculation result.

8. The quantitative calculation method according to claim 1, characterized in that: The quantization module performs data quantization on the first calculation result based on the offset value corresponding to the sign conversion process to obtain a quantized output result, including: The quantization module determines a target quantization formula used by the quantization module; The quantization module obtains a target quantization parameter corresponding to the target quantization formula; The quantization module performs data quantization on the first calculation result according to the target quantization parameter and the offset value to obtain a quantized output result.

9. The quantitative calculation method according to claim 8, characterized in that: The target quantization parameter includes a first quantization parameter corresponding to the second input data, a second quantization parameter corresponding to the weight data, and an auxiliary quantization parameter. The first quantization parameter includes a first quantization value corresponding to the second input data, a first zero point value carried by the second input data quantization, and a first cumulative sum corresponding to the second input data. The second quantization parameter includes a second quantization value corresponding to the weight data, a second zero point value carried by the weight data quantization, and a second cumulative sum corresponding to the weight data. The auxiliary quantization parameter includes a third zero point value carried by the output quantization, a quantization value scale, and a vector length involved in the calculation. The weight data is normally distributed, and the second zero point value is zero. The quantization module performs data quantization on the first calculation result according to the target quantization parameter and the offset value to obtain a quantized output result, including: The quantization module performs data quantization on the first calculation result according to the first zero-point value, the second zero-point value, the third zero-point value, the quantization value scale, the vector length, the first quantization value, the second quantization value, the first cumulative sum, the second cumulative sum and the offset value to obtain a quantized output result, wherein the vector length, the first zero-point value and the second zero-point value are multiplied, and the second zero-point value is multiplied with the first cumulative sum.

10. The quantitative calculation method according to claim 9, characterized in that: The quantization module performs data quantization on the first calculation result according to the target quantization parameter and the offset value to obtain a quantized output result, including: The quantization module multiplies the offset value and the second accumulated sum to obtain a first sub-result; The quantization module multiplies the first zero point value and the second accumulated sum to obtain a second sub-result; The quantization module subtracts the first sub-result and the second sub-result from the first calculation result to obtain a third sub-result; The quantization module multiplies the third sub-result by the quantization value scale to obtain a fourth sub-result; The quantization module adds the fourth sub-result and the third zero point value to obtain the quantization output result.

11. A quantitative calculation device, characterized in that: The quantitative calculation device includes a conversion module, a calculation module and a quantization module, wherein: The conversion module is used to obtain first input data with a sign, perform a sign conversion process on the first input data with a sign, and obtain second input data without a sign; The calculation module is used to obtain the second input data, perform in-memory calculation on the second input data and the weight data with a sign stored in the calculation module, and obtain a first calculation result; The quantization module is used to perform data quantization on the first calculation result based on the offset value corresponding to the sign conversion process to obtain a quantized output result.

12. An electronic device, characterized in that: The electronic device includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection and communication between the processor and the memory. When the program is executed by the processor, the quantitative calculation method according to any one of claims 1 to 10 is realized.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the quantitative calculation method according to any one of claims 1 to 10.

Images