Internal memory, electronic device, and related control method

Abstract

The embodiments of the present application disclose an internal memory, an electronic device, and a related control method. The internal memory comprises a first bank and a first processing unit; a first control unit sending a first control signal to a first dequantization circuit of the first processing unit; the first dequantization circuit receiving the first control signal and a first sequence, the length of the first sequence being L; dividing the first sequence into L / M first sub-sequences, and performing dequantization on the L / M first sub-sequences to obtain L / M first target sequences; the first control unit sending a second control signal to the first dequantization circuit; the first dequantization circuit receiving the second control signal and a second sequence, the length of the second sequence being L; and dividing the second sequence into L / S second sub-sequences, and performing dequantization on the L / S second sub-sequences to obtain L / S second target sequences. Implementing the embodiments of the present application allows for data having multiple quantization precisions to be dequantized in internal memory.

Description

Memory-in-memory, electronic device and related control method

[0001] The present application claims priority to the Chinese patent application No. 202411381648.8, filed on September 27, 2024, with the State Intellectual Property Office of China, and entitled "Memory-in-memory, electronic device and related control method", the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of electronic circuits, and in particular to a memory-in-memory, electronic device and related control method. BACKGROUND

[0003] With the development of artificial intelligence technology, generative AI has emerged, such as ChatGPT, etc. Deploying large models on the terminal side for AI inference has become one of the main competitive advantages of terminals to improve user experience and productivity. There is a large amount of data that needs to be moved from the memory to the central processor, or the graphics processing unit, or the neural network processor for calculation during the large model inference process, which causes the bottleneck of the current terminal side AI inference performance to mainly concentrate on the memory bandwidth. At present, in order to improve the terminal side AI inference performance, processing in memory (PIM) has become mainstream, that is, multiple independent processing units (PU) can be integrated in the memory, and part of the computing task can be offloaded to the memory for calculation, which can improve the computing performance of the terminal and accelerate the large model inference. In addition, using the processing in memory technology can reduce data movement, reduce terminal energy consumption, and improve terminal endurance.

[0004] With the continuous increase of the size of large models, the demand for computing and storage resources of large models also increases. In order to enable these large models to run on devices with limited resources, researchers are exploring various algorithm optimization techniques, among which quantization technology is an important direction. That is, by converting the high-precision format (such as 32-bit floating-point number FP32 or 16-bit floating-point number FP16) used in large models into a low-precision format (such as 8-bit integer INT8 or 4-bit integer INT4), the storage size of the model can be effectively reduced, which helps to reduce the demand of large models on hardware resources, so that large models can run on devices with limited computing power, while also speeding up inference and reducing power consumption. However, in this process, the precision of the large model may be affected, that is, the accuracy of the large model when performing tasks may decrease. In order to minimize this precision loss, researchers have begun to focus on dequantization technology. The core idea of dequantization technology is to restore or improve the precision of the quantized model through a dequantization algorithm.

[0005] At present, the memory does not support the dequantization of data of multiple different precisions, that is, the multiple different low-precision data cannot be restored to the original high-precision data, which may cause the accuracy of the calculation result to decrease. Therefore, how to dequantize the data of multiple quantization precisions in the memory is an urgent problem to be solved. SUMMARY

[0006] Embodiments of the present application provide a memory, an electronic device and a related control method, so as to dequantize data of multiple quantization precisions in the memory.

[0007] In a first aspect, embodiments of the present application provide a memory, the memory comprising a plurality of storage areas Bank, a plurality of calculation units and a first control unit; each calculation unit comprising a dequantization circuit; wherein the first Bank corresponds to the first calculation unit, the first Bank is one of the plurality of Banks, and the first calculation unit is one of the plurality of calculation units; the first calculation unit comprises a first dequantization circuit; the first control unit is configured to send a first control signal to the first dequantization circuit according to a first sequence; the first control signal is used to indicate that the quantization precision of the first sequence is M bits; the first sequence is data stored in the first Bank; the first dequantization circuit is configured to receive the first control signal and take out the first sequence from the first Bank, the length of the first sequence being L bits; divide the first sequence into L / M first sub-sequences according to M bits, and dequantize the L / M first sub-sequences to obtain L / M first target sequences; the length of the first sub-sequence is less than the length of the first target sequence; the first control unit is further configured to send a second control signal to the first dequantization circuit according to a second sequence; the second control signal is used to indicate that the quantization precision of the second sequence is S bits, and S is greater than M; the second sequence is data stored in the first Bank; the first dequantization circuit is further configured to receive the second control signal and take out the second sequence from the first Bank, the length of the second sequence being L bits; divide the second sequence into L / S second sub-sequences according to S bits, and dequantize the L / S second sub-sequences to obtain L / S second target sequences; the length of the second sub-sequence is less than the length of the second target sequence.

[0008] In the embodiment of the present application, the internal memory supports the dequantization of data of multiple precisions. The main reason is that a first control unit is added in the internal memory. The processor can send the quantization precision of data to the first control unit in advance. Before the first dequantization circuit of the first control unit performs dequantization, the first control unit sends a control signal to the first dequantization circuit. The control signal informs the first dequantization circuit of the quantization precision of the data to be received. Then, the first dequantization circuit can dequantize the data based on the quantization precision after receiving the data. In this way, the first dequantization circuit can avoid the situation that it cannot know the quantization precision of the data to be processed in advance and can only dequantize the data according to the pre-set quantization precision. If the pre-set quantization precision is different from the quantization precision of the data to be received, the first dequantization circuit cannot correctly dequantize the data. In the present application, when the internal memory dequantizes data of multiple precisions, the first control unit can inform the first dequantization circuit of the quantization precision of the data to be received in advance. Therefore, the first dequantization circuit can dequantize the data based on the quantization precision of each sequence. In this way, the first dequantization circuit can correctly dequantize data of different quantization precisions, improve the dequantization precision, and improve the flexibility and adaptability of the internal memory, so that the internal memory can process data input of multiple quantization precisions, and thus improve the overall computing efficiency and accuracy.

[0009] In some embodiments, the first dequantization circuit includes at least L / H dequantization units, the maximum input sequence length of each dequantization unit in the L / H dequantization units is H bits, and each dequantization unit is provided with H / M output ports, and H is the least common multiple of S and M.

[0010] In the embodiment of the present application, in order to enable the internal memory to process data input of multiple quantization precisions, not only the first control unit needs to be added in the internal memory, but also the dequantization circuit needs to be designed. The first dequantization circuit supports the dequantization of data with quantization precisions of S bits and M bits to ensure that the internal memory can effectively process data of different precisions. The number of dequantization units can be determined according to the least common multiple H of S and M. This is because the processing of data often needs to be converted between different bit widths. The use of the least common multiple H can ensure that all quantization precisions can be effectively integrated and processed, thereby avoiding unnecessary errors or information loss in the data conversion process, improving the flexibility and adaptability of the internal memory, enabling the internal memory to process data input of multiple quantization precisions, and thus improving the overall computing efficiency and accuracy.

[0011] In some embodiments, each computing unit further comprises a computing circuit, wherein the first computing circuit of the first computing unit comprises L / M multiply-accumulate units, and the first computing circuit is configured to: perform computation on the L / M first target sequences by the L / M multiply-accumulate units, and output a first result; and the first computing circuit is further configured to: perform computation on the L / S second target sequences by L / S multiply-accumulate units of the L / M multiply-accumulate units, and output a second result.

[0012] In the embodiments of the present application, the computing unit PU not only comprises a dequantization circuit, but also comprises a computing circuit, the input of the computing circuit can be the output of the dequantization circuit. In order to enable the computing unit to perform dequantization on data of multiple different quantization precisions, and continue to perform parallel computing operation on the dequantized data, at least L / M multiply-accumulate units are arranged in the first computing circuit, so as to ensure that the first computing circuit can perform dequantization and computation on data of at least two quantization precisions (S and M). When the first dequantization circuit receives the first sequence, the first sequence is dequantized to obtain L / M first target sequences according to the quantization precision M bits of the first sequence. Further, the first dequantization circuit inputs the L / M first target sequences into the L / M multiply-accumulate units of the first computing circuit for computation to output a first result, and one multiply-accumulate unit performs computation on one first target sequence. When the first dequantization circuit receives the second sequence, the second sequence is dequantized to obtain L / S second target sequences according to the quantization precision S bits of the second sequence. Further, the first dequantization circuit inputs the L / S second target sequences into the L / S multiply-accumulate units of the first computing circuit for computation to output a second result, and one multiply-accumulate unit performs computation on one second target sequence.

[0013] In some embodiments, the first dequantization circuit is specifically configured to: dequantize the L / M first sub-sequences by the L / H dequantization units, and output the L / M first target sequences in parallel through L / M output ports of the L / H dequantization units; and the first dequantization circuit is specifically configured to: dequantize the L / S second sub-sequences by the L / H dequantization units, and output the L / S second target sequences in parallel through L / S output ports of the L / H dequantization units.

[0014] In the embodiment of the present application, when the first dequantization circuit needs to dequantize data with quantization precision M, the first dequantization circuit can dequantize L / M first sub-sequences through L / H dequantization units, and output L / M first target sequences through L / M output ports of the L / H dequantization units in parallel. When the first dequantization circuit needs to dequantize data with quantization precision S, the first dequantization circuit dequantizes L / S second sub-sequences through L / H dequantization units, and outputs L / S second target sequences through L / S output ports of the L / H dequantization units in parallel. The remaining (L / M-L / S) output ports can be in an idle state.

[0015] In some embodiments, L / M output ports of the L / H dequantization units are connected to L / M multiply-accumulate units, wherein each output port of each dequantization unit is connected to one multiply-accumulate unit.

[0016] In the embodiment of the present application, each output port of the dequantization unit is connected to one multiply-accumulate unit (MAC), L / M output ports of the L / H dequantization units are connected to L / M multiply-accumulate units, and the output result of the dequantization unit can be directly input into the multiply-accumulate unit for calculation, thereby improving the calculation performance of the memory.

[0017] In some embodiments, L=256, M=2, and S=4, the first calculation circuit includes 128 multiply-accumulate units, the first dequantization circuit includes 64 dequantization units, each dequantization unit is provided with 2 output ports, and the maximum input sequence length of each dequantization unit is 4 bits; a first output port of an i-th dequantization unit is connected to a 2i-th multiply-accumulate unit, a second output port of the i-th dequantization unit is connected to a 2i+1-th multiply-accumulate unit; or, the first output port of the i-th dequantization unit is connected to the i-th multiply-accumulate unit, and the second output port of the i-th dequantization unit is connected to the i+64-th multiply-accumulate unit.

[0018] In the embodiment of the present application, when L=256, M=2, and S=4, the first dequantization circuit needs to be compatible with dequantizing data with quantization precision of 2 bits and 4 bits, and therefore at least 64 dequantization units are included in the first dequantization circuit, and the maximum input sequence length of each dequantization unit is 4 bits, that is, at least two input modes exist for each dequantization unit, 2 groups of 2 bits and 1 group of 4 bits. Furthermore, each dequantization unit should be provided with 2 output ports, and after dequantizing 2 groups of 2 bits of data, the dequantization result can be output through the 2 output ports; after dequantizing 1 group of 4 bits of data, the dequantization result can be output through one of the output ports, and the other output port can be in an idle state. In addition, different connection modes can exist between the dequantization units and the MAC units, and different connection modes correspond to different data arrangement modes input to the MAC units. Through the circuit design, the memory can process data input with multiple quantization precisions, thereby improving the overall calculation efficiency and accuracy.

[0019] In some embodiments, the first control unit is further configured to send a third control signal to the first dequantization circuit according to a third sequence; the third control signal is used to indicate that the quantization precision of the third sequence is N bits; the third sequence is data stored in the first Bank; the first dequantization circuit is further configured to receive the third control signal and take out the third sequence from the first Bank, the length of the third sequence is L bits; divide the third sequence into L / N first sub-sequences according to N bits, and dequantize the L / N third sub-sequences to obtain L / N third target sequences; the length of the third sub-sequence is less than the length of the third target sequence.

[0020] In the embodiment of the present application, the memory also supports dequantizing data with more precisions, such as dequantizing data with quantization precision of N bits. Before the first dequantization unit dequantizes the data, the first control unit can send a third control signal to the first dequantization circuit to inform the first dequantization circuit of the quantization precision of the data to be received, and then the first dequantization circuit can dequantize the data based on the quantization precision after receiving the data, so that the first dequantization circuit can correctly dequantize data with different quantization precisions, improve the dequantization precision, and improve the flexibility and adaptability of the memory, so that it can process data input with multiple quantization precisions, thereby improving the overall calculation efficiency and accuracy.

[0021] In some embodiments, when N=3, 21 dequantization units are further included in the first dequantization circuit, and an output port of the jth dequantization unit of the 21 dequantization units is connected to the 65+jth multiply-accumulate unit of the 128 multiply-accumulate units; or, an output port of the jth dequantization unit of the 21 dequantization units is connected to the 1+jth multiply-accumulate unit of the 128 multiply-accumulate units.

[0022] In the embodiments of the present application, in order to enable the first calculation unit to be compatible with dequantization of data of more quantization precisions, more dequantization units can be additionally included in the first dequantization circuit. When the first calculation unit also needs to be compatible with dequantization of data of a quantization precision of 3 bits, 21 dequantization units need to be additionally added in the first dequantization circuit. An output port of the jth dequantization unit of the 21 dequantization units can be connected to the 65+jth multiply-accumulate unit of the 128 multiply-accumulate units; or, an output port of the jth dequantization unit of the 21 dequantization units can be connected to the 1+jth multiply-accumulate unit of the 128 multiply-accumulate units, so as to realize that the 21 additional dequantization units and the first 64 dequantization units can share the same multiply-accumulate unit, thereby only the number of dequantization units needs to be increased on the circuit, and the number of multiply-accumulate units does not need to be newly added, thereby saving the on-chip area.

[0023] In some embodiments, the first dequantization circuit includes at least L / I dequantization units, the maximum input sequence length of each dequantization unit of the L / I dequantization units is I bits, and each dequantization unit is provided with I / M output ports, and I is the least common multiple of S, N and M.

[0024] In the embodiments of the present application, in order to enable the calculation unit to be compatible with the dequantization of data with more quantization precision and save chip resources, the circuit needs to be designed. The first dequantization circuit 2021 includes at least L / I dequantization units, the maximum input sequence length of each dequantization unit is I bits, I is the least common multiple of S, N and M, M is less than S, N is greater than M and less than S, and the I bits can include one or more groups of low-precision data. Each dequantization unit can simultaneously dequantize at most I / M groups of low-precision data and output high-precision data after dequantization through I / M output ports. In the first dequantization circuit, supporting dequantization of data with quantization precision of S bits, N bits and M bits is to ensure that the internal memory can effectively process data with different precisions. The number of dequantization units can be determined according to the least common multiple I of S, N and M, because data processing often needs to be converted between different bit widths. The use of the least common multiple I can ensure that all quantization precisions can be effectively integrated and processed, thereby avoiding unnecessary errors or information loss in the data conversion process, improving the flexibility and adaptability of the internal memory, enabling it to process data input with multiple quantization precisions, and thus improving the overall calculation efficiency and accuracy.

[0025] In some embodiments, L=256, M=2, N=3, S=4, the first calculation circuit includes 128 multiply-accumulate units, the first dequantization circuit includes 21 first dequantization units and 1 second dequantization unit, each first dequantization unit is provided with 6 output ports, and the maximum input sequence length of each first dequantization unit is 12 bits; the second dequantization unit is provided with 2 output ports, and the maximum input sequence length of the second dequantization unit is 4 bits; wherein the first output port of the i th first dequantization unit is connected to the i th multiply-accumulate unit, the second output port of the i th first dequantization unit is connected to the i+1 th multiply-accumulate unit, the third output port of the i th first dequantization unit is connected to the i+2 th multiply-accumulate unit, the fourth output port of the i th first dequantization unit is connected to the i+3 th multiply-accumulate unit, the fifth output port of the i th first dequantization unit is connected to the i+4 th multiply-accumulate unit, the fifth output port of the i th first dequantization unit is connected to the i+4 th multiply-accumulate unit, and the sixth output port of the i th first dequantization unit is connected to the i+4 th multiply-accumulate unit; the first output port of the second dequantization unit is connected to the 126 th multiply-accumulate unit, and the second output port of the second dequantization unit is connected to the 127 th multiply-accumulate unit.

[0026] In the embodiment of the present application, when L=256, M=2, S=4, and N=3, the first dequantization circuit needs to be compatible with dequantization of data with quantization precision of 2 bits, 3 bits, and 4 bits, and therefore at least 21 first dequantization units and 1 second dequantization unit are included in the first dequantization circuit. Each first dequantization unit includes 6 output ports, and the maximum input sequence length of each first dequantization unit is 12 bits, that is, each first dequantization unit can have at least three input modes, 6 groups of 2 bits, 3 groups of 4 bits, and 4 groups of 3 bits. The second dequantization unit is provided with 2 output ports, and the maximum input sequence length of the second dequantization unit is 4 bits. The second dequantization unit can have at least three input modes, 2 groups of 2 bits, 1 group of 3 bits, and 1 group of 4 bits. The first output port of the i th first dequantization unit is connected to the i th multiply-accumulate unit, the second output port of the i th first dequantization unit is connected to the i+1 th multiply-accumulate unit, the third output port of the i th first dequantization unit is connected to the i+2 th multiply-accumulate unit, the fourth output port of the i th first dequantization unit is connected to the i+3 th multiply-accumulate unit, the fifth output port of the i th first dequantization unit is connected to the i+4 th multiply-accumulate unit, the sixth output port of the i th first dequantization unit is connected to the i+4 th multiply-accumulate unit, and the first output port of the second dequantization unit is connected to the 126 th multiply-accumulate unit. The second output port of the second dequantization unit is connected to the 127 th multiply-accumulate unit. There can be different connection modes between the dequantization unit and the MAC unit, and different connection modes correspond to different data arrangement modes input to the MAC unit. Through the circuit design, the memory can process data input with multiple quantization precisions, thereby improving the overall calculation efficiency and accuracy.

[0027] In some embodiments, the memory further includes a first register storing a first mapping table, the first mapping table including a mapping relationship between the first sub-sequences and the first target sequences, and a mapping relationship between the second sub-sequences and the second target sequences; and the first dequantization circuit is specifically configured to determine L / M first target sequences corresponding to the L / M first sub-sequences based on the first mapping table and the L / M first sub-sequences; and the first dequantization circuit is specifically configured to determine L / S second target sequences corresponding to the L / S second sub-sequences based on the first mapping table and the L / S second sub-sequences.

[0028] In the embodiment of the present application, the internal memory further includes a first register, and the first register stores a first mapping table. The first mapping table can record the mapping relationship between the first sub-sequence and the first target sequence, and the mapping relationship between the second sub-sequence and the second target sequence, that is, the mapping relationship between the first sub-sequence corresponding to the low-precision parameter A' and the first target sequence corresponding to the high-precision parameter A, and the mapping relationship between the second sub-sequence corresponding to the low-precision parameter B' and the second target sequence corresponding to the high-precision parameter B. If the first sequence and the second sequence are quantized based on the first mapping table, when the dequantization is performed, the first mapping table can be searched to achieve the dequantization, that is, each low-precision data can be converted into corresponding high-precision data through the table lookup, so as to realize the dequantization of the low-precision data.

[0029] In some embodiments, each computing unit further includes a calibration circuit, and the first calibration circuit of the first computing unit is configured to receive the first sequence and / or the second sequence sent by the first Bank, and perform error correction calibration on the first sequence and the second sequence.

[0030] In the embodiment of the present application, when the data error rate of the first Bank is high, the calibration circuit (Error Checking and Correction, ECC) can be added in the computing processing unit PU, so as to perform error correction on the data taken out from the first Bank. The data subjected to the error correction of the ECC circuit is transmitted to the dequantization circuit to perform the dequantization operation, so as to reduce the data error rate.

[0031] In a second aspect, the embodiments of the present application provide a control method applied to an in-memory, the in-memory comprising a plurality of memory banks, a plurality of computing units and a first control unit; each computing unit comprises a dequantization circuit; wherein a first memory bank corresponds to a first computing unit, the first memory bank being one of the plurality of memory banks, and the first computing unit being one of the plurality of computing units; the first computing unit comprises a first dequantization circuit; the method comprises: sending, by the first control unit, a first control signal to the first dequantization circuit according to a first sequence; the first control signal is used to indicate that the quantization precision of the first sequence is M bits; the first sequence is data stored in the first memory bank; receiving, by the first dequantization circuit, the first control signal and taking out the first sequence from the first memory bank, the length of the first sequence being L bits; dividing the first sequence into L / M first subsequences according to M bits, and dequantizing the L / M first subsequences to obtain L / M first target sequences; the length of the first subsequence is less than the length of the first target sequence; sending, by the first control unit, a second control signal to the first dequantization circuit according to a second sequence; the second control signal is used to indicate that the quantization precision of the second sequence is S bits, and S is greater than M; the second sequence is data stored in the first memory bank; receiving, by the first dequantization circuit, the second control signal and taking out the second sequence from the first memory bank, the length of the second sequence being L bits; dividing the second sequence into L / S second subsequences according to S bits, and dequantizing the L / S second subsequences to obtain L / S second target sequences; the length of the second subsequence is less than the length of the second target sequence.

[0032] In a third aspect, the embodiments of the present application provide an electronic device, comprising the in-memory of any one of the embodiments of the first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0033] FIG. 1 is a schematic diagram of a chip system according to an embodiment of the present application.

[0034] FIG. 2A is a structural schematic diagram of an in-memory using in-memory computing technology according to an embodiment of the present application.

[0035] FIG. 2B is a structural schematic diagram of a PIM-Die according to an embodiment of the present application.

[0036] FIG. 2C is a structural schematic diagram of a bank according to an embodiment of the present application.

[0037] FIG. 3 is a structural schematic diagram of an in-memory according to an embodiment of the present application.

[0038] FIG. 4A is a schematic diagram of dequantization of a first computing unit according to an embodiment of the present application.

[0039] FIG. 4B is a schematic diagram of another first calculation unit for inverse quantization according to an embodiment of the present application.

[0040] FIG. 5 is a schematic diagram of another first calculation unit for inverse quantization according to an embodiment of the present application.

[0041] FIG. 6 is a schematic diagram of a memory including a calibration circuit according to an embodiment of the present application.

[0042] FIG. 7 is a schematic diagram of a calculation circuit according to an embodiment of the present application.

[0043] FIG. 8A is a schematic diagram of a calculation circuit according to an embodiment of the present application.

[0044] FIG. 8B is a schematic diagram of another calculation circuit according to an embodiment of the present application.

[0045] FIG. 9 is a schematic diagram of another calculation circuit according to an embodiment of the present application.

[0046] FIG. 10 is a schematic diagram of a first inverse quantization circuit according to an embodiment of the present application.

[0047] FIG. 11 is a schematic diagram of a connection relationship between a first inverse quantization circuit and a first calculation circuit according to an embodiment of the present application.

[0048] FIG. 12 is a schematic diagram of a connection relationship between another first inverse quantization circuit and a first calculation circuit according to an embodiment of the present application.

[0049] FIG. 13 is a schematic diagram of a circuit compatible with INT2 and INT4 mixed quantization precision calculation according to an embodiment of the present application.

[0050] FIG. 14 is a schematic diagram of another circuit compatible with INT2 and INT4 mixed quantization precision calculation according to an embodiment of the present application.

[0051] FIG. 15 is a schematic diagram of another circuit compatible with INT2 and INT4 mixed quantization precision calculation according to an embodiment of the present application.

[0052] FIG. 16 is a schematic diagram of another circuit compatible with INT2 and INT4 mixed quantization precision calculation according to an embodiment of the present application.

[0053] FIG. 17 is a schematic diagram of an inverse quantization unit compatible with INT2, INT3 and INT4 inverse quantization operations according to an embodiment of the present application.

[0054] FIG. 18 is a schematic diagram of a circuit compatible with INT2, INT3 and INT4 mixed quantization precision calculation according to an embodiment of the present application.

[0055] FIG. 19 is a schematic diagram of a dequantization unit according to an embodiment of the present application, when the input mode is "2 groups * 2 bits".

[0056] FIG. 20 is a schematic diagram of a dequantization unit according to an embodiment of the present application, when the input mode is "1 group * 4 bits".

[0057] FIG. 21 is a schematic diagram of a dequantization unit according to an embodiment of the present application, which is compatible with INT2, INT3 and INT4 dequantization operations.

[0058] FIG. 22 is a schematic diagram of a circuit for calculating mixed quantization precision compatible with INT2, INT3 and INT4 according to an embodiment of the present application.

[0059] FIG. 23 is a schematic diagram of a model parameter quantization and corresponding dequantization process according to an embodiment of the present application.

[0060] FIG. 24 is a schematic diagram of a circuit for implementing nonlinear table lookup dequantization and linear grouping dequantization according to an embodiment of the present application.

[0061] FIG. 25 is a schematic diagram of a control method according to an embodiment of the present application. DETAILED DESCRIPTION

[0062] The embodiments of the present application will be described below with reference to the accompanying drawings.

[0063] The terms "first", "second", "third", and "fourth" and the like in the description and in the claims of the present application and the accompanying drawings are used to distinguish between similar objects, not necessarily described in a particular order. Also, the terms "comprises", "comprising", "includes", "including" and the like are to be construed open-ended, allowing for instances where there are equivalents. For example, a process, method, article, or apparatus that comprises a list of steps or elements is not necessarily limited to the listed steps or elements, but can include additional steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Additionally, the term "comprises" is used herein to mean that the method, process, or apparatus includes the recited steps or elements, but not excluding others.

[0064] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase "in an embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is expressly understood that any of the embodiments described herein can be combined with any of the other embodiments unless specifically noted otherwise.

[0065] For the purpose of facilitating understanding, the following describes some concepts related to the embodiments of the present application by way of example. As described below:

[0066] (1) Quantization: Quantization is the process of mapping a continuous, unbounded range of values to a smaller, discrete set of values. This often involves converting high-precision floating-point numbers (such as 32-bit floating-point numbers) to lower-precision data types (such as 8-bit integers). The purpose of quantization is to reduce storage requirements and computational complexity while trying to maintain the accuracy and information content of the data.

[0067] (2) Dequantization: The purpose of dequantization is to recover the original high-precision data from the low-precision data as much as possible, in order to perform further processing or analysis.

[0068] The chip system related to the embodiments of the present application is introduced as follows:

[0069] The embodiments of the present application provide a chip system. Referring to FIG. 1, FIG. 1 is a schematic diagram of a chip system provided by the embodiments of the present application, which can include but is not limited to a system on chip 101 (Soc), an internal memory 102, and an external memory 103. The chip system can be integrated in any electronic device, such as a computer, a mobile phone, a tablet, a personal digital assistant, a smart wearable device, a smart car, or a smart home appliance, and the like, and the chip system can work under necessary software drivers.

[0070] The system on chip 101 (Soc) refers to a technology of integrating a complete system on a single chip and grouping all or part of necessary electronic circuits. The system on chip 101 can include multiple processing units and a controller 1014. The multiple processing units can include one or more of a central processing unit 1011 (CPU), a modem processing unit, a graphics processing unit 1012 (GPU), an image signal processor (ISP), a video coding unit, a digital signal processor (DSP), a baseband processing unit, and a neural-network processing unit 1013 (NPU), etc. Optionally, the different processing units can communicate with each other through a primary bus, and each processing unit can also communicate with the controller 1014 through the primary bus.

[0071] The central processing unit 1011, which can also be referred to as a central processing unit (CPU) or a processor, can be a multi-core processor, i.e., a plurality of processor cores are integrated in one chip, or a single-core processor, which is not limited in the present application. The processor core is also referred to as a kernel, which is the most important component of the CPU. It is made of single-crystal silicon by a certain production process. All calculations, receiving instructions or storing instructions, and processing data of the CPU are performed by the processor core. The processor core can load instructions or data stored in the external memory 103 (such as a disk) into the internal memory 102 through the controller 1014, and transfer instructions or data that need to be operated from the internal memory 102 to the processor core through the controller 1014 for operation. When the operation is completed, the processor core can temporarily store the result in the internal memory 102, and store instructions or data that need to be stored for a long time in the external memory 103 through the controller 1014. The processor core can run an operating system, a file system (such as a flash file system F2FS) or an application program, etc. to control a plurality of hardware or software elements connected to the CPU, and can process various data and perform operations. Optionally, the central processing unit 1011 can include a memory, which can be a cache. The cache can include one or more of a level 1 cache (L1 Cache), a level 2 cache (L2 Cache), a level 3 cache (L3 Cache), etc. The cache can save instructions or data that have just been used or are frequently used by the central processing unit 1011. If the central processing unit 1011 needs to use the instructions or data again, it can be directly called from the cache. This avoids repeated access and reduces the waiting time of the processor core, thereby improving the efficiency of the central processing unit 1011.

[0072] The controller 1014 can be used to manage and control the communication between the central processing unit 1011 and the internal memory 102. The controller 1014 can also be used to manage and control the communication between the central processing unit 1011 and the external memory 103 (such as a disk), and to provide a standardized interface (such as a universal flash storage UFS standard) for communication between the central processing unit 1011 and the external storage device. It can also be understood that the central processing unit 1011 can send access instructions (such as read / write instructions) and control instructions to the internal memory 102 through the controller 1014, and the central processing unit 1011 can also send access instructions (such as read / write instructions) to the external memory 103 through the controller 1014.

[0073] The internal memory 102 (Memory) can be located outside the system on chip 101, and is usually a power-off volatile memory that loses the content stored thereon when power off, and can also be referred to as a main memory. The internal memory 102 in the present application includes a readable and writable running memory, which is used to temporarily store the operation data required by the central processing unit 1011, and interact with the external memory 103 or other external memories, and can be used as a temporary data storage medium for the operating system or other programs running.

[0074] The internal memory 102 can include one or more of a dynamic random access memory (DRAM), a static random access memory (SRAM), a synchronous dynamic random access memory (SDRAM), etc. Among them, the DRAM includes a double data rate synchronous dynamic random access memory (DDR SDRAM) referred to as DDR, a second generation double data rate synchronous dynamic random access memory (DDR2), a third generation double data rate synchronous dynamic random access memory (DDR3), a fourth generation low power double data rate synchronous dynamic random access memory (Low Power Double Data Rate 4, LPDDR4), and a fifth generation low power double data rate synchronous dynamic random access memory (Low Power Double Data Rate 5, LPDDR5), etc.

[0075] The external memory 103 is a non-volatile memory, and the content stored therein will not be lost after power off. The external memory 103 can be used to store instructions and data related to the operation of the central processing unit 1011 for a long time, such as startup programs, operating systems, application programs, and data, etc. Since the central processing unit 1011 cannot directly read instructions and data in the external memory 103, nor can it directly write instructions or data to the external memory 103, therefore, when executing a read (or load) instruction, the central processing unit 1011 actually temporarily loads the content to be read (including instructions and / or data) stored in the external memory 103 to the internal memory 102 through the controller 1014, and then reads it from the internal memory 102; and when executing a write (i.e. storage) instruction, the central processing unit 1011 actually temporarily writes the data to be stored (including instructions and / or data) to the internal memory 102, and then stores it from the internal memory 102 to the external memory 103 through the controller 1014.

[0076] The external memory 103 can include one or more of a Flash memory (e.g., a NAND Flash memory, a NOR Flash memory, etc.), a universal flash storage (UFS), an embedded Multi-Media Card (eMMC), a universal flash storage multi-chip package (uMCP) memory, an embedded Multi-Media Card multi-chip package (eMCP) memory, a solid state drive (SSD), etc.

[0077] It can be understood that the chip system in FIG. 1 is only some exemplary embodiments provided by the embodiments of the present application, and the chip system in the embodiments of the present application includes but is not limited to the above implementation.

[0078] With the development of artificial intelligence technology, generative AI is emerging, such as ChatGPT, etc. Deploying large models for AI inference on the terminal side to improve user experience and productivity has become one of the main competitiveness of the terminal. There is a large amount of data in the large model inference process that needs to be moved from the internal memory 102 to the central processing unit 1011, or the graphics processing unit 1012, or the neural network processing unit 1013 for calculation, which causes the current terminal side AI inference performance bottleneck to mainly concentrate on the memory bandwidth. At present, in order to improve the terminal side AI inference performance, processing in memory (PIM) has become mainstream, that is, multiple independent computing units (Processing Units, PU) can be integrated in the internal memory 102, and part of the computing task is offloaded to the internal memory for calculation, which can improve the computing performance of the terminal and accelerate the large model inference. At the same time, using the processing in memory technology can reduce data movement, reduce terminal energy consumption, and improve terminal endurance. The structure of the internal memory 102 and the process of reading data from the internal memory 102 and executing a computing task in the internal memory 102 will be described in detail below in conjunction with FIGS. 2A-2C.

[0079] Referring to FIG. 2A, which is a structural diagram of a memory storage using a memory computing technology according to an embodiment of the present application. The memory storage 102 can include one or more ranks (Rank). Each Rank refers to a set of memory dies (Die), and memory dies in the same set share the same address and control signals. Each Rank is usually 64 bits wide, which means it can process 64 bits of data at the same time. The memory storage 102 can have one or more Ranks, such as a single Rank (1R) and a double Rank (2R). The double Rank can include two Ranks, and each Rank can be independently accessed, thereby improving the bandwidth and performance of the memory storage 102. For example, as shown in FIG. 2A, the Rank 0 can include a plurality of Dies, and the Rank 0 can be independently used to access data. The Rank 1 can include a plurality of PIM-Dies, and the PIM-Die is a Die with a computing unit (PU) integrated inside. The Rank 1 can be independently used to access data and perform computing tasks. It should be noted that the same structure of the Die can be included in the same Rank, or different structures of the Die can be included in the same Rank, which is not limited in the present application. The structure of the PIM-Die will be described in detail below in conjunction with FIG. 2B.

[0080] Referring to FIG. 2B, FIG. 2B is a structural schematic diagram of a PIM-Die provided in an embodiment of the present application. Each PIM-Die can include a plurality of storage areas (Banks), such as Banks A-J shown in FIG. 2B. A Bank is a basic unit for storing and managing data. Each Bank can be regarded as a storage area and can process data requests independently. Different Banks allow parallel access and operation, which means that a memory controller can access multiple Banks simultaneously, thereby improving data read efficiency. For example, a PIM-Die includes 4 Banks, and data is stored in the 4 Banks. When data is accessed, the memory controller can activate data in the 4 Banks simultaneously. Each PIM-Die also integrates a plurality of computing units (PUs), such as PUs 1-5 shown in FIG. 2B. A PU is used to perform a computing task to achieve direct computing in the memory 102, instead of transferring data to an external system-on-a-chip 101 for processing, thereby improving computing efficiency. Each PU can correspond to one or more Banks. For example, PU1 shown in FIG. 2B can correspond to Banks A and F, that is, data read from Banks A and F can be directly sent to PU1 for computing. Each PIM-Die can also integrate a plurality of mode registers (Mode Register, MR), such as mode register 1 and mode register 2 shown in FIG. 2B. An MR can be used to configure the operation mode and functions of the memory 102, for example, to set parameters such as data width, delay, refresh rate, etc. The presence of the MR enables the memory controller to configure and optimize the memory 102 according to the needs of the system, thereby improving performance and compatibility. By programming these registers, the system can adjust the behavior of the memory 102 to adapt to different application scenarios and performance requirements. Each PIM-Die can also include a global buffer that can be used as a cache to store frequently accessed data to reduce access delay and improve system response speed. The structure of the Bank will be described in detail below in conjunction with FIG. 2C.

[0081] Please refer to FIG. 2C, which is a structural diagram of a Bank according to an embodiment of the present application. The Bank can include a storage array and a row buffer. The storage array is responsible for actual data storage and is composed of a plurality of storage units, which are usually organized in the form of rows and columns, i.e., the storage array can be composed of M rows * N columns of storage units, allowing efficient data access. The row buffer is used to store the row data of a certain row read from the storage array. Since data is stored in the storage array of the Bank by row and column, and in order to improve data reading efficiency, when reading data from the memory 102, a row of data in the Bank needs to be activated first, which can also be referred to as row activation. The purpose of row activation is to load the data of a specific row in the Bank to the temporary storage area of the row buffer, so that the data in the row can be quickly accessed. Once the row is activated, the row data will be loaded into the row buffer of the corresponding Bank, and then the CPU can send a compute instruction (COMPUTE) to the memory 102, which includes a column address for specifying the specific column data of the row data stored in the row buffer, and the column data is transferred to the PU corresponding to the Bank for calculation.

[0082] With the continuous increase in the size of large models, the demand for computing and storage resources of large models also increases. In order to enable these large models to run on devices with limited resources, researchers are exploring various algorithm optimization techniques, among which quantization technology is an important direction. By converting the high-precision format (such as 32-bit floating-point number FP32 or 16-bit floating-point number FP16) used in large models to a low-precision format (such as 8-bit integer INT8 or 4-bit integer INT4), the storage size of the model can be effectively reduced, which helps to reduce the demand of large models for hardware resources, so that large models can run on devices with limited computing power, while also speeding up inference and reducing power consumption. However, in this process, the precision of the large model may be affected, i.e., the accuracy of the large model when performing tasks may decrease. In order to minimize this precision loss, researchers have begun to focus on dequantization technology. The core idea of dequantization technology is to restore or improve the precision of the quantized model through a dequantization algorithm. However, at present, the memory 102 does not support dequantization of data of multiple precisions, i.e., it cannot restore multiple low-precision data to the original high-precision data, which may lead to a decrease in the precision of the calculation result. Therefore, in the present application, the memory 102 is improved so that the memory 102 can dequantize data of multiple precisions to maintain the precision of the model and improve the performance of memory computing.

[0083] Next, the internal memory involved in the present application will be introduced in combination with the drawings. As shown in FIG. 3, FIG. 3 is a structural schematic diagram of an internal memory provided by an embodiment of the present application. The internal memory 102 can include a plurality of storage areas Bank, and the plurality of storage areas Bank include a first Bank 201. The first Bank 201 can store data, which can be low-precision data after quantization. The internal memory 102 can also include a plurality of calculation units, and the plurality of calculation units include a first calculation unit 202. The first Bank 201 corresponds to the first calculation unit 202, that is, the data taken out of the first Bank 201 can be sent to the first calculation unit 202 for calculation. It should be noted that one Bank can correspond to only one calculation unit, but one calculation unit can correspond to one or more Banks. In addition, each calculation unit includes a dequantization circuit. The first calculation unit 202 includes a first dequantization circuit 2021, which can be used to dequantize the low-precision data (i.e., the data after quantization) stored in the first Bank 201.

[0084] The first control unit 203 is configured to send a first control signal to the first dequantization circuit 2021.

[0085] Specifically, the first control unit 203 can be connected with the first dequantization circuit 2021 of the first calculation unit 202, so that the first control unit 203 can send a control signal to the first dequantization circuit 2021. Optionally, the first control unit 203 can also be connected with the dequantization circuits of other calculation units, so that the first control unit 203 can send control signals to the dequantization circuits of multiple calculation units at the same time. The first control signal is used to indicate that the quantization precision of a first sequence (low-precision data) stored in the first Bank 201 and to be received by the first dequantization circuit 2021 of the first calculation unit 202 is M bits, that is, in the first sequence, M bits are used to represent one quantized low-precision parameter A', for example, M = 2, and in the first sequence, two consecutive bits are used to represent one quantized low-precision parameter A'.

[0086] The first dequantization circuit 2021 is configured to receive the first control signal and take out the first sequence from the first Bank 201.

[0087] Specifically, after receiving the first control signal sent by the first control unit 203, the first dequantization circuit 2021 can know that the quantization precision of the first sequence to be received is M bits, and then the first dequantization circuit 2021 can dequantize the first sequence based on the quantization precision after receiving the first sequence. After receiving the first control signal, the first dequantization circuit 2021 can also receive the first sequence sent by the first Bank 201. The sequence length of the first sequence can be L bits. The first sequence is data taken from the first Bank 201, and the data is quantized data.

[0088] The first dequantization circuit 2021 is also configured to divide the first sequence into L / M first sub-sequences based on M bits, and dequantize the L / M first sub-sequences to obtain L / M first target sequences.

[0089] Specifically, the first dequantization circuit 2021 knows that the quantization precision of the first sequence to be received is M bits through the first control unit 203. The first dequantization circuit 2021 can divide the first sequence into L / M first sub-sequences according to the quantization precision of the first sequence being M bits. Each first sub-sequence corresponds to a quantized low-precision parameter A'. It can also be understood that the first sequence is divided into L / M groups of data, and each group of data includes M bits.

[0090] Further, the first dequantization circuit 2021 can dequantize the L / M first sub-sequences in parallel to obtain L / M first target sequences. The first target sequence is a bit sequence corresponding to the dequantized high-precision parameter A. The length of the first target sequence is greater than the length of the first sub-sequence.

[0091] The first control unit 203 is also configured to send a second control signal to the first dequantization circuit 2021.

[0092] Specifically, the second control signal is used to indicate that the quantization precision of the second sequence (low-precision data) stored in the first Bank 201 and to be received by the first dequantization circuit 2021 is S bits, that is, S consecutive bits in the second sequence represent a parameter. S is greater than M, for example, S = 4 and M = 2. In the first sequence, two consecutive bits represent a quantized low-precision parameter A'. In the second sequence, four consecutive bits represent a quantized low-precision parameter B'.

[0093] The first dequantization circuit 2021 is also configured to receive the second control signal and take the second sequence from the first Bank 201.

[0094] Specifically, after receiving the second control signal sent by the first control unit 203, the first dequantization circuit 2021 can know that the quantization precision of the second sequence to be received is S bits, and then the first dequantization circuit 2021 can dequantize the second sequence based on the quantization precision after receiving the second sequence. After receiving the second control signal, the first dequantization circuit 2021 can also receive the second sequence sent by the first bank 201. The sequence length of the second sequence can be L bits. The second sequence is data taken from the first bank 201. The data is quantized data.

[0095] The first dequantization circuit 2021 is further configured to divide the second sequence into L / S second sub-sequences based on S bits, and dequantize the L / S second sub-sequences to obtain L / S second target sequences.

[0096] Specifically, the first dequantization circuit 2021 knows that the quantization precision of the second sequence to be received is S bits through the first control unit 203. The first dequantization circuit 2021 can divide the second sequence into L / S second sub-sequences according to the quantization precision of the second sequence being S bits. Each second sub-sequence corresponds to a quantized low-precision parameter B'. It can also be understood that the second sequence is divided into L / S groups of data, and each group of data includes S bits. Further, the first dequantization circuit 2021 can dequantize the L / S second sub-sequences in parallel to obtain L / S second target sequences. The second target sequence is a bit sequence corresponding to the dequantized high-precision parameter B. The length of the second target sequence is greater than the length of the second sub-sequence.

[0097] In summary, in the present application, the internal memory 102 supports the dequantization of data of multiple precisions, the main reason is that the first control unit 203 is added in the internal memory 102, the processor can send the quantization precision of the data to the first control unit 203 in advance, the first control unit 203 will send a control signal to the first dequantization circuit 2021 before the first dequantization circuit 2021 in the first calculation unit 202 performs dequantization, through the control signal, the first dequantization circuit 2021 is informed of the quantization precision of the data to be received, and then the first dequantization circuit 2021 can dequantize the data based on the quantization precision after receiving the data, avoiding that the first dequantization circuit 2021 cannot know the quantization precision of the data to be processed in advance, and can only dequantize the data according to the pre-set quantization precision, if the pre-set quantization precision is different from the quantization precision of the data to be received, the first dequantization circuit 2021 cannot correctly dequantize the data. In the present application, when dequantizing data of multiple precisions in the internal memory 102, the first control unit 203 can inform the first dequantization circuit 2021 of the quantization precision of the data to be received in advance, and the first dequantization circuit 2021 can dequantize the data based on the quantization precision after receiving the data, so that the first dequantization circuit 2021 can correctly dequantize the data, improve the dequantization precision, and also improve the flexibility and adaptability of the internal memory, so that it can process data input of multiple quantization precisions, and thus improve the overall calculation efficiency and accuracy.

[0098] For example, the first computing unit 202 supports dequantization of the first sequence and the second sequence, wherein the first sequence and the second sequence have different quantization precisions, the quantization precision of the first sequence is M bits, and the quantization precision of the second sequence is S bits. As shown in FIG. 4A, which is a schematic diagram of dequantization of the first computing unit according to an embodiment of the present application, when dequantization of the first sequence is needed, the first control unit 203 needs to send a first control signal to the first dequantization circuit 2021 to inform the first dequantization circuit 2021 that the quantization precision of the first sequence to be received is M bits. Then, after the first dequantization circuit 2021 receives the first sequence, the first dequantization circuit 2021 can dequantize the first sequence according to the quantization precision of the first sequence to output L / M first target sequences. As shown in FIG. 4B, which is another schematic diagram of dequantization of the first computing unit according to an embodiment of the present application, when dequantization of the second sequence is needed, the first control unit 203 needs to send a second control signal to the first dequantization circuit 2021 to inform the first dequantization circuit 2021 that the quantization precision of the second sequence to be received is S bits. Then, after the first dequantization circuit 2021 receives the second sequence, the first dequantization circuit 2021 can dequantize the second sequence according to the quantization precision of the second sequence to output L / S second target sequences. Since the first control unit 203 can inform the first dequantization circuit 2021 of the quantization precision of the data to be received in advance, the first dequantization circuit 2021 can dequantize each sequence based on the quantization precision of the sequence, so that the first dequantization circuit 2021 can correctly dequantize data with different quantization precisions, thereby improving the dequantization precision.

[0099] In some embodiments, referring to FIG. 5, which is another schematic diagram of dequantization of the first computing unit according to an embodiment of the present application, the memory 102 further includes a first register 204, the first register 204 stores a first mapping table, the first mapping table includes a mapping relationship between the first sub-sequences and the first target sequences and a mapping relationship between the second sub-sequences and the second target sequences; and the first dequantization circuit 2021 is specifically configured to determine L / M first target sequences corresponding to the L / M first sub-sequences respectively based on the first mapping table and the L / M first sub-sequences; and the first dequantization circuit 2021 is specifically configured to determine L / S second target sequences corresponding to the L / S second sub-sequences respectively based on the first mapping table and the L / S second sub-sequences.

[0100] Specifically, the internal memory 102 further includes a first register 204, which stores a first mapping table (LUT) that records a mapping relationship between the first sub-sequence and the first target sequence, and a mapping relationship between the second sub-sequence and the second target sequence, i.e., a mapping relationship between the first sub-sequence corresponding to the low-precision parameter A' and the first target sequence corresponding to the high-precision parameter A, and a mapping relationship between the second sub-sequence corresponding to the low-precision parameter B' and the second target sequence corresponding to the high-precision parameter B. If the first sequence and the second sequence are quantized based on the first mapping table, when dequantization is performed, the first mapping table can be searched to achieve dequantization, i.e., each low-precision data can be converted into corresponding high-precision data through table lookup, so as to achieve dequantization of the data.

[0101] As shown in FIG. 5, taking the quantization precision of data as INT4, and the data of INT4 needing to be dequantized as data of INT8 as an example for description. The data of INT4 needs 4 bits to be represented, and there are a total of 2 4 = 16 values, so the LUT table item size can be 16 * 8bit = 128bit. When the data of INT4 is 0b0010 = 2, the first mapping table is searched through the index value 2 to obtain the dequantized parameter 00000111, the precision of which is INT8, so as to achieve dequantization.

[0102] Optionally, the first mapping table includes a plurality of entries, each entry of the plurality of entries records a mapping relationship between a first sub-sequence and a first target sequence, or a mapping relationship between a second sub-sequence and a second target sequence; and the table item size of the first mapping table is determined according to the number of entries. For example, INT4 is dequantized through table lookup to be INT8, and the required LUT table item size is 2 4 * 8bit = 128bit, i.e., there are 16 entries in the table item, and each entry stores a parameter with a precision of INT8; INT2 is dequantized through table lookup to be INT8, and the required LUT table item size is 2 2 * 8bit = 32bit. Therefore, in order to be compatible with the calculation of mixed quantization precision of INT2 and INT4, the LUT table item size is configured as 128bit, and the first mapping table can be stored in the register with a size of 128bit in the internal memory 102.

[0103] Optionally, one PU corresponds to one LUT, or a plurality of PUs correspond to one LUT, or a plurality of LUTs correspond to one PU, which is not limited in the present application.

[0104] In some embodiments, referring to FIG. 6, which is a structural diagram of a memory including a calibration circuit according to an embodiment of the present application, the calibration circuit is further included in each calculation unit, the first calibration circuit 2022 of the first calculation unit 202 is configured to: receive the first sequence and / or the second sequence sent by the first Bank 201, and perform error correction calibration on the first sequence and the second sequence.

[0105] Specifically, when the data error rate of the first Bank 201 is high, the calibration circuit (Error Checking and Correction, ECC) can be added in the calculation processing unit PU, so as to correct the data taken out from the first Bank 201, and the data corrected by the ECC circuit is transmitted to the dequantization circuit for dequantization operation, thereby reducing the data error rate.

[0106] In some embodiments, referring to FIG. 7, which is a schematic diagram of a calculation circuit according to an embodiment of the present application, the calculation circuit is further included in each calculation unit, wherein the first calculation circuit 2023 of the first calculation unit 202 includes L / M multiply-accumulate units (Multiply Accumulate, MAC), and the first calculation circuit 2023 is configured to: calculate L / M first target sequences by using the L / M multiply-accumulate units, and output a first result; and the first calculation circuit 2023 is further configured to: calculate L / S second target sequences by using L / S multiply-accumulate units in the L / M multiply-accumulate units, and output a second result.

[0107] Specifically, the calculation unit PU in the inner memory 102 not only includes the dequantization circuit, but also includes a calculation circuit, the input of which can be the output of the dequantization circuit. In order to enable the calculation unit to dequantize data of multiple different quantization precisions and continue to perform parallel calculation operation on the dequantized data, the number of multiply-accumulate units (MAC) included in the first calculation circuit 2023 needs to be designed. In order to enable the first calculation unit 202 to dequantize and calculate data with quantization precision of M bits, L / M multiply-accumulate units (MAC) need to be included in the first calculation circuit 2023. In order to enable the first calculation unit 202 to dequantize and calculate data with quantization precision of S bits, L / S multiply-accumulate units (MAC) need to be included in the first calculation circuit 2023. Since M is less than S, at least L / M multiply-accumulate units need to be provided in the first calculation circuit 2023, ensuring that the first calculation circuit 2023 can at least dequantize and calculate data of the above two quantization precisions. When the first dequantization circuit 2021 receives the first sequence, according to the quantization precision M bits of the first sequence, the first sequence is dequantized to obtain L / M first target sequences. Further, the first dequantization circuit 2021 inputs the L / M first target sequences into the L / M multiply-accumulate units of the first calculation circuit 2023 for calculation and outputs the first result, and one multiply-accumulate unit calculates one first target sequence. When the first dequantization circuit 2021 receives the second sequence, according to the quantization precision S bits of the second sequence, the second sequence is dequantized to obtain L / S second target sequences. Further, the first dequantization circuit 2021 inputs the L / S second target sequences into the L / S multiply-accumulate units of the first calculation circuit 2023 for calculation and outputs the second result, and one multiply-accumulate unit calculates one second target sequence. At this time, the remaining (L / M-L / S) multiply-accumulate units in the first calculation circuit 2023 can be in an idle state.

[0108] For example, the first computing circuit 2023 includes L / M multiply-accumulate units (MACs). Referring to FIG. 8A, which is a working schematic of a computing circuit according to an embodiment of the present application, the first dequantization circuit 2021 inputs L / M first target sequences into the L / M multiply-accumulate units of the first computing circuit 2023, and the L / M first target sequences can be used as weight parameters. The first computing circuit 2023 can also obtain an input vector V. The L / M multiply-accumulate units of the first computing circuit 2023 perform computation on the L / M first target sequences and the input vector V and output a first result. Referring to FIG. 8B, which is another working schematic of a computing circuit according to an embodiment of the present application, the first dequantization circuit 2021 inputs L / S second target sequences into the L / S multiply-accumulate units of the first computing circuit 2023, and the L / S second target sequences can be used as weight parameters. The first computing circuit 2023 can also obtain an input vector V. The L / S multiply-accumulate units of the first computing circuit 2023 perform computation on the L / S second target sequences and the input vector V and output a second result.

[0109] Optionally, the input vector V can be stored in a global buffer of the internal memory 102 or in another bank accessible by the first computing unit 202, or be issued to the internal memory 102 by the processor, such as through an interface circuit.

[0110] In some embodiments, the L / M multiply-accumulate units can be composed of multiplication circuits, addition circuits, and accumulators.

[0111] In some embodiments, the first bank 201 is composed of a storage array including multiple rows and multiple columns of storage units. When reading data from the first bank 201, a row of data needs to be loaded into a row buffer first, and then a column (col.) of data, i.e., data of one storage unit (i.e., the first sequence or the second sequence), is taken out from the row buffer and input into the first dequantization circuit 2021 for dequantization. For example, the data taken out from one storage unit is 256 bits, and the lengths of the first sequence and the second sequence can be 256 bits.

[0112] For example, as shown in FIG. 9, which is a working schematic diagram of another calculation circuit provided in an embodiment of the present application, the first control unit 203 can also be referred to as a precision control module, or the first control unit 203 can include a precision control module. The quantization precision of the weight parameters (which can be understood as the first sequence and the second sequence described above) stored in the first Bank 201 can have multiple forms, such as INT2, INT3, INT4, INT8, etc. The precision control module can control the mode input by the first dequantization circuit 2021 according to the precision of the weight parameter Q(W) stored in the first Bank 201, so as to complete the corresponding dequantization calculation. The first dequantization circuit 2021 can convert the low-precision weight parameter Q(W) stored in the first Bank 201 into a high-precision parameter W, and the conversion mode can be multiple, such as table lookup dequantization. The first dequantization circuit 2021 has multiple input modes, thereby supporting different input data formats to realize the dequantization operation compatible with mixed quantization precision parameters. The multiply-accumulate unit in the first calculation circuit 2023 can be composed of a multiplication circuit, an addition circuit, an accumulator (which can include one or more accumulators), etc. When calculating, the input vector V is input as an operand 1 into the multiply-accumulate calculation circuit; the weight parameter W obtained through the first dequantization circuit 2021 is input as an operand 2 into the multiply-accumulate calculation circuit; the operand 1 and the operand 2 complete the calculation in the multiply-accumulate calculation circuit to obtain a result R, which can be transmitted back to the host or stored in the internal memory 102.

[0113] For example, one PU corresponds to one Bank, one first control unit 203 interfaces all PUs, and all PUs share one LUT. The weight parameter Q(Wi) stored in each Bank is transmitted to the corresponding PU, and is dequantized by querying the LUT through the respective dequantization circuit; the input vector V can be placed in a global buffer (Global Buffer) and then broadcast to each PU. The dequantized weight parameter Wi and the input vector V are operated through the multiply-accumulate circuit to obtain the calculation result.

[0114] In some embodiments, referring to FIG. 10, which is a schematic diagram of a first dequantization circuit provided in an embodiment of the present application, the first dequantization circuit 2021 includes at least L / H dequantization units, each dequantization unit in the L / H dequantization units has a maximum input sequence length of H bits, and each dequantization unit is provided with H / M output ports, and H is the least common multiple of S and M.

[0115] Specifically, in order to enable the dequantization circuit to dequantize data of multiple quantization precisions, not only is the first control unit 203 added in the internal memory 102, but also the number of dequantization units in the dequantization circuit needs to be designed. The input of each dequantization unit is low-precision data, and the output is high-precision data after dequantization. Each dequantization unit can simultaneously dequantize one or more groups of low-precision data and output the high-precision data after dequantization through one or more output ports. The first dequantization circuit 2021 includes at least L / H dequantization units, the maximum input sequence length of each dequantization unit is H bits, H is the least common multiple of S and M, M is less than S, and H bits can include one or more groups of low-precision data. Each dequantization unit can simultaneously dequantize at most H / M groups of low-precision data and output high-precision data after dequantization through H / M output ports. In the first dequantization circuit 2021, supporting dequantization of data with quantization precision of S bits and M bits is to ensure that the internal memory 102 can effectively process data of different precisions. The number of dequantization units can be determined according to the least common multiple H of S and M, because data processing often needs to be converted between different bit widths. The use of the least common multiple H can ensure that all quantization precisions can be effectively integrated and processed, thereby avoiding unnecessary errors or information loss during data conversion, improving the flexibility and adaptability of the internal memory 102, enabling it to process data input of multiple quantization precisions, and thus improving the overall computing efficiency and accuracy.

[0116] In some embodiments, the first dequantization circuit 2021 is specifically configured to dequantize the L / M first sub-sequences through the L / H dequantization units and output the L / M first target sequences in parallel through the L / M output ports of the L / H dequantization units. The first dequantization circuit 2021 is specifically configured to dequantize the L / S second sub-sequences through the L / H dequantization units and output the L / S second target sequences in parallel through the L / S output ports of the L / H dequantization units.

[0117] Specifically, since the first dequantization circuit 2021 includes at least L / H dequantization units, and each dequantization unit is provided with H / M output ports. When the first dequantization circuit 2021 needs to dequantize data with quantization precision M, the first dequantization circuit 2021 can dequantize L / M first sub-sequences through L / H dequantization units, and output L / M first target sequences through L / M output ports of the L / H dequantization units in parallel. When the first dequantization circuit 2021 needs to dequantize data with quantization precision S, the first dequantization circuit 2021 dequantizes L / S second sub-sequences through L / H dequantization units, and outputs L / S second target sequences through L / S output ports of the L / H dequantization units in parallel. The remaining (L / M-L / S) output ports can be in an idle state. Optionally, the remaining (L / M-L / S) output ports can output 0.

[0118] In some embodiments, referring to FIG. 11, which is a schematic diagram of a connection relationship between a first dequantization circuit and a first calculation circuit according to an embodiment of the present application, L / M output ports of L / H dequantization units are connected to L / M multiply-accumulate units, wherein each output port of each dequantization unit is connected to one multiply-accumulate unit.

[0119] Specifically, each output port of each dequantization unit is connected to one multiply-accumulate unit (MAC), and L / M output ports of L / H dequantization units are connected to L / M multiply-accumulate units. The output result of the dequantization unit can be directly input into the multiply-accumulate unit for calculation, thereby improving the calculation performance of the memory 102. When the dequantization unit dequantizes data with different quantization precisions, the number of output ports used is different, and there can be idle ports. The idle ports can input 0 to the corresponding multiply-accumulate units.

[0120] In some embodiments, referring to FIG. 12, which is another schematic diagram of a connection relationship between a first dequantization circuit and a first calculation circuit according to an embodiment of the present application, L=256, M=2, and S=4. The first calculation circuit 2023 includes 128 multiply-accumulate units, the first dequantization circuit 2021 includes 64 dequantization units, and each dequantization unit is provided with 2 output ports. The maximum input sequence length of each dequantization unit is 4 bits, and each output port can be connected to one multiply-accumulate unit.

[0121] Specifically, when L = 256, it represents that the data size read out from the first Bank 201 is 256 bits, i.e., the length of the first sequence and the second sequence is 256 bits; when M = 2, it represents that the quantization precision of the first sequence is 2 bits, i.e., the length of each first sub-sequence is 2 bits; when S = 4, it represents that the quantization precision of the second sequence is 4 bits, i.e., the length of each second sub-sequence is 4 bits. When L = 256, M = 2, and S = 4, the first dequantization circuit 2021 needs to be compatible with dequantizing data with quantization precisions of 2 bits and 4 bits, and therefore at least 64 dequantization units are included in the first dequantization circuit 2021, and the maximum input sequence length of each dequantization unit is 4 bits, i.e., each dequantization unit can have at least two input modes, 2 groups of 2 bits and 1 group of 4 bits, i.e., a dequantization unit supports dequantizing two first sub-sequences at the same time, and also supports dequantizing one second sub-sequence. Further, each dequantization unit should be provided with 2 output ports, which can output the dequantization result after dequantizing 2 groups of 2-bit data through the 2 output ports; and can output the dequantization result through one of the output ports after dequantizing 1 group of 4-bit data, and the other output port can be in an idle state and can output 0. Through the circuit design, the first calculation unit 202 can process data input with multiple quantization precisions, thereby improving the overall calculation efficiency and accuracy.

[0122] It should be noted that the weight parameter quantization precision is taken as INT2 and INT4, and the calculation is taken as an example for explanation after table lookup dequantization to INT8. Considering that the data of one col. in the storage array is 256 bits, when the weight parameter quantization precision is INT2, one col. can store 128 low-precision parameters, and each PU needs 128 MAC units; when the weight parameter quantization precision is INT4, one col. can store 64 low-precision parameters, and each PU only needs 64 MAC units. Therefore, in order to be compatible with INT2 / INT4 mixed quantization precision calculation, each PU needs 128 MAC multiplication and accumulation calculation units. Each PU needs 64 dequantization units, the input of each dequantization unit has two modes: “2 groups of 2 bits” or “1 group of 4 bits”, i.e., the input bit number of each dequantization unit is 4 bits, and according to the input mode, two INT2 low-precision parameters or one INT4 low-precision parameter can be obtained; the output bit number of each dequantization unit is 16 bits, i.e., two INT8 parameters are represented.

[0123] In some embodiments, the first output port of the i-th dequantization unit is connected to the 2i-th multiplication and accumulation unit, and the second output port of the i-th dequantization unit is connected to the 2i+1-th multiplication and accumulation unit.

[0124] Specifically, the output port of the dequantization unit is sequentially connected with a multiply-accumulate unit, so that the output result of the dequantization unit can be directly input into the multiply-accumulate unit for calculation, thereby improving the calculation performance of the memory 102.

[0125] It should be noted that one output port of the dequantization unit can be directly connected with the multiplier of one MAC.

[0126] For example, when the weight parameter Q(W) is INT2 quantization, the first control unit 203 controls the input mode selection of each dequantization unit to be "2 groups * 2 bits" mode, and sends two INT2 low-precision parameters into one dequantization unit, i.e., sends two first sub-sequences into one dequantization unit for dequantization, and each INT2 parameter is dequantized into an INT8-precision parameter by querying the LUT. Each dequantization unit completes the dequantization of two INT2, and one PU has 64 dequantization units, which can simultaneously complete the dequantization operation of 64*2=128 INT2 parameters. As shown in FIG. 13, which is a circuit schematic diagram for compatible INT2 and INT4 mixed quantization precision calculation provided by the embodiment of the present application, the 128 INT8 weight parameters W obtained by dequantization are sent into the 128 MAC units of the PU, i.e., MAC0, MAC1, …, MAC126, MAC127, and together with the input data V (explained in INT8 precision) sent into the 128 MAC units, the following operation is completed:

[0127] W i [1,128] *V [128,1] =[W i,0 ,W i,1 ,…,W i,j ,…,W i,127 ]*[V0,V1,…,V j ,…,V 127 ]=R i

[0128] Wherein, the input of the jth MAC unit is W i,j and V j， j is an integer from 0 to 127.

[0129] For example, when the weight parameter Q(W) is quantized as INT4, the first control unit 203 controls the input mode selection of the dequantization unit as "1 group * 4bit" mode, and sends one INT4 parameter to one dequantization unit, i.e. sends one second subsequence to one dequantization unit for dequantization, and dequantizes to an INT8 precision parameter by querying the LUT. Each dequantization unit completes the dequantization of one INT4, and one PU has 64 dequantization units, which can simultaneously complete the dequantization operation of 64*1=64 INT4 parameters. Since the dequantization unit is compatible with the dequantization of INT2, the output of the dequantization unit is 16 bits (i.e. 2 INT8). For the dequantization operation of the INT4 parameter, the effective output of each dequantization unit is 8 bits (1 INT8), and therefore in the "1 group * 4bit" mode, the 16 bits output by the dequantization unit need to be truncated to the high 8 bits or the low 8 bits as the effective output, and the remaining 8 bits are discarded. The 64 INT8 parameters obtained by dequantization only need 64 MAC units to complete the following operation. As shown in FIG. 14, the output of each dequantization unit is connected to two adjacent MAC units, and in the "1 group * 4bit" mode, only one MAC unit can obtain the valid INT8 weight parameter for calculation. Therefore, in this mode, only MAC0, MAC2, …, MAC124, MAC126 (or MAC1, MAC3, …, MAC125, MAC127) in total 64 MAC units are enabled to complete the following calculation:

[0130] W i [1,64] *V [64,1] =[W i,0 ,W i,1 ,…,W i,j ,…,W i,63 ]*[V0,V1,…,V j ,…,V 63 ]=R i

[0131] wherein the input of the 2jth MAC unit is W i,j and V j (or the input of the 2j+1th MAC unit is W i,j and V j ), and j is an integer from 0 to 63.

[0132] In some embodiments, the first output port of the ith dequantization unit is connected to the ith multiplication and accumulation unit, and the second output port of the ith dequantization unit is connected to the i+64th multiplication and accumulation unit.

[0133] Specifically, there can be different connection modes between the dequantization units and the MAC units, and different connection modes correspond to different data arrangement modes input to the MAC units. The first output port of the ith dequantization unit is connected to the ith multiply-accumulate unit, and the second output port of the ith dequantization unit is connected to the ith+64 multiply-accumulate unit, so that the output results of the dequantization units can be directly input to the multiply-accumulate units for calculation, thereby improving the calculation performance of the memory 102.

[0134] To be compatible with the table lookup dequantization operation and multiply-accumulate calculation of INT2 / 4, 64 dequantization units and 128 MAC calculation units are needed. There can be different connection modes between the dequantization units and the MAC units, and different connection modes correspond to different data arrangement modes input to the MAC units. In addition to the connection modes shown in FIGS. 13 and 14, the following takes the output of the ith dequantization unit connected to the ith MAC unit and the ith+64 MAC unit (i takes an integer value of 0-63) as an example for further description:

[0135] In the “2 groups*2bit” mode, when the connection mode between the dequantization units and the MAC units is as shown in FIG. 15, FIG. 15 is another circuit schematic diagram provided by an embodiment of the present application for compatible INT2 and INT4 hybrid quantization precision calculation. To complete the following multiply-accumulate calculation:

[0136] W i [1,128] *V [128,1] =[W i,0 ,W i,1 ,…,W i,j ,…,W i,127 ]*[V0,V1,…,V j ,…,V 127 ]=R i

[0137] When j is even, the parameters W i,j and V j are input to the j / 2th MAC unit for calculation; when j is odd, the parameters W i,j and V j are input to the (j-1) / 2+64th MAC unit for calculation (as shown in Table 1).

[0138] Table 1: Data arrangement input to the MAC unit in the “2 groups*2bit” mode

[0139] In the “1 group*4bit” mode, as shown in FIG. 16, FIG. 16 is another circuit schematic diagram provided by an embodiment of the present application for compatible INT2 and INT4 hybrid quantization precision calculation, the parameters W i,jand V j sent to the jth MAC unit for calculation, i.e., MAC units 0-63 are enabled for calculation at the same time, or parameter W i,j and V j sent to the j+64th MAC unit for calculation, i.e., MAC units 64-127 are enabled for calculation at the same time.

[0140] In summary, the scheme solves the problem that the precision of an operand is strongly coupled with a hardware circuit during calculation in the existing in-memory computing technology, and multiple data formats of the operand cannot be compatible. The scheme supports different data precisions, and can maximize the utilization of device bandwidth under different data precisions, thereby maximizing the inference performance of a large model and bringing a better experience to users.

[0141] In some embodiments, the first dequantization circuit 2021 further includes K dequantization units, and each output port of each dequantization unit is connected to one of the L / M multiply-accumulate units, where K is an integer greater than 0.

[0142] Specifically, to enable the first calculation unit 202 to be compatible with dequantization of more types of data with quantization precision, the first dequantization circuit 2021 can further include more dequantization units, improving the flexibility and adaptability of the memory 102, enabling it to process multiple types of data input with quantization precision, and thereby improving the overall calculation efficiency and accuracy. These dequantization units can be the same as or different from the L / M dequantization units described above. It should be noted that each output port of each dequantization unit in the K dequantization units is connected to one of the L / M multiply-accumulate units to realize the multiplexing of the multiply-accumulate units and reduce the chip area. When K = 21, the circuit not only supports dequantization of data with quantization precision of 2 bits and 4 bits, but also supports dequantization of data with quantization precision N of 3 bits.

[0143] In some embodiments, when N = 3, the first dequantization circuit 2021 further includes 21 dequantization units, and one output port of the jth dequantization unit in the 21 dequantization units is connected to the 65+jth multiply-accumulate unit of the 128 multiply-accumulate units; or, one output port of the jth dequantization unit in the 21 dequantization units is connected to the 1+jth multiply-accumulate unit of the 128 multiply-accumulate units.

[0144] Specifically, in order to enable the first calculation unit 202 to be compatible with the dequantization of data of more quantization precisions, more dequantization units can be additionally included in the first dequantization circuit 2021. When the first calculation unit also needs to be compatible with the dequantization of data of a quantization precision of 3 bits, 21 dequantization units need to be additionally added to the first dequantization circuit 2021. An output port of the jth dequantization unit of the 21 dequantization units can be connected to the 65th+j multiply-accumulate unit of the 128 multiply-accumulate units, or an output port of the jth dequantization unit of the 21 dequantization units can be connected to the 1st+j multiply-accumulate unit of the 128 multiply-accumulate units, so as to realize that the additional 21 dequantization units and the first 64 dequantization units can share the same multiply-accumulate unit, thereby only the number of dequantization units needs to be increased on the circuit, without the need to increase the number of multiply-accumulate units, thereby saving the on-chip area.

[0145] Taking the case of quantization precision of the weight parameters of the model as INT2, INT3, and INT4, and the need to perform calculation after table lookup dequantization to INT8 as an example for description. It is considered that the data of one col. of the storage array is 256 bits, the quantization precision of the weight parameters is INT2, one col. can store 128 parameters, and each PU needs 128 MAC units. When the quantization precision of the weight parameters is INT3, one col. can store 85 parameters, and each PU needs 85 MAC units. When the quantization precision of the weight parameters is INT4, one col. can store 64 parameters, and each PU only needs 64 MAC units. Therefore, in order to be compatible with the calculation of INT2 / INT3 / INT4 mixed quantization precision, each PU needs 128 MAC multiply-accumulate calculation units. The LUT table item size required for INT4 table lookup dequantization to INT8 is 2 4 * 8bit = 64bit; and the LUT table item size required for INT2 table lookup dequantization to INT8 is 2 3 * 8bit = 64bit; and the LUT table item size required for INT2 table lookup dequantization to INT8 is 2 2 * 8bit = 32bit. Therefore, in order to be compatible with the calculation of INT2 / INT3 / INT4 mixed quantization precision, the LUT table item size is configured as 128 bits. As shown in FIG. 17, FIG. 17 is a dequantization unit schematic diagram compatible with INT2, INT3, and INT4 dequantization operations provided by an embodiment of the present application. Each PU needs 85 dequantization units, the input of each dequantization unit has three modes: “2 groups * 2bit”, “1 group * 4bit”, and “1 group * 3bit”, that is, the input is two INT2 parameters or one INT4 parameter or one INT3 parameter; and the output bit number of the dequantization unit is 16 bits, that is, two INT8 data.

[0146] When the weight parameter Q(W) is INT3 quantization, the first control unit 203 controls the input mode selection of the dequantization unit to be "1 group * 3bit" mode, that is, 1 INT3 low-precision parameter is input into one dequantization unit, and is dequantized into an INT8 precision parameter by querying the LUT. Each dequantization unit completes the dequantization of 1 INT3, and 85 dequantization units of a PU can complete the dequantization operation of 85*1=85 INT3 parameters at the same time. Since INT2 dequantization is compatible, the output of each dequantization unit is 16 bits (that is, 2 INT8). As shown in FIG. 18, which is a circuit schematic diagram for calculating the mixed quantization precision of INT2, INT3 and INT4 compatible according to an embodiment of the present application, for the dequantization operation of the INT3 parameter, the effective output of each dequantization unit is 8 bits (1 INT8), and therefore in the "1 group * 3bit" mode, the high 8 bits or the low 8 bits of the output of the dequantization unit are taken as the effective output, and the remaining 8 bits can be discarded. The 85 INT8 weight parameters W obtained by dequantization are input into the 85 MAC units of the PU, such as MAC0, MAC1, …, MAC83, MAC84, and together with the input data V (described in INT8 precision) input into the 85 MAC units, complete the following multiplication and accumulation operation:

[0147] W i [1,85] *V [85,1] = [W i,0 , W i,1 , …, W i,j , …, W i,84 ]*[V0, V1, …, V j , …, V 84 ]=R i

[0148] , where the input of the jth MAC unit is W i,j and V j , and j is an integer from 0 to 84.

[0149] Please refer to FIG. 19, which is a dequantization unit schematic diagram when the input mode is "2 group * 2bit" according to an embodiment of the present application. When the weight parameter Q(W) is INT2 quantization, the first control unit 203 controls the input mode selection of the dequantization unit to be "2 group * 2bit" mode, and each dequantization unit can complete the dequantization of 2 INT2 parameters, that is, the effective output of each dequantization unit is 2 INT8. From the 85 dequantization units of the PU, 64 dequantization units are selected (for example, dequantization units 0-63 are selected in FIG. 19), that is, the dequantization operation of 64*2=128 INT2 parameters can be completed.

[0150] Please refer to FIG. 20, which is a schematic diagram of the inverse quantization unit when the input mode is "1 group * 4bit" according to an embodiment of the present application. When the weight parameter Q(W) is INT4 quantization, the first control unit 203 controls the input mode of the inverse quantization unit to be "1 group * 4bit". Each inverse quantization unit can complete the inverse quantization of one INT4 parameter, and the high 8 bits or the low 8 bits of the output of the inverse quantization unit are taken as the valid output. From the 85 inverse quantization units of the PU, 64 inverse quantization units (for example, the inverse quantization units 0-63 in FIG. 20) are selected, which can complete the inverse quantization of 64*1=64 INT4 parameters.

[0151] In order to be compatible with the table lookup inverse quantization operation and the multiply-accumulate calculation of INT2 / 3 / 4, 85 inverse quantization units and 128 MAC calculation units are needed. The inverse quantization units and the MAC units can have different connection modes, and different connection modes correspond to different data arrangement modes input to the MAC units. The following three connection modes are taken as examples for description.

[0152] Connection mode one: for the kth inverse quantization unit, when k takes an integer from 0 to 63, the high 8 bits and the low 8 bits of the output are connected to the kth MAC unit and the k+64th MAC unit respectively; when k takes an integer from 64 to 84, only in the "1 group * 3bit" mode, the valid 8 bits of the output are connected to the kth MAC unit, and the other 8 bits are invalid output and can not be connected to any MAC unit. In this connection mode, in order to be compatible with the following three calculations, the data arrangement input to each MAC calculation unit is shown in Table 2.

[0153] Table 2: Data arrangement input to the MAC unit in different modes of connection mode one

[0154] Connection mode two: for the kth inverse quantization unit, when k takes an integer from 0 to 63, the high 8 bits and the low 8 bits of the output are connected to the kth MAC unit and the k+1th MAC unit respectively; when k takes an integer from 64 to 84, only in the "1 group * 3bit" mode, the valid 8 bits of the output are connected to the (k-64)*2+1th MAC unit, and the other 8 bits are invalid output and can not be connected to any MAC unit. In this connection mode, in order to be compatible with the INT2 / 3 / 4 mixed precision calculation, the data arrangement input to each MAC calculation unit is shown in Table 3.

[0155] Table 3: Data arrangement input to the MAC unit in different modes of connection mode two

[0156] Connection mode three: for the kth dequantization unit, when k takes even values in the range of 0-84, the high 8 bits and the low 8 bits of its output are connected to the k / 2*3th MAC unit and the k / 2*3+1th MAC unit respectively; when k takes odd values in the range of 0-63, the high 8 bits and the low 8 bits of its output are connected to the (k-1) / 2*3+2th MAC unit and the (k-1) / 2+96th MAC unit respectively; when k takes odd values in the range of 64-84, the high 8 bits of its output are connected to the (k-1) / 2*3+2th MAC unit, and the low 8 bits of its output can not be connected to any MAC unit. Under this connection mode, to be compatible with INT2 / 3 / 4 mixed precision calculation, the data arrangement input to each MAC calculation unit is shown in Table 4.

[0157] Table 4: Data arrangement input to MAC unit under different modes of connection mode three

[0158] In some embodiments, the first control unit is further configured to send a third control signal to the first dequantization circuit 2021 according to a third sequence; the third control signal is used to indicate that the quantization precision of the third sequence is N bits; the third sequence is data stored in the first Bank; the first dequantization circuit 2021 is further configured to receive the third control signal and take out the third sequence from the first Bank, the length of the third sequence is L bits; divide the third sequence into L / N first subsequences according to N bits, and dequantize the L / N third subsequences to obtain L / N third target sequences; the length of the third subsequence is less than the length of the third target sequence.

[0159] Specifically, the internal memory also supports dequantization of more kinds of precision data, such as dequantization of data with quantization precision of N bits. Before dequantizing the data in the first dequantization unit, the first control unit can send a third control signal to the first dequantization circuit 2021 to inform the first dequantization circuit 2021 of the quantization precision of the data to be received through the control signal, so that the first dequantization circuit 2021 can dequantize the data based on the quantization precision after receiving the data. Thus, the first dequantization circuit 2021 can correctly dequantize data with different quantization precisions, improve the dequantization precision, and improve the flexibility and adaptability of the internal memory, so that it can process data input with multiple quantization precisions, thereby improving the overall calculation efficiency and accuracy.

[0160] In some embodiments, the first dequantization circuit 2021 includes at least L / I dequantization units, the maximum input sequence length of each dequantization unit in the L / I dequantization units is I bits, and each dequantization unit is provided with I / M output ports, and I is the least common multiple of S, N and M.

[0161] Specifically, in order to enable the computing unit to be compatible with the dequantization of data with more quantization precision and save chip resources, the circuit needs to be designed. The first dequantization circuit 2021 includes at least L / I dequantization units, the maximum input sequence length of each dequantization unit is I bits, I is the least common multiple of S, N and M, M is less than S, N is greater than M and less than S, and the I bits can include one or more groups of low-precision data. Each dequantization unit can simultaneously dequantize I / M groups of low-precision data at most and output the dequantized high-precision data through I / M output ports. In the first dequantization circuit 2021, supporting the dequantization of data with quantization precision of S bits, N bits and M bits is to ensure that the internal memory 102 can effectively process data with different precision. The number of dequantization units can be determined according to the least common multiple I of S, N and M, because the processing of data often needs to be converted between different bit widths. The use of the least common multiple I can ensure that all quantization precisions can be effectively integrated and processed, thereby avoiding unnecessary errors or information loss in the data conversion process, improving the flexibility and adaptability of the internal memory 102, enabling it to process data input with multiple quantization precisions, and thus improving the overall computing efficiency and accuracy.

[0162] For example, L=256, M=2, S=4, N=3, as shown in FIG. 21, which is a schematic diagram of another dequantization unit compatible with INT2, INT3 and INT4 inverse quantization operation provided in an embodiment of the present application, for table lookup dequantization compatible with INT2 / INT3 / INT4 mixed precision, the input of each dequantization unit has three modes: "6 groups*2bit", "3 groups*4bit", "4 groups*3bit", that is, the input bit number of the dequantization unit is 12bit. The output of the dequantization unit is 48bit. For the "6 groups*2bit" mode, the effective bit number of the output is 48bit (6*INT8 data); for the "3 groups*4bit" mode, the effective bit number of the output is 24bit (3*INT8 data); for the "4 groups*3bit" mode, the effective bit number of the output is 32bit (4*INT8 data). 21 dequantization units work simultaneously to complete the dequantization operation of 12bit*21=252bit data. Considering that the data amount of one col. in the array of the in-memory computing device is 256bit, an additional dequantization unit is needed, and the three modes of the input of the dequantization unit are "2 groups*2bit", "1 group*3bit", "1 group*4bit", and the output bit number is 16bit (2*INT8 data). Therefore, for table lookup dequantization compatible with INT2 / INT3 / INT4, 22 dequantization units are needed for each PU, and each dequantization unit supports three input modes, but the input bit number of the first 21 dequantization units is maximally 12bit, and the output is maximally 48bit; the input bit number of the last dequantization unit is maximally 4bit, and the output is maximally 16bit. The dequantization unit shown in FIGS. 17-20 is compatible with INT2 / INT3 / INT4 table lookup dequantization, and the input of each dequantization unit has three modes, and the maximum input bit number is 4bit. However, when dequantizing the parameter with the precision of INT3, the effective input of the dequantization unit is only 3bit, that is, 1bit of the input of each dequantization unit is wasted, and 85 dequantization units are needed to work simultaneously for each PU to complete the dequantization of one col. of INT3 data; and for the dequantization of INT2 and INT4, only 64 of the 85 dequantization units work simultaneously. Therefore, the dequantization unit shown in FIGS. 17-20 is compatible with INT2 / INT3 / INT4 dequantization, and there is a certain waste in circuit design.

[0163] In some embodiments, L=256, M=2, N=3, S=4, 128 multiply-accumulate units are included in the first calculation circuit 2023, 21 first dequantization units and 1 second dequantization unit are included in the first dequantization circuit 2021, each first dequantization unit is provided with 6 output ports, and the maximum input sequence length of each first dequantization unit is 12 bits; the second dequantization unit is provided with 2 output ports, and the maximum input sequence length of the second dequantization unit is 4 bits; wherein the first output port of the i-th first dequantization unit is connected to the i-th multiply-accumulate unit, the second output port of the i-th first dequantization unit is connected to the i+1-th multiply-accumulate unit, the third output port of the i-th first dequantization unit is connected to the i+2-th multiply-accumulate unit, the fourth output port of the i-th first dequantization unit is connected to the i+3-th multiply-accumulate unit, the fifth output port of the i-th first dequantization unit is connected to the i+4-th multiply-accumulate unit, the fifth output port of the i-th first dequantization unit is connected to the i+4-th multiply-accumulate unit, and the sixth output port of the i-th first dequantization unit is connected to the i+4-th multiply-accumulate unit; the first output port of the second dequantization unit is connected to the 126-th multiply-accumulate unit, and the second output port of the second dequantization unit is connected to the 127-th multiply-accumulate unit.

[0164] Specifically, when L = 256, M = 2, S = 4, and N = 3, the first dequantization circuit 2021 needs to be compatible with dequantization of data with quantization precisions of 2 bits, 3 bits, and 4 bits. Therefore, the first dequantization circuit 2021 includes at least 21 first dequantization units and 1 second dequantization unit. Each first dequantization unit includes 6 output ports, and the maximum input sequence length of each first dequantization unit is 12 bits. That is, each first dequantization unit can have at least three input modes: 6 groups of 2 bits, 3 groups of 4 bits, and 4 groups of 3 bits. The second dequantization unit is provided with 2 output ports, and the maximum input sequence length of the second dequantization unit is 4 bits. The second dequantization unit can have at least three input modes: 2 groups of 2 bits, 1 group of 3 bits, and 1 group of 4 bits. The first output port of the i-th first dequantization unit is connected to the i-th multiply-accumulate unit, the second output port of the i-th first dequantization unit is connected to the i+1-th multiply-accumulate unit, the third output port of the i-th first dequantization unit is connected to the i+2-th multiply-accumulate unit, the fourth output port of the i-th first dequantization unit is connected to the i+3-th multiply-accumulate unit, the fifth output port of the i-th first dequantization unit is connected to the i+4-th multiply-accumulate unit, the sixth output port of the i-th first dequantization unit is connected to the i+4-th multiply-accumulate unit, and the first output port of the second dequantization unit is connected to the 126-th multiply-accumulate unit. The second output port of the second dequantization unit is connected to the 127-th multiply-accumulate unit. The dequantization units and the MAC units can have different connection modes, and different connection modes correspond to different data arrangement modes input to the MAC units. Through the circuit design, the memory can process data input with multiple quantization precisions, thereby improving the overall calculation efficiency and accuracy.

[0165] As shown in FIG. 22, FIG. 22 is a circuit schematic diagram compatible with INT2, INT3, and INT4 mixed quantization precision calculation provided by an embodiment of the present application. For the look-up table inverse quantization operation and the multiply-add calculation operation compatible with INT2 / 3 / 4, 22 inverse quantization units and 128 MAC calculation units are needed. There can be different connection modes between the inverse quantization units and the MAC units, and the different connection modes correspond to different data arrangement modes input to the MAC units. The following takes one connection mode as an example for description. For the kth inverse quantization unit, when k takes an integer value of 0-20, its output is connected to the 6*k, 6*k+1, 6*k+2, 6*k+3, 6*k+4, and 6*k+5 MAC units; when k takes the value of 21, its output is connected to the 126th and 127th MAC units. Under this connection mode, for the INT2 / 3 / 4 mixed precision calculation, the data arrangement input to each MAC calculation unit is shown in Table 5.

[0166] Table 5: Data arrangement input to the MAC unit under different modes

[0167] In addition, as shown in FIG. 23, FIG. 23 is a model parameter quantization and its corresponding inverse quantization process schematic diagram provided by an embodiment of the present application. When the quantization process of the large model parameter is: the model parameter W is first obtained through linear quantization to obtain Q', and then obtained through nonlinear quantization to obtain Q", finally Q" can be stored in the internal memory 102. For this kind of quantization scheme, the inverse quantization process is: Q" is read from the internal memory calculation device, Q' = LUT(Q") is obtained through look-up table inverse quantization, then the model parameter W = Q'*Scale (this takes zero point Z = 0 as an example for description) is obtained by multiplying the quantization factor Scale, and finally W can participate in the actual calculation process.

[0168] As shown in FIG. 24, FIG. 24 is a circuit schematic diagram for implementing the nonlinear lookup table dequantization and linear grouping dequantization according to an embodiment of the present application. To simultaneously implement the lookup table dequantization and grouping dequantization in the memory, hardware logic for accumulating after multiplication with the quantization factor Scale needs to be added to the above circuit. The dequantization unit and the multiply-accumulate unit are grouped according to the number of quantization groups, and each group respectively completes the lookup table dequantization and multiply-accumulate calculation to obtain a first multiply-accumulate result. The result is multiplied with the quantization factor corresponding to each group to obtain the final calculation result. Take an example of a model parameter first quantized to precision INT8 (the number of quantization groups is 64, that is, 64 parameters share one Scale value) through grouping linear quantization and then quantized to precision INT2 through nonlinear quantization. When the data in one col. of the in-memory computing device is 256 bits and the data precision stored is INT2, one col. can store 128 parameters. Since the number of quantization groups is 64, the parameters are divided into two quantization groups, and the quantization factors of the two groups are Scale0 and Scale1 respectively. First, 64 dequantization units are used to dequantize the 128 INT2 weight parameters to 128 INT8 through lookup table dequantization as the input of 128 MAC calculation units. Since the weight parameters also need to be dequantized through grouping linear dequantization (the number of quantization groups is 64), the multiply-accumulate result MAC_result0 of the left 64 parameters is multiplied with the quantization factor Scale0, the multiply-accumulate result MAC_result1 of the right 64 parameters is multiplied with the quantization factor Scale1, and finally the two results are accumulated to obtain the final result.

[0169] In summary, the in-memory storage supports dequantization of data with multiple precisions. The main reason is that the first control unit is added in the in-memory storage and the dequantization circuit is improved. The processor can send the quantization precision of data to the first control unit in advance. Before the data is dequantized by the first dequantization circuit of the first calculation unit, the first control unit sends a control signal to the first dequantization circuit. The control signal informs the first dequantization circuit of the quantization precision of the data to be received. Then, the first dequantization circuit can dequantize the data based on the quantization precision after receiving the data.

[0170] As shown in FIG. 25, FIG. 25 is a flowchart of a control method according to an embodiment of the present application. The method is applied to the above in-memory storage, which includes a plurality of storage regions Bank, a plurality of calculation units and a first control unit. Each calculation unit includes a dequantization circuit. The first Bank corresponds to the first calculation unit, the first Bank is one of the plurality of Banks, and the first calculation unit is the calculation unit corresponding to the first Bank in the plurality of calculation units. As shown in FIG. 25, the control method can include:

[0171] S301: sending, by the first control unit, a first control signal to the first dequantization circuit according to a first sequence.

[0172] Specifically, the first control signal is used to indicate that the quantization precision of the first sequence is M bits; and the first sequence is data stored in the first Bank.

[0173] S302: receiving, by the first dequantization circuit, the first control signal and taking out the first sequence from the first Bank, dividing the first sequence into L / M first subsequences according to M bits, and dequantizing the L / M first subsequences to obtain L / M first target sequences.

[0174] Specifically, the length of the first sequence is L bits; and the length of the first subsequence is less than the length of the first target sequence.

[0175] S303: sending, by the first control unit, a second control signal to the first dequantization circuit according to a second sequence.

[0176] Specifically, the second control signal is used to indicate that the quantization precision of the second sequence is S bits, and S is greater than M; and the second sequence is data stored in the first Bank.

[0177] S304: receiving, by the first dequantization circuit, the second control signal and taking out the second sequence from the first Bank, dividing the second sequence into L / S second subsequences according to S bits, and dequantizing the L / S second subsequences to obtain L / S second target sequences.

[0178] Specifically, the length of the second sequence is L bits; and the length of the second subsequence is less than the length of the second target sequence.

[0179] The electronic device provided by the embodiment of the present application comprises the memory as described above.

[0180] In the above embodiments, the description of each embodiment focuses on different aspects, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0181] It should be noted that, for the above-mentioned method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, some steps can be performed in other order or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0182] In several embodiments provided in the present application, it should be understood that the disclosed apparatus can be implemented in other manners. For example, the division of the apparatus embodiments described above is merely a logical division, and there can be another division manner in actual implementation. For example, two or more units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling or direct coupling or communication connection between units can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0183] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0184] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0185] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc., and specifically can be a processor in the computer device) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium can include: a U disk, a mobile hard disk, a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), and various program codes that can be stored in the storage medium.

[0186] The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An in-memory storage, characterized by, The internal memory includes a plurality of storage areas Bank, a plurality of computing units and a first control unit; each of the computing units includes a dequantization circuit; wherein the first Bank corresponds to the first computing unit, the first Bank is one of the plurality of Banks, and the first computing unit is one of the plurality of computing units; the first computing unit includes a first dequantization circuit; The first control unit is configured to send a first control signal to the first dequantization circuit according to a first sequence; the first control signal is used to indicate that the quantization precision of the first sequence is M bits; and the first sequence is data stored in the first Bank. The first dequantization circuit is configured to receive the first control signal and take out the first sequence from the first Bank, the length of the first sequence is L bits; divide the first sequence into L / M first subsequences according to the M bits, and dequantize the L / M first subsequences to obtain L / M first target sequences; the length of the first subsequence is less than the length of the first target sequence. The first control unit is further configured to send a second control signal to the first dequantization circuit according to a second sequence; the second control signal is used to indicate that the quantization precision of the second sequence is S bits, and S is greater than M; and the second sequence is data stored in the first Bank. The first dequantization circuit is further configured to receive the second control signal and take out the second sequence from the first Bank, the length of the second sequence is L bits; divide the second sequence into L / S second subsequences according to the S bits, and dequantize the L / S second subsequences to obtain L / S second target sequences; the length of the second subsequence is less than the length of the second target sequence.

2. The memory of claim 1, wherein, The first dequantization circuit includes at least L / H dequantization units, each of the L / H dequantization units has a maximum input sequence length of H bits, and each of the dequantization units is provided with H / M output ports, and H is the least common multiple of S and M.

3. The memory of claim 2, wherein, The first dequantization circuit is specifically configured to: dequantize the L / M first subsequences through the L / H dequantization units, and output the L / M first target sequences in parallel through L / M output ports of the L / H dequantization units; dequantize the L / S second subsequences through the L / H dequantization units, and output the L / S second target sequences in parallel through L / S output ports of the L / H dequantization units.

4. The internal memory according to any one of claims 1 to 3, characterized in that, The first computing unit further includes a first computing circuit, and the first computing circuit includes L / M multiply-accumulate units, The first computing circuit is configured to calculate the L / M first target sequences through the L / M multiply-accumulate units, and calculate the L / S second target sequences through L / S multiply-accumulate units in the L / M multiply-accumulate units.

5. The memory of claim 4, wherein, L / M output ports of the L / H dequantization units are connected to the L / M multiply-accumulate units, wherein each output port of each dequantization unit is connected to one multiply-accumulate unit.

6. The memory of claim 5, wherein, L=256, M=2, S=4, the first calculation circuit includes 128 multiply-accumulate units, the first dequantization circuit includes 64 dequantization units, each dequantization unit is provided with 2 output ports, and the maximum input sequence length of each dequantization unit is 4 bits; a first output port of an i-th dequantization unit is connected to a 2i-th multiply-accumulate unit, and a second output port of the i-th dequantization unit is connected to a 2i+1-th multiply-accumulate unit; or, a first output port of an i-th dequantization unit is connected to an i-th multiply-accumulate unit, and a second output port of the i-th dequantization unit is connected to an i+64-th multiply-accumulate unit.

7. The internal memory according to any one of claims 1 to 6, characterized in that, The first control unit is further configured to send a third control signal to the first dequantization circuit according to a third sequence; the third control signal is used to indicate that the quantization precision of the third sequence is N bits; and the third sequence is data stored in the first Bank. The first dequantization circuit is further configured to receive the third control signal and take out the third sequence from the first Bank, the length of the third sequence is L bits; divide the third sequence into L / N first subsequences according to the N bits, and dequantize the L / N third subsequences to obtain L / N third target sequences; and the length of the third subsequence is less than the length of the third target sequence.

8. The memory of claim 7, wherein, When N=3, the first dequantization circuit further includes 21 dequantization units, one output port of a j-th dequantization unit in the 21 dequantization units is connected to a 65+j-th multiply-accumulate unit in the 128 multiply-accumulate units; or, one output port of a j-th dequantization unit in the 21 dequantization units is connected to a 1+j-th multiply-accumulate unit in the 128 multiply-accumulate units.

9. The memory of claim 7, wherein, The first dequantization circuit includes at least L / I dequantization units, the maximum input sequence length of each dequantization unit in the L / I dequantization units is I bits, and each dequantization unit is provided with I / M output ports, and I is the least common multiple of S, N and M.

10. The memory of claim 9, wherein, L=256, M=2, N=3, S=4, the first calculation circuit includes 128 multiply-accumulate units, the first dequantization circuit includes 21 first dequantization units and one second dequantization unit, each first dequantization unit is provided with 6 output ports, and the maximum input sequence length of each first dequantization unit is 12 bits; the second dequantization unit is provided with 2 output ports, and the maximum input sequence length of the second dequantization unit is 4 bits; The first output port of the i-th first dequantization unit is connected to the 6i-th multiply-accumulate unit, the second output port of the i-th first dequantization unit is connected to the 6i+1-th multiply-accumulate unit, the third output port of the i-th first dequantization unit is connected to the 6i+2-th multiply-accumulate unit, the fourth output port of the i-th first dequantization unit is connected to the 6i+3-th multiply-accumulate unit, the fifth output port of the i-th first dequantization unit is connected to the 6i+4-th multiply-accumulate unit, and the sixth output port of the i-th first dequantization unit is connected to the 6i+5-th multiply-accumulate unit; the first output port of the second dequantization unit is connected to the 126-th multiply-accumulate unit, and the second output port of the second dequantization unit is connected to the 127-th multiply-accumulate unit.

11. The internal memory according to any one of claims 1-10, characterized in that, The internal memory further includes a first register storing a first mapping table, the first mapping table including mapping relationships between the first sub-sequences and the first target sequences and between the second sub-sequences and the second target sequences; and the first dequantization circuit is specifically configured to: determine, based on the first mapping table and the L / M first sub-sequences, L / M first target sequences corresponding to the L / M first sub-sequences, respectively; the first dequantization circuit is specifically configured to: determine, based on the first mapping table and the L / S second sub-sequences, L / S second target sequences corresponding to the L / S second sub-sequences, respectively.

12. An electronic device, comprising: comprises: The internal memory of any one of claims 1-10.

13. A control method characterized by, The internal memory comprises a plurality of storage areas Bank, a plurality of calculation units and a first control unit; each of the calculation units comprises a dequantization circuit; wherein a first Bank corresponds to a first calculation unit, the first Bank is one of the plurality of Banks, and the first calculation unit is one of the plurality of calculation units; the first calculation unit comprises a first dequantization circuit; and the method comprises: The first control unit sends a first control signal to the first dequantization circuit according to a first sequence; the first control signal is used to indicate that the quantization precision of the first sequence is M bits; and the first sequence is data stored in the first Bank; The first dequantization circuit receives the first control signal and takes out the first sequence from the first Bank, the length of the first sequence is L bits; divides the first sequence into L / M first sub-sequences according to the M bits, and dequantizes the L / M first sub-sequences to obtain L / M first target sequences; and the length of the first sub-sequence is less than the length of the first target sequence; The first control unit sends a second control signal to the first dequantization circuit according to a second sequence; the second control signal is used to indicate that the quantization precision of the second sequence is S bits, and S is greater than M; and the second sequence is data stored in the first Bank. The first inverse quantization circuit receives the second control signal and takes out the second sequence from the first Bank, the length of the second sequence being L bits; the second sequence is divided into L / S second sub-sequences according to the S bits, and the L / S second sub-sequences are inverse quantized to obtain L / S second target sequences; the length of the second sub-sequence is less than the length of the second target sequence.

Images