Multiply-accumulate operation method and apparatus, neural network processing apparatus, and electronic device

Abstract

This application discloses a multiply-accumulate operation method, apparatus, neural network processing apparatus, and electronic device. The multiply-accumulate operation method encodes input activation data into encoded activation data and original weight data into encoded weight units, enabling the multiplication result to be obtained in the memory array simply by selecting a shift operation, replacing complex multiplier circuits and significantly reducing computational power consumption. By shifting and accumulating the first intermediate result to obtain the second intermediate result, and using computation-based compensation codes for precise compensation, the correctness of the result is ensured while avoiding the additional cycle overhead and complex circuitry caused by processing unsigned data or sign extension in traditional schemes. This improves computational throughput while effectively enhancing computing power and optimizing power consumption, ultimately comprehensively improving the energy efficiency ratio and overall processing efficiency of the multiply-accumulate operation.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a multiplication-accumulation operation method, apparatus, neural network processing apparatus, and electronic device. Background Technology

[0002] With the widespread application of deep learning and artificial intelligence (AI) technologies in fields such as inference, autonomous driving, and large-scale language models, unprecedented demands have been placed on the computing power and energy efficiency of underlying hardware. The core computing paradigm of modern AI involves massive multiply-accumulate operations, and the core multiply-accumulate units of existing neural network processors primarily rely on multiply-accumulate (MAC) operations. In the traditional von Neumann architecture, the processor and memory are separated, requiring frequent data transfer during computation, resulting in significant data movement energy consumption and access latency—the "memory wall" problem—which severely restricts the performance and energy efficiency improvement of AI chips. To fundamentally overcome the bottleneck of the "memory wall," in-memory computing technology has emerged. Its core concept is to complete computation within or near the memory unit, thereby minimizing data movement. Among these, digital in-memory computing architectures based on Static Random-Access Memory (SRAM) and compatible with Complementary Metal Oxide Semiconductor (CMOS) processes have become a key technological path for realizing high-energy-efficiency AI accelerators due to their high computational bandwidth and strong scalability.

[0003] In in-memory computing architectures, efficiently implementing multiply-accumulate operations is the core of the design. Early solutions often employed bit-sequential computation, multiplying and accumulating the input bit by bit with the weights. While structurally simple, completing one N-bit multiplication required N clock cycles, resulting in high computational latency and low throughput, becoming a major bottleneck for system performance. To accelerate computation, existing technologies have introduced an optimized scheme based on radix-4 Buss coding. This scheme encodes two consecutive bits of the input, thereby reducing the theoretical number of computation cycles from N to approximately N / 2, significantly improving computational throughput.

[0004] However, the optimization scheme based on Booth coding introduces new and more complex inherent problems in hardware implementation. Since the Booth coding scheme is inherently designed for signed operations, it requires an additional "zero-expansion" preprocessing cycle when processing unsigned data. The most common approach is to zero-expansion the N-bit unsigned number to N+1 bits (i.e., padding with a '0' before the most significant bit) before computation, making it appear as a positive signed number before feeding it into the encoder. This expansion and reformatting operation forcibly occupies at least one extra clock cycle, causing the theoretical cycle reduction benefit (e.g., from N cycles to N / 2 cycles) to be unrealized when processing unsigned data streams, resulting in a significant decrease in actual speedup and limited improvement in computational throughput. Furthermore, this scheme requires the hardware to integrate complex inversion, shifting, and dynamic sign extension circuits (e.g., dedicated inversion and shifting circuits integrated on or inside the memory array) to generate and process signed binary numbers for partial products. This not only increases system area and power consumption but also increases the combinational logic depth because these circuits are located on critical computation paths, thus limiting the system's operating frequency and introducing timing convergence problems and power overhead. Moreover, since negative partial products are involved, all intermediate operations must be performed in the two's complement domain, forcing the design to incorporate dynamic sign extension logic. These additional inversion, shifting, and sign extension operations significantly increase circuit area and power consumption.

[0005] Therefore, the existing technology for implementing multiply-accumulate calculations, while improving throughput, leads to increased power consumption, affecting the computing power and energy efficiency of multiply-accumulate operations, and also impacting the calculation cycle. Summary of the Invention

[0006] To address the aforementioned technical problems, embodiments of this application provide a multiplication-accumulation operation method, apparatus, neural network processing apparatus, and electronic device.

[0007] In a first aspect, embodiments of this application provide a multiplication-accumulation operation method, including:

[0008] The acquired input activation data is encoded to generate encoded activation data, and the encoded activation data is distributed to the in-memory array. The process of encoding the acquired input activation data to generate encoded activation data includes: splitting the input activation data into several bit units according to a preset number of bits, and encoding each bit unit to generate encoded activation data.

[0009] The acquired raw weight data is split into several weight units, and the unsigned weight units in the several weight units are encoded so that each weight unit is a signed number and the value range of each weight unit is unified to the target value range. The encoded weight units are then output to the weight storage unit in the storage array.

[0010] Based on the weighted storage unit of the target row and column in the storage array, a selection shift operation is performed on the coded activation data distributed to the target row and column, thereby obtaining the multiplication result of the weighted storage unit of the target row and column and the coded activation data of the target row and column.

[0011] The multiplication results in the same column of the storage array are added together to obtain the first intermediate result of that column;

[0012] The first intermediate results of all columns in the storage array are shifted and accumulated to obtain the second intermediate result;

[0013] The compensation code is calculated based on the input activation data and the original weight data, and then the second intermediate result is compensated by the compensation code to obtain the final multiplication and accumulation result of the input activation data and the original weight data.

[0014] In the multiply-accumulate operation method provided in this application embodiment, the encoded activation data includes a high-order part representing a positive number and a low-order part representing a negative number; the step of performing a selection shift operation on the encoded activation data of the target row and column according to the weighted storage unit of the target row and column in the storage array after encoding processing includes:

[0015] Obtain the value of the encoded weight unit in the weight storage unit of the target row and column in the storage array;

[0016] If the value is positive or negative, select the high-order part representing a positive number or the low-order part representing a negative number in the encoded activation data as the target part; shift the target part to use the shifted result as the multiplication result, or use the selected target part as the multiplication result;

[0017] If the value is 0, output 0 as the result of the multiplication.

[0018] In the multiply-accumulate operation method provided in this application embodiment, when the bit width of the weight unit after encoding processing is two bits, if the value is positive or negative, the high-order part representing a positive number or the low-order part representing a negative number in the encoded activation data is selected as the target part; the target part is shifted to use the shifted result as the multiplication result, or the selected target part is used as the multiplication result, including:

[0019] If the value is 1, the high-order part representing a positive number in the encoded activation data is selected as the target part, and the target part is used as the multiplication result;

[0020] If the value is -1, the lower-order part representing the negative number in the encoded activation data is selected as the target part, and the target part is used as the multiplication result;

[0021] If the value is -2, the lower-order part representing the negative number in the encoded activation data is selected as the target part, and the target part is shifted left by one bit. The target part after shifting left by one bit is used as the multiplication result.

[0022] In the multiply-accumulate operation method provided in this application embodiment, the step of encoding each bit unit to generate encoded activation data includes:

[0023] Each bit unit is sign-extended to obtain a first signed number;

[0024] Calculate the opposite of the first signed number to obtain the second signed number;

[0025] Take the first M bits of the first signed number, the first M bits of the second signed number, and the common least significant bit to generate encoded activation data; where M is a positive integer and is related to the preset bit position.

[0026] In the multiply-accumulate operation method provided in this application embodiment, the preset bit position is two bits, and the step of taking the first M bits of the first signed number, the first M bits of the second signed number, and the common least significant bit to generate encoded activation data includes:

[0027] The first two bits of the first signed number are taken as the first and second bits of the encoding activation data, the first two bits of the second signed number are taken as the third and fourth bits of the encoding activation data, and the least common bit of the first signed number and the second signed number is taken as the fifth bit of the encoding activation data, thereby forming the encoding activation data.

[0028] In the multiplication-accumulation method provided in the embodiments of this application, the target value range is {0, ±W, ±2W}, where W is the basic weight value of the weight unit.

[0029] In the multiplication-accumulation method provided in this application embodiment, the step of encoding the unsigned weight units among the plurality of weight units to make each weight unit a signed number and the value range of each weight unit uniformly the target value range includes:

[0030] The original weight data is split into several weight units in groups of two bits.

[0031] For each unsigned weight unit with a value range of {0, 1, 2, 3}, a subtraction operation of 2 is performed to convert it into a signed weight unit, so that each weight unit is a signed number and the value range of each weight unit is unified to the target value range {-2, -1, 0, +1}.

[0032] In the multiplication-accumulation method provided in this application embodiment, the step of adding the multiplication results of the same column in the storage array to obtain the first intermediate result of that column includes:

[0033] Invert the highest bit of the multiplication result in each row of the storage array to generate an inverted result;

[0034] The inverted result is input into an addition tree composed of multiple adders, so that the inverted results of the same column are accumulated in multiple stages in sequence to obtain the first intermediate result of each column.

[0035] In the multiply-accumulate operation method provided in this application embodiment, the step of calculating a compensation code based on the input activation data and the original weight data, and then compensating the second intermediate result using the compensation code to obtain the final multiply-accumulate result of the input activation data and the original weight data, includes:

[0036] The compensation parameters are determined based on the sign type and bit width of the original weight data;

[0037] Calculate the sum of the values ​​of all input active data participating in the current multiply-accumulate operation in the storage array;

[0038] The compensation code is calculated based on the compensation parameters and the sum, and then the second intermediate result is compensated using the compensation code to obtain the final multiplication and accumulation result of the input activation data and the original weight data.

[0039] Secondly, embodiments of this application provide a multiply-accumulate arithmetic device, including:

[0040] An activation data encoding module is used to encode the acquired input activation data to generate encoded activation data, and to distribute the encoded activation data to the in-memory array. The step of encoding the acquired input activation data to generate encoded activation data includes: splitting the input activation data into several bit units according to a preset number of bits, and encoding each bit unit to generate encoded activation data.

[0041] The weight data encoding module is used to split the acquired raw weight data into several weight units, and to encode the unsigned weight units in the several weight units so that each weight unit is a signed number and the value range of each weight unit is unified to the target value range, and to output the encoded weight units to the weight storage unit in the storage array.

[0042] The in-memory array includes several weight storage units, several selection shift modules, and several addition processing modules. The weight storage units and selection shift modules are arranged in an array. The selection shift module is used to perform a selection shift operation on the encoded activation data distributed to the target row and column based on the encoded weight units in the weight storage units of the target row and column, thereby obtaining the multiplication result of the encoded weight units and the encoded activation data of the target row and column. The addition processing modules are used to perform addition processing on the multiplication results of the same column in the in-memory array to obtain the first intermediate result of that column.

[0043] The shift-accumulate module is used to shift and accumulate the first intermediate result of all columns in the storage array to obtain the second intermediate result;

[0044] The compensation calculation module is used to calculate a compensation code based on the input activation data and the original weight data, and then use the compensation code to compensate the second intermediate result to obtain the final multiplication and accumulation result of the input activation data and the original weight data.

[0045] In the multiply-accumulate arithmetic apparatus provided in this application embodiment, the encoded activation data includes a high-order bit portion representing a positive number and a low-order bit portion representing a negative number; the selection shift module is used for:

[0046] Obtain the value of the encoded weight unit in the weight storage unit of the target row and column in the storage array;

[0047] If the value is positive or negative, select the high-order part representing a positive number or the low-order part representing a negative number in the encoded activation data as the target part;

[0048] The target portion is shifted so that the shifted result is used as the multiplication result, or the selected target portion is used as the multiplication result;

[0049] If the value is 0, output 0 as the result of the multiplication.

[0050] In the multiply-accumulate arithmetic device provided in this application embodiment, the target value range is {0, ±W, ±2W}, where W is the basic weight value of the weight unit.

[0051] In the multiply-accumulate arithmetic device provided in this application embodiment, the in-memory array is an array based on in-memory computing.

[0052] Thirdly, embodiments of this application provide a neural network processing apparatus, including the multiply-accumulate operation apparatus described above.

[0053] In the neural network processing apparatus provided in this application embodiment, the neural network processing apparatus includes a compiler, and the weight data encoding module reuses the compiler so that the weight encoding operation is completed when the compiler is offline.

[0054] Fourthly, embodiments of this application provide an electronic device including the multiply-accumulate operation device described above.

[0055] In summary, the multiply-accumulate operation method provided in this application encodes the input activation data into encoded activation data and the original weight data into encoded weight units, so that the multiplication result can be obtained in the in-memory array simply by selecting a shift operation, replacing the complex multiplier circuit and significantly reducing computational power consumption. By shifting and accumulating the first intermediate result to obtain the second intermediate result, and using a computation-based compensation code for precise compensation, the correctness of the result is ensured while avoiding the additional cycle overhead and complex circuits caused by processing unsigned data or sign extension in traditional schemes. Thus, while improving the computational throughput, it achieves effective enhancement of computing power and optimization of power consumption, ultimately comprehensively improving the energy efficiency ratio and overall processing efficiency of the multiply-accumulate operation. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a flowchart illustrating the multiplication and accumulation operation method provided in the embodiments of this application.

[0058] Figure 2 This is a flowchart illustrating the process of splitting and encoding the original weight data provided in an embodiment of this application.

[0059] Figure 3 This is a schematic diagram of the structure of the in-memory array provided in the embodiments of this application.

[0060] Figure 4 This is a schematic diagram of the addition process provided in the embodiments of this application.

[0061] Figure 5This is a flowchart illustrating the compensation calculation logic provided in the embodiments of this application.

[0062] Figure 6 This is a schematic diagram of the structure of the multiply-accumulate arithmetic device provided in the embodiments of this application.

[0063] Figure 7 This is another schematic diagram of the structure of the in-memory array provided in the embodiments of this application. Detailed Implementation

[0064] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of systems and methods consistent with some aspects of this application as detailed in the appended claims.

[0065] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0066] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0067] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0068] In the description of this application, it should be noted that the terms "upper," "lower," "left," "right," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and for simplifying the description, and do not indicate or imply that the system or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. In addition, terms such as "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0069] Based on this, embodiments of this application provide a multiplication-accumulation operation method, apparatus, neural network processing apparatus, and electronic device. Specifically, the multiplication-accumulation operation apparatus can be integrated into an electronic device, which can be a server or a terminal, etc. The terminal can include mobile phones, wearable smart devices, tablet computers, laptops, and personal computers (PCs), etc., as well as other computer and auxiliary devices. The server can be a single server or a server cluster composed of multiple servers, and can be a physical server or a virtual server.

[0070] The technical solutions shown in this application will be described in detail below through specific embodiments. It should be noted that the order of description of the following embodiments is not intended to limit the priority of the embodiments.

[0071] Please see Figure 1 , Figure 1 This is a flowchart illustrating the multiplication-accumulation operation method provided in an embodiment of this application. The specific flow of this multiplication-accumulation operation method can be as follows:

[0072] 101. Encode the acquired input activation data to generate encoded activation data, and distribute the encoded activation data to the in-memory array.

[0073] The input activation data consists of the input feature map data or the output data of intermediate layers during neural network inference. The input activation data can have different bit widths (e.g., 8 bits, 16 bits) and sign types (signed or unsigned).

[0074] In practice, the input activation data can be divided into several bit units according to preset bit positions, and each bit unit can be encoded to generate encoded activation data.

[0075] For example, when the preset bit is two bits, a multi-bit input activation data (e.g., 8-bit input activation data A8) can be sequentially split into four groups of two-bit units (A2). When the preset bit is four bits, a multi-bit input activation data (e.g., 16-bit input activation data A16) can be sequentially split into four groups of four-bit units (A4). When the preset bit is eight bits, a multi-bit input activation data (e.g., 64-bit input activation data A64) can be sequentially split into eight groups of eight-bit units (A8).

[0076] The process of encoding each bit unit to generate encoded activation data can be described as follows:

[0077] First, each bit unit is sign-extended to obtain the first signed number (denoted as ac). The sign-extending method depends on the sign type of each bit unit. For example, when the bit unit is unsigned, 0s are added before the high-order bits; when the bit unit is signed, a value identical to that high-order bit is added before the high-order bits. For instance, for an unsigned 2-bit unit 01 (decimal 1), the sign-extended first signed number ac is 001. For a signed 2-bit unit 10 (decimal 1), the sign-extended first signed number ac is 110.

[0078] Next, calculate the opposite of the first signed number ac (i.e., perform the inversion and increment operation on ac) to obtain the second signed number (denoted as acc). For example, the opposite of the first signed number ac, which is 001, is 111 (decimal -1).

[0079] Finally, the first M bits of the first signed number ac, the first M bits of the second signed number acc, and the least common bit are taken to generate the encoded activation data. Here, M is a positive integer and is related to the preset bit position. Specifically, when the preset bit position is two bits, M=2; when the preset bit position is four bits, M=4.

[0080] For example, when the bit unit is a two-bit unit, the first two bits of the first signed number ac are taken as the first and second bits of the encoding activation data, the first two bits of the second signed number acc are taken as the third and fourth bits of the encoding activation data, and the least significant bit (i.e., the last bit) of the common data of the first signed number ac and the second signed number acc is taken as the fifth bit of the encoding activation data. That is, ao[4:0] = {ac[2:1], acc[2:1], ac[0]}. This 5-bit ao is the encoding activation data.

[0081] In some embodiments, when the bit unit is a two-bit unit (A2), the above encoding rules can be shown in Tables 1 and 2 below:

[0082]

[0083] Table 1

[0084]

[0085] Table 2

[0086] In Tables 1 and 2 above, a<1:0> represents a two-bit unit (A2). For each value of a<1:0>, based on its sign type (signed or unsigned), the corresponding first signed number ac is obtained through sign extension, and its inverse acc is further calculated. Finally, by extracting the first two bits of ac and acc, and their common least significant bit, a five-bit encoded output ao<4:0> is generated, which is the encoded activation data. For example, when a<1:0> is an unsigned number 10 (decimal 2), its ac is 010, its acc is 110, and the final generated ao<4:0> is 01110.

[0087] It should be noted that the above encoding is described with a preset bit depth of two bits, and the encoding activation data is five bits. Understandably, based on the same encoding principle (i.e., merging the high-order bits of the first sign number after sign expansion and its opposite), for an M-bit unit (M being a positive integer), encoding activation data with a width of 2M+1 bits can be generated. For example, when M=3, the 3-bit unit is expanded to 4 bits (ac), generating 4 bits (acc). Then, the first 3 bits of ac are used as the first, second, and third bits of ao, and the first 3 bits of acc are used as the fourth, fifth, and sixth bits of ao. The least significant common bit of ac and acc is taken as the seventh bit (1 bit), merging them into a 7-bit encoding activation data.

[0088] After obtaining the encoded activation data (ao) through the above embodiments, the encoded activation data (ao) can be distributed to the in-memory array. Specifically, for any row in the in-memory array, its corresponding encoded activation data will be simultaneously distributed to the weight storage units and computation logic of all columns corresponding to that row. That is, all columns in the same row are operated on using the exact same encoded activation data within the same computation cycle. This distribution mechanism allows the power consumption of the external encoding logic to be shared by the concurrent computation tasks of the entire row and multiple columns, achieving effective power consumption dilution while ensuring computational parallelism.

[0089] 102. The acquired raw weight data is split into several weight units, and the unsigned weight units in the weight units are encoded so that each weight unit is a signed number and the value range of each weight unit is unified to the target value range. The encoded weight units are then output to the weight storage units in the storage array.

[0090] The original weight data can be the model parameters obtained after training. The original weight data can have different bit widths (such as 8 bits or 16 bits) and sign types (signed or unsigned).

[0091] In some embodiments, such as Figure 2 As shown, step 102 may include the following steps:

[0092] 1021. The original weight data is split according to the preset bit positions to obtain several weight units.

[0093] For example, when the preset bit depth is two bits, an 8-bit original weight data (W8) can be split into four 2-bit weight units (W2). Specifically, a signed number W8 consisting of 8 bits (wb7, wb6, wb5, wb4, wb3, wb2, wb1, wb0) can be split into four groups of 2-bit weight units: (wb7, wb6), (wb5, wb4), (wb3, wb2), and (wb1, wb0). For example, when the preset bit depth is four bits, a 16-bit original weight data (W16) can be split into four 4-bit weight units (W4). For example, when the preset bit depth is eight bits, a 64-bit original weight data (W64) can be split into eight 8-bit weight units (W8).

[0094] 1022. Identify the sign type of several weight units, encode the unsigned weight units of several weight units, so that the value range of each weight unit is unified to the target value range, generate the encoded weight units, and output the encoded weight units to the weight storage units in the memory array.

[0095] In this embodiment of the application, the target value range is {0, ±W, ±2W}, where W is the basic weight value of the weight unit.

[0096] Let's take the most common two-bit weight unit as an example for specific explanation. In this case, the basic weight value W=1, and the target value range {0, ±W, ±2W} is concretized into the numerical set {0, +1, -1, -2}. This embodiment can achieve this mapping through a specific set of encoding conversion rules. Specifically, for the two-bit unsigned weight unit with a value range of {0, 1, 2, 3}, a "minus 2" operation is performed. The result after encoding conversion is: the original value 0 is converted to -2, the original value 1 is converted to -1, the original value 2 is converted to 0, and the original value 3 is converted to +1. Finally, all weight units are uniformly converted to the signed numerical set {-2, -1, 0, +1}, that is, {-2W, -W, 0, +W}.

[0097] It is understandable that for a two-bit signed weight unit, its value range is already within the target value range {-2,-1, 0, 1}, so no encoding processing is required and its original value can be preserved.

[0098] It should be noted that after the above encoding conversion, the +2W case in the target value range {0, ±W, ±2W} has been eliminated. This is because the original unsigned values ​​2 and 3 (corresponding to +2W and +2W+W respectively) are converted to 0 and +W respectively through the "subtract 2" operation, thus simplifying the hardware implementation of subsequent multiplication operations. In the in-memory array, the selection and shift module only needs to configure the corresponding selection and shift according to the four cases {0, +W, -W, -2W} (for example, selecting the negative information part for -W, and selecting the negative information part and shifting it left by one bit for -2W), without having to design an additional complex path of "selecting the positive information part and shifting it left by one bit" to handle the +2W case.

[0099] For weight units with higher bit widths (such as 4 bits, 8 bits, and 16 bits), they consist of multiple two-bit weight units. Each two-bit unit independently follows the above encoding conversion rules and is uniformly mapped to {0, ±W, ±2W}, thereby generating its corresponding partial product. During calculation, high-precision multiplication and accumulation operations can be achieved by shifting and accumulating the partial products corresponding to different weight units, while maintaining the simplicity and efficiency of the underlying calculation logic.

[0100] Based on the above, in some embodiments, the step "encoding the unsigned weight units in the several weight units so that each weight unit is a signed number and the value range of each weight unit is unified to the target value range" can be: splitting the original weight data into several weight units in groups of two bits; for each unsigned weight unit with a value range of {0, 1, 2, 3}, performing a subtraction operation of 2 to convert it into a signed weight unit so that each weight unit is a signed number and the value range of each weight unit is unified to the target value range {-2, -1, 0, +1}.

[0101] The encoding rules for the two-bit unsigned weight unit (unsigned W2) are shown in Table 3:

[0102]

[0103] Table 3

[0104] In Table 3, w<1:0> represents the weight unit before encoding, with a value range of {0, 1, 2, 3}; wc<1:0> represents the weight unit after encoding, with its value range transformed to the target value range {-2, -1, 0, 1}. It can be understood that during encoding, the values ​​of w<1:0> can be converted to the corresponding wc<1:0> values ​​according to the mapping relationship in Table 3, thereby generating the encoded weight units.

[0105] It should be noted that the weight encoding operation in this embodiment is completed when the compiler is offline. The weight units after encoding are output to the weight storage units in the in-memory array, which can be directly called in subsequent calculations. Therefore, there is no need to add extra real-time encoding overhead in the hardware circuit.

[0106] 103. Based on the weighted storage unit of the target row and column in the storage array, perform a selection shift operation on the coded activation data distributed to the target row and column, and then obtain the multiplication result of the weighted storage unit of the target row and column and the coded activation data of the target row and column.

[0107] In the embodiments of this application, such as Figure 3 As shown, the storage array mainly includes weight storage units (i.e., Figure 3 The multiplication module includes the W2 weight, selection shift module (MUX), addition processing module (adder, addition tree), and shift accumulation module. The result of this multiplication is the partial product mentioned above.

[0108] Specifically, we can first obtain the value of the weight unit after encoding processing in the weight storage unit of the target row and column in the storage array; if the value is positive or negative, we select the high part of the encoded activation data that represents a positive number or the low part of the encoded activation data that represents a negative number as the target part, and shift the target part to use the shifted result as the multiplication result, or use the selected target part as the multiplication result; if the value is 0, we output 0 as the multiplication result.

[0109] The in-memory array in this embodiment is itself a structure that supports large-scale concurrent computation. For example, such as Figure 3As shown, assuming the in-memory array has a size of 64 rows and 192 columns (per bit), it can complete 64 rows of multiply-accumulate operations in parallel channels at a time, with each channel calculating an independent A2 * W2 multiply-accumulate result. The calculation formula is as follows:

[0110]

[0111] Where n represents the nth weight column (0~95), mac_results(n) is the multiplication and summation result corresponding to this weight column; k represents the row index (0~63), A2 k (n) and W2 k (n) represents the encoded activation data and the weighted unit after encoding processing corresponding to the k-th row and n-th column, respectively.

[0112] The in-memory array in this embodiment can flexibly support calculations with multiple bit widths. For example, by shifting and accumulating A2, calculation results corresponding to higher bit width input activation data (such as A4, A8, A16) can be obtained, resulting in combinations such as A4W2, A8W2, and A16W2. Simultaneously, by shifting and adding adjacent W2 weight columns, calculation results corresponding to higher bit width original weight data (such as W4, W8, W16) can be obtained, resulting in combinations such as A2W4, A2W8, and A2W16. Therefore, by comprehensively utilizing the above two methods, the in-memory array can complete combined operations of input activation data and original weight data with various bit widths, such as A2W2, A4W2, A8W8, and A16W16.

[0113] For example, when the preset bit depth is two bits, the activation data encoding module can provide 5 bits of encoded activation data (ao<4:0>) to the memory array each time. Each weight storage unit in the memory array, based on the value of its stored encoded weight unit (W2, with a value range of {-2, -1, 0, 1}), performs a selection shift operation on the input encoded activation data through a selection shift module (MUX), thereby replacing the traditional multiplier to perform multiplication. The selection shift operation can be specifically described as follows:

[0114] If the value is -2, the low-order part of the negative information in ao (ao<2:0>) is selected as the target part, and a left shift operation is performed. If the value is -1, the low-order part of the negative information in ao (ao<2:0>) is selected as the target part, and no shift is performed. If the value is 0, the output of MUX is zero. If the value is 1, the high-order part of the positive information in ao (ao<4:2>) is selected as the target part, and no shift is performed.

[0115] The lower part (ao<2:0>) includes the components of ao <2> 、ao <1> 、ao <0> These 3 bits constitute the part. The higher-order part (ao<4:2>) consists of ao <4> 、ao <3> 、ao <2> These 3 bits make up the composition.

[0116] That is, in some embodiments, the step "selecting the high-order part of the encoded activation data representing a positive number or the low-order part of the encoded activation data representing a negative number as the target part, and shifting or holding the target part to obtain the multiplication result" may include: when the value is 1, selecting the high-order part of the encoded activation data representing a positive number as the target part, and using the target part as the multiplication result; when the value is -1, selecting the low-order part of the encoded activation data representing a negative number as the target part, and using the target part as the multiplication result; when the value is -2, selecting the low-order part of the encoded activation data representing a negative number as the target part, performing a left shift operation on the target part by one bit, and using the left-shifted target part as the multiplication result.

[0117] It is understandable that for a coded activation data with a total bit width of (2M + 1) bits (where M is a positive integer), its structure can be uniformly defined as follows: the high-order part occupies the first (M+1) bits of the coded activation data, i.e., [2M : M]. The low-order part occupies the last (M+1) bits of the coded activation data, i.e., [M : 0].

[0118] Through the selection and shift operations described above, signed multiplication can be performed, saving power and area. At this point, the target part output by each selection-shift module (MUX) is a signed number represented in two's complement form.

[0119] 104. Perform addition on the multiplication results of the same column in the stored array to obtain the first intermediate result of that column.

[0120] In practical implementation, to simplify subsequent accumulation calculations and avoid complex sign bit extension logic, the memory array can invert the most significant bit (MSB) of each target part (the result of multiplication) before performing addition. This operation can convert a signed number represented in two's complement to an unsigned number. For example, the two's complement of 3 * (-2) = -6 is 11010, and after inverting the MSB, it becomes 01010 (decimal 10); the two's complement of 3 * 2 = 6 is 00110, and after inverting the MSB, it becomes 10110 (decimal 22).

[0121] This operation allows subsequent addition to be performed in the unsigned number field without dynamically expanding the sign bit. It's important to note that this MSB inversion operation introduces a fixed numerical offset (e.g., adding 16 bits in the example above), which will need to be corrected in subsequent compensation steps.

[0122] All target parts generated in parallel and transformed by MSB inversion are input into an addition tree consisting of multiple adders for multi-level accumulation. The final accumulated sum output by the addition tree is the first intermediate result (mac_results).

[0123] In other words, in the specific implementation process, the most significant bit (MSB) of the multiplication result (i.e. the output of MUX) of each row in the storage array can be inverted first to generate the inverted result. Then, the inverted result is input into the addition tree composed of multiple adders to perform multi-level accumulation of the inverted results of the same column in turn, thereby obtaining the first intermediate result of each column.

[0124] In some embodiments, such as Figure 4 As shown, to invert the result of a column in the storage array, the first addition is performed on rows 0 and 1 to obtain the first result; then the second and third rows are added to obtain the second result, and so on, until all inverted results of the column have been processed. Afterwards, the first and second results are added together, the third and fourth results are added together, and so on, until the last adder in the addition tree is reached. The output at this point is the first intermediate result for that column.

[0125] 105. Shift and accumulate the first intermediate results of all columns in the storage array to obtain the second intermediate result.

[0126] To support calculations with multiple bit widths, the first intermediate result can be further processed by the shift-accumulate-fusion module. This shift-accumulate-fusion module can perform bit-weighted shifting and summation fusion on the first intermediate results (i.e., multiple mac_results) of all columns in the storage array to construct a second intermediate result with a higher bit width. This second intermediate result is the initial multiplication and accumulation result of the input activation data and the original weight data.

[0127] Specifically, the basic computational unit of the in-memory array (i.e., the multiply-accumulate unit) performs the multiplication and accumulation (A2W2) of two-bit activation data and two-bit weight data, outputting multiple mac_results. When the computation task requires a higher weight bit width (such as W4, W8, W16) or a higher activation bit width (such as A4, A8, A16), the shift-accumulate fusion unit performs an arithmetic left shift (corresponding to multiplying by a power of 2) on the corresponding multiple mac_results according to the target bit width specification before accumulating them. For example, by shifting and adding the A2W2 results corresponding to adjacent columns, results such as A2W4 and A2W8 can be obtained; by shifting and accumulating the A2W2 results of different rows, results such as A4W2 and A8W2 can be obtained. For mixed bit width combinations such as A8W8, shift-accumulate fusion is applied simultaneously in both row and column dimensions.

[0128] Therefore, the essential purpose of the shift-accumulate fusion unit is to reorganize (shift and accumulate) multiple mac_results calculated in parallel by the in-memory array based on the basic granularity (A2*W2) according to the bit width structure of the high-bit-width data (such as A8, W8) by bit weight, thereby constructing a second intermediate result that reflects the multiplication and accumulation relationship between the complete high-bit-width input activation data and the high-bit-width weight data. This enables the in-memory array to flexibly and efficiently complete calculations of various bit widths such as A2W2, A4W2, A8W8, and A16W16, realizing the unification and efficient reuse of hardware computing resources.

[0129] 106. Calculate the compensation code based on the input activation data and the original weight data, and then use the compensation code to compensate the second intermediate result to obtain the final multiplication and summation result of the input activation data and the original weight data.

[0130] It is understandable that, due to the unified encoding of the original weight data in the above embodiments (for example, mapping the value range of unsigned weight units to the value range {-2, -1, 0, 1}, which is equivalent to performing a "subtract 2" operation on the unsigned weight units), and the MSB inversion operation performed on the multiplication result (converting signed numbers to unsigned numbers), the directly obtained first intermediate result (mac_results) has a systematic bias. Therefore, a compensation code needs to be calculated based on the input activation data and the original weight data to correct the first intermediate result in order to obtain the correct final multiplication-accumulation result.

[0131] Specifically, step 106 may include the following steps:

[0132] 1061. Determine the compensation parameters based on the sign type and bit width of the original weight data.

[0133] The values ​​of these compensation parameters (e.g., compensation parameters N and M) are directly related to whether the original weight data is signed or unsigned, and its specific bit width (e.g., W2, W4, W8, etc.). These compensation parameters can be calculated in advance according to rules or obtained by looking up a table. For example, they can be determined by referring to Table 4 below.

[0134]

[0135] Table 4

[0136] In Table 4 above, Weight Sign indicates the sign type of the original weight data, that is, whether the original weight data is a signed number (corresponding value of 1 in the table) or an unsigned number (corresponding value of 0 in the table). Weight type indicates the bit width of the original weight data. For example, W2, W4, and W8 represent original weight data with a bit width of 2 bits, 4 bits, and 8 bits, respectively. N is a compensation parameter related to the sign type and bit width of the original weight data. It quantifies the coefficients that need to be introduced in the compensation code calculation because the value range of the weight units after encoding is uniformly encoded to the target value range {-2, -1, 0, 1} (i.e., the "minus 2" operation). Its value can be pre-calculated according to the sign type and bit width. As shown in Table 4, for unsigned W8, N is 85 (i.e., 1+4+16+64), and for signed W8, N is 21 (i.e., 1+4+16). M is a compensation parameter related to the bit width of the original weight data. In the compensation code calculation formula, it is multiplied by a fixed base value (e.g., 43520) to correct the numerical offset introduced by inverting the most significant bit (MSB) of the target portion (converting a signed number to an unsigned number). Its value is usually related to some internal representation or scaling factor corresponding to the bit width.

[0137] 1062. Calculate the sum of the values ​​of all input active data in the storage array that participate in the current multiply-accumulate operation.

[0138] This sum can be denoted as ΣAct. In some embodiments, a dedicated multi-input adder or addition tree can be set in the compensation calculation module to sum multiple rows of input activation data. For example, in an in-memory array with 64 rows of parallel multiply-accumulate units, this embodiment performs a summation operation on these 64 rows of input activation data. If the precision of the input activation data is A8 (8 bits), then the summation operation is the addition of 64 8-bit integers.

[0139] 1063. Calculate the compensation code based on the compensation parameters and values, and then use the compensation code to compensate the second intermediate result to obtain the final multiplication and accumulation result of the input activation data and the original weight data.

[0140] In this embodiment of the application, the compensation code (FEC) can be calculated according to the following formula: FEC=2N*ΣAct–baseline value*M.

[0141] The reference value is a constant determined according to the system configuration (such as the active data bit width and the number of array rows) and is used to compensate for the numerical offset introduced by inverting the MSB of the target part (for example, for an A8 bit width and a 64-row array, the reference value can be 43520).

[0142] Specifically, the compensation code FEC can be subjected to arithmetic operations (such as addition or subtraction, depending on the direction of the deviation) with the second intermediate result to complete the numerical correction and output the final multiplication and accumulation result of the input activation data and the original weight data.

[0143] Although the embodiments of this application introduce compensation code calculation logic, the additional overhead it brings is negligible compared to the massive computational scale of the in-memory array, and it can be exchanged for a significant improvement in overall energy efficiency. For example, in a specific implementation scenario, the in-memory array can complete 24 multiply-accumulate operations of 64 rows of A8*W8 in 4 computation cycles, which is equivalent to performing 2464 A8*W8 multiplications and a massive number of first intermediate result additions in parallel. In contrast, the compensation code calculation logic usually only needs to perform a summation of multiple rows of original activation data once (such as calculating the sum of 64 8-bit data), and its total computation only accounts for about 1% to 2% of the overall computational workload of the in-memory array. Similarly, since the output of the activation data encoding module is shared by a large number of column-parallel multiply-accumulate units in the in-memory array, the fixed overhead of the hardware area and power consumption of the activation data encoding module is amortized by the massive computational scale, thus being significantly diluted at the system level. Furthermore, this shared encoding mechanism allows the in-memory array to use extremely low-power shift selection modules instead of traditional high-power complex multipliers by encoding the input activation data. Therefore, while introducing negligible coding overhead, the power consumption of the core computing path was drastically reduced, ultimately achieving a significant improvement in overall energy efficiency.

[0144] Therefore, this application embodiment introduces an encoding strategy and replaces complex multipliers with selection and shift operations, enabling flexible support for calculations with various bit widths, including A2, A4, A8, and A16, while maintaining hardware complexity similar to the single-bit data (A1) processing scheme. This not only improves computational throughput but also significantly reduces the overall power consumption of the in-memory array by nearly half due to the simplification of the core computation path. This application embodiment completes the complex encoding and compensation calculations outside the in-memory array, which itself consists of a large number of multi-row, multi-column computational logics, resulting in a massive amount of computation completed in a single operation. Therefore, the power consumption and area of ​​the logic circuits added externally are diluted by the massive scale of parallel computation within the array. In other words, this application embodiment achieves a significant simultaneous improvement in computing power and energy efficiency at the cost of minimal external logic overhead.

[0145] In summary, the multiply-accumulate operation method provided in this application encodes the input activation data into encoded activation data and the original weight data into encoded weight units, so that the multiplication result can be obtained in the in-memory array simply by selecting a shift operation, replacing the complex multiplier circuit and significantly reducing the computational power consumption. By shifting and accumulating the first intermediate result to obtain the second intermediate result, and using a computation-based compensation code for precise compensation, the correctness of the result is ensured while avoiding the additional cycle overhead and complex circuits caused by processing unsigned data or sign extension in traditional schemes. Thus, while improving the computational throughput, the computing power is effectively enhanced and the power consumption is optimized, ultimately improving the energy efficiency ratio and overall processing efficiency of the multiply-accumulate operation.

[0146] It should be noted that the multiply-accumulate operation method provided in this application can be applied to various scenarios that require multiply-accumulate operations. It is particularly suitable for digital in-memory computing systems with stringent requirements for energy efficiency and throughput, such as artificial intelligence chips, especially AI inference chips, high-performance digital signal processors (DSPs), neural network processing units (NPUs) in mobile devices, and customized acceleration cores in edge computing devices.

[0147] To facilitate better implementation of the multiplication-accumulation operation method provided in the embodiments of this application, the embodiments of this application also provide a multiplication-accumulation operation device. The meanings of the terms used are the same as in the multiplication-accumulation operation method described above, and specific implementation details can be found in the descriptions in the method embodiments.

[0148] Please see Figures 6-7 , Figure 6 This is a schematic diagram of the structure of the multiply-accumulate arithmetic device provided in the embodiments of this application. Figure 7This is a schematic diagram of the structure of the in-memory array provided in an embodiment of this application. The multiply-accumulate operation device may include an activation data encoding module 201, a weight data encoding module 202, an in-memory array 203, a shift-accumulate module 204, and a compensation calculation module 205. Among them,

[0149] The activation data encoding module 201 is used to encode the acquired input activation data to generate encoded activation data and distribute the encoded activation data to the in-memory array. Encoding the acquired input activation data to generate encoded activation data includes: splitting the input activation data into several bit units according to preset bit positions and encoding each bit unit to generate encoded activation data.

[0150] The weight data encoding module 202 is used to split the acquired raw weight data into several weight units, and to encode the unsigned weight units in the several weight units so that each weight unit is a signed number and the value range of each weight unit is unified to the target value range, and to output the encoded weight units to the weight storage unit in the storage array.

[0151] The in-memory array 203 includes several weight storage units 2031, several selection shift modules 2032, and several addition processing modules 2033. The weight storage units 2031 and the selection shift modules 2032 are arranged in an array. The selection shift module 2032 is used to perform a selection shift operation on the encoded activation data distributed to the target row and column according to the weight units after encoding processing in the weight storage units 2031 of the target row and column, thereby obtaining the multiplication result of the weight units after encoding processing in the target row and column and the encoded activation data of the target row and column. The addition processing module 2033 is used to perform addition processing on the multiplication results of the same column in the in-memory array to obtain the first intermediate result of the column.

[0152] The shift-accumulate module 204 is used to shift and accumulate the first intermediate results of all columns in the storage array to obtain the second intermediate result;

[0153] The compensation calculation module 205 is used to calculate the compensation code based on the input activation data and the original weight data, and then use the compensation code to compensate the second intermediate result to obtain the final multiplication and accumulation result of the input activation data and the original weight data.

[0154] In some embodiments, the in-memory computing array 203 described above is an in-memory computing-based array.

[0155] In some embodiments, the encoded activation data includes a high-order part representing a positive number and a low-order part representing a negative number; the selection shift module 2032 is used to: obtain the value of the encoded weight unit in the weight storage unit of the target row and column in the storage array; if the value is positive or negative, select the high-order part representing a positive number or the low-order part representing a negative number in the encoded activation data as the target part; shift the target part to use the shifted result as the multiplication result, or use the selected target part as the multiplication result; if the value is 0, output 0 as the multiplication result.

[0156] In this embodiment of the application, the target value range is {0, ±W, ±2W}, where W is the basic weight value of the weight unit.

[0157] For specific implementation methods of each of the above units, please refer to the embodiments of the multiplication and accumulation operation method described above, which will not be repeated here.

[0158] In summary, the multiply-accumulate operation device provided in this application can encode input activation data into encoded activation data and original weight data into encoded weight units, so that the multiplication result can be obtained in the in-memory array simply by selecting a shift operation, replacing the complex multiplier circuit and significantly reducing computational power consumption. By shifting and accumulating the first intermediate result to obtain the second intermediate result, and using a computation-based compensation code for precise compensation, the correctness of the result is ensured while avoiding the additional cycle overhead and complex circuits caused by processing unsigned data or sign extension in traditional schemes. Thus, while improving the computational throughput, it achieves effective enhancement of computing power and optimization of power consumption, and ultimately comprehensively improves the energy efficiency ratio and overall processing efficiency of the multiply-accumulate operation.

[0159] This application also provides a neural network processing device, which includes the aforementioned multiply-accumulate operation device. By integrating the multiply-accumulate operation device, this neural network processing device can efficiently perform calculations on core layers such as convolutional layers and fully connected layers in a neural network model, thereby significantly improving the energy efficiency and speed of model inference.

[0160] In some embodiments, the neural network processing apparatus further includes a compiler, and the weight data encoding module can reuse the compiler so that the weight encoding operation is performed when the compiler is offline, thereby further reducing the overhead of real-time hardware computation.

[0161] This application also provides an electronic device that may integrate the multiply-accumulate arithmetic unit of this application. This electronic device can achieve the following functions through the multiply-accumulate arithmetic unit:

[0162] The acquired input activation data is encoded to generate encoded activation data, and the encoded activation data is distributed to the in-memory array. Encoding the acquired input activation data to generate encoded activation data includes: splitting the input activation data into several bit units according to preset bit positions, and encoding each bit unit to generate encoded activation data.

[0163] The acquired raw weight data is split into several weight units, and the unsigned weight units in the weight units are encoded so that each weight unit is a signed number and the value range of each weight unit is unified to the target value range. The encoded weight units are then output to the weight storage units in the in-memory array.

[0164] Based on the weighted storage unit of the target row and column in the storage array, a selection shift operation is performed on the coded activation data distributed to the target row and column, thereby obtaining the multiplication result of the weighted storage unit of the target row and column and the coded activation data of the target row and column.

[0165] Add the results of multiplication in the same column of the storage array to obtain the first intermediate result of that column;

[0166] The first intermediate results of all columns in the storage array are shifted and accumulated to obtain the second intermediate result;

[0167] The compensation code is calculated based on the input activation data and the original weight data. Then, the compensation code is used to compensate the second intermediate result to obtain the final multiplicative sum of the input activation data and the original weight data.

[0168] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0169] Therefore, embodiments of this application provide a storage medium storing multiple instructions that can be loaded by a processor to execute the steps of any of the methods provided in this application. The storage medium may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0170] Since the instructions stored in the storage medium can execute the steps of any method provided in the embodiments of this application, the beneficial effects that any method provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0171] The multiplication and accumulation operation method, apparatus, neural network processing apparatus, and electronic device provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for multiplication and accumulation operations, characterized in that, include: The acquired input activation data is encoded to generate encoded activation data, and the encoded activation data is distributed to the in-memory array. The process of encoding the acquired input activation data to generate encoded activation data includes: splitting the input activation data into several bit units according to a preset number of bits, and encoding each bit unit to generate encoded activation data. The acquired raw weight data is split into several weight units, and the unsigned weight units in the several weight units are encoded so that each weight unit is a signed number and the value range of each weight unit is unified to the target value range. The encoded weight units are then output to the weight storage unit in the storage array. Based on the weighted storage unit of the target row and column in the storage array, a selection shift operation is performed on the coded activation data distributed to the target row and column, thereby obtaining the multiplication result of the weighted storage unit of the target row and column and the coded activation data of the target row and column. The multiplication results in the same column of the storage array are added together to obtain the first intermediate result of that column; The first intermediate results of all columns in the storage array are shifted and accumulated to obtain the second intermediate result; The compensation code is calculated based on the input activation data and the original weight data, and then the second intermediate result is compensated by the compensation code to obtain the final multiplication and accumulation result of the input activation data and the original weight data.

2. The multiplication and accumulation method as described in claim 1, characterized in that, The encoded activation data includes a high-order part representing positive numbers and a low-order part representing negative numbers; the step of performing a selection shift operation on the encoded activation data of the target row and column according to the weighted storage unit of the target row and column in the storage array after encoding processing includes: Obtain the value of the encoded weight unit in the weight storage unit of the target row and column in the storage array; If the value is positive or negative, select the high-order part representing a positive number or the low-order part representing a negative number in the encoded activation data as the target part; shift the target part to use the shifted result as the multiplication result, or use the selected target part as the multiplication result; If the value is 0, output 0 as the result of the multiplication.

3. The multiplication and accumulation method as described in claim 2, characterized in that, When the bit width of the weight unit after encoding is two bits, if the value is positive or negative, the high-order part representing a positive number or the low-order part representing a negative number in the encoded activation data is selected as the target part. Shifting the target portion to use the shifted result as the multiplication result, or using the selected target portion as the multiplication result, includes: If the value is 1, the high-order part representing a positive number in the encoded activation data is selected as the target part, and the target part is used as the multiplication result; If the value is -1, the lower-order part representing the negative number in the encoded activation data is selected as the target part, and the target part is used as the multiplication result; If the value is -2, the lower-order part representing the negative number in the encoded activation data is selected as the target part, and the target part is shifted left by one bit. The target part after shifting left by one bit is used as the multiplication result.

4. The multiplication and accumulation operation method as described in claim 1, characterized in that, The process of encoding each bit unit to generate encoded activation data includes: Each bit unit is sign-extended to obtain a first signed number; Calculate the opposite of the first signed number to obtain the second signed number; Take the first M bits of the first signed number, the first M bits of the second signed number, and the common least significant bit to generate encoded activation data; where M is a positive integer and is related to the preset bit position.

5. The multiplication and accumulation operation method as described in claim 4, characterized in that, The preset bit position is two bits. The step of taking the first M bits of the first signed number, the first M bits of the second signed number, and the common least significant bit to generate encoded activation data includes: The first two bits of the first signed number are taken as the first and second bits of the encoding activation data, the first two bits of the second signed number are taken as the third and fourth bits of the encoding activation data, and the least common bit of the first signed number and the second signed number is taken as the fifth bit of the encoding activation data, thereby forming the encoding activation data.

6. The multiplication and accumulation method according to any one of claims 1 to 5, characterized in that, The target value range is {0, ±W, ±2W}, where W is the basic weight value of the weight unit.

7. The multiplication and accumulation method as described in claim 6, characterized in that, The encoding process for the unsigned weight units among the plurality of weight units, so that each weight unit is a signed number and the value range of each weight unit is unified to the target value range, includes: The original weight data is split into several weight units in groups of two bits. For each unsigned weight unit with a value range of {0, 1, 2, 3}, a subtraction operation of 2 is performed to convert it into a signed weight unit, so that each weight unit is a signed number and the value range of each weight unit is unified to the target value range {-2, -1, 0, +1}.

8. The multiplication and accumulation method according to any one of claims 1 to 5, characterized in that, The step of adding the multiplication results of the same column in the storage array to obtain the first intermediate result of that column includes: Invert the highest bit of the multiplication result in each row of the storage array to generate an inverted result; The inverted result is input into an addition tree composed of multiple adders, so that the inverted results of the same column are accumulated in multiple stages in sequence to obtain the first intermediate result of each column.

9. The multiplication and accumulation method according to any one of claims 1 to 5, characterized in that, The step of calculating a compensation code based on the input activation data and the original weight data, and then compensating the second intermediate result using the compensation code to obtain the final multiplicative sum of the input activation data and the original weight data includes: The compensation parameters are determined based on the sign type and bit width of the original weight data; Calculate the sum of the values ​​of all input active data participating in the current multiply-accumulate operation in the storage array; The compensation code is calculated based on the compensation parameters and the sum, and then the second intermediate result is compensated using the compensation code to obtain the final multiplication and accumulation result of the input activation data and the original weight data.

10. A multiply-accumulate arithmetic device, characterized in that, include: An activation data encoding module is used to encode the acquired input activation data to generate encoded activation data, and to distribute the encoded activation data to the in-memory array. The step of encoding the acquired input activation data to generate encoded activation data includes: splitting the input activation data into several bit units according to a preset number of bits, and encoding each bit unit to generate encoded activation data. The weight data encoding module is used to split the acquired raw weight data into several weight units, and to encode the unsigned weight units in the several weight units so that each weight unit is a signed number and the value range of each weight unit is unified to the target value range, and to output the encoded weight units to the weight storage unit in the storage array. The in-memory array includes several weight storage units, several selection shift modules, and several addition processing modules. The weight storage units and selection shift modules are arranged in an array. The selection shift module is used to perform a selection shift operation on the encoded activation data distributed to the target row and column based on the encoded weight units in the weight storage units of the target row and column, thereby obtaining the multiplication result of the encoded weight units and the encoded activation data of the target row and column. The addition processing modules are used to perform addition processing on the multiplication results of the same column in the in-memory array to obtain the first intermediate result of that column. The shift-accumulate module is used to shift and accumulate the first intermediate result of all columns in the storage array to obtain the second intermediate result; The compensation calculation module is used to calculate a compensation code based on the input activation data and the original weight data, and then use the compensation code to compensate the second intermediate result to obtain the final multiplication and accumulation result of the input activation data and the original weight data.

11. The multiply-accumulate arithmetic device as described in claim 10, characterized in that, The encoded activation data includes a high-order bit portion representing positive numbers and a low-order bit portion representing negative numbers; the selection shift module is used for: Obtain the value of the encoded weight unit in the weight storage unit of the target row and column in the storage array; If the value is positive or negative, select the high-order part representing a positive number or the low-order part representing a negative number in the encoded activation data as the target part; The target portion is shifted so that the shifted result is used as the multiplication result, or the selected target portion is used as the multiplication result; If the value is 0, output 0 as the result of the multiplication.

12. The multiply-accumulate arithmetic device as described in claim 10, characterized in that, The target value range is {0, ±W, ±2W}, where W is the basic weight value of the weight unit.

13. The multiply-accumulate arithmetic device as described in claim 10, characterized in that, The in-memory computing array is an array based on in-memory computing.

14. A neural network processing device, characterized in that, Includes the multiply-accumulate arithmetic device as described in any one of claims 10-12.

15. The neural network processing apparatus as described in claim 14, characterized in that, The neural network processing device includes a compiler, and the weight data encoding module reuses the compiler so that the weight encoding operation is performed when the compiler is offline.

16. An electronic device, characterized in that, Includes the multiply-accumulate arithmetic device as described in any one of claims 10-12.

Images