Multiply-accumulate operation method and apparatus, neural network processing apparatus, and electronic device

Abstract

The application discloses a multiply-accumulate operation method and device, a neural network processing device and electronic equipment. According to the data types of original weight data and input activation data, the multiply-accumulate operation method converts the original weight data and the input activation data into signed weight data and signed activation data respectively, and performs Booth coding on the signed activation data to generate coded activation values. In each selection and shifting module, one of the signed weight data is selected as target data according to the coded activation values, and then shifted to obtain the multiplication result of the signed weight data and the coded activation values; the multiplication results of multiple rows in the same column are added to obtain the row accumulation result in the column; all the row accumulation results in the column are shifted and accumulated to generate the initial multiply-accumulate result, and the initial multiply-accumulate result is compensated in combination with a compensation code. The application can improve the energy efficiency ratio and efficiency of the multiply-accumulate operation while improving the throughput, optimizing the power consumption and improving the computing power.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a multiplication-accumulation operation method, apparatus, neural network processing apparatus, and electronic device. Background Technology

[0002] With the widespread application of deep learning and artificial intelligence (AI) technologies in fields such as inference, autonomous driving, and large-scale language models, unprecedented demands have been placed on the computing power and energy efficiency of underlying hardware. The core computing paradigm of modern AI involves massive multiply-accumulate (MAC) operations, and the core computing units of existing neural network processors primarily rely on MAC operations. In the traditional von Neumann architecture, the processor and memory are separated, requiring frequent data transfer during computation, resulting in significant energy consumption and access latency due to data movement—the "memory wall" problem—which severely restricts the performance and energy efficiency of AI chips. To fundamentally overcome the bottleneck of the "memory wall," in-memory computing technology has emerged. Its core concept is to complete computation within or near the memory unit, thereby minimizing data movement. Among these, digital in-memory computing architectures based on Static Random-Access Memory (SRAM) and compatible with Complementary Metal Oxide Semiconductor (CMOS) processes have become a key technological path for achieving high-energy-efficiency AI accelerators due to their high computational bandwidth and strong scalability.

[0003] In in-memory computing architectures, efficiently implementing multiply-accumulate operations is the core of the design. Early solutions often employed bit-sequential computation, multiplying and accumulating the input bit by bit with the weights. While structurally simple, completing one N-bit multiplication required N clock cycles, resulting in high computational latency and low throughput, becoming a major bottleneck for system performance. To accelerate computation, existing technologies have introduced an optimized scheme based on radix-4 Buss coding. This scheme encodes two consecutive bits of the input, thereby reducing the theoretical number of computation cycles from N to approximately N / 2, significantly improving computational throughput.

[0004] However, the optimization scheme based on Booth coding introduces new and more complex inherent problems in hardware implementation. Since the Booth coding scheme is inherently designed for signed operations, it requires an additional "zero-expansion" preprocessing cycle when processing unsigned data. The most common approach is to zero-expansion the N-bit unsigned number to N+1 bits (i.e., padding with a '0' before the most significant bit) before computation, making it appear as a positive signed number before feeding it into the encoder. This expansion and reformatting operation forcibly occupies at least one extra clock cycle, causing the theoretical cycle reduction benefit (e.g., from N cycles to N / 2 cycles) to be unrealized when processing unsigned data streams, resulting in a significant decrease in actual speedup and limited improvement in computational throughput. Furthermore, this scheme requires the hardware to integrate complex inversion, shifting, and dynamic sign extension circuits (e.g., dedicated inversion and shifting circuits integrated on or inside the memory array) to generate and process signed binary numbers for partial products. This not only increases system area and power consumption but also increases the combinational logic depth because these circuits are located on critical computation paths, thus limiting the system's operating frequency and introducing timing convergence problems and power overhead. Moreover, since negative partial products are involved, all intermediate operations must be performed in the two's complement domain, forcing the design to incorporate dynamic sign extension logic. These additional inversion, shifting, and sign extension operations significantly increase circuit area and power consumption.

[0005] Therefore, the existing technology for implementing multiply-accumulate calculations, while improving throughput, leads to increased computational power consumption when the input data is unsigned, which also affects the computing power and energy efficiency of multiply-accumulate operations, and also affects the cycle of multiply-accumulate calculations. Summary of the Invention

[0006] To address the aforementioned technical problems, embodiments of this application provide a multiplication-accumulation operation method, apparatus, neural network processing apparatus, and electronic device.

[0007] In a first aspect, embodiments of this application provide a multiplication-accumulation operation method, including:

[0008] Based on the data type of the original weight data, the corresponding signed weight data is obtained. The signed weight data includes two sets of weight data, positive and negative. The signed weight data is then broadcast to the selection shift module in the in-memory array. The in-memory array includes the selection shift module, which has m rows and n columns.

[0009] Based on the data type of the input activation data, the corresponding signed activation data is obtained, and the signed activation data is encoded by Booz, generating several encoded activation values, and the encoded activation values ​​are broadcast to the selection shift module in the in-memory array.

[0010] Based on the type of the encoded activation value received by the selection shift module, the selection shift module selects one of the received signed weight data as the target data; and performs a shift operation on the target data to obtain the product of the signed weight data and the encoded activation value.

[0011] For each of the selection shift modules in the storage array, the multiplication results of multiple rows in the same column are added to obtain the cumulative result of the row within the column;

[0012] Shift and sum all the row sums in the columns to generate an initial multiplication and summation result;

[0013] Based on the data type of the original weight data, the data type of the input activation data, and the encoded activation value, a compensation code is determined, and the initial multiply-accumulate result is compensated using the compensation code to obtain the final multiply-accumulate result.

[0014] In the multiply-accumulate operation method provided in this application embodiment, obtaining the corresponding signed weight data according to the data type of the original weight data includes:

[0015] If the original weight data is of unsigned type, then the first offset is subtracted from the original weight data to convert the unsigned original weight data into signed weight data that includes both positive and negative weight data.

[0016] If the original weight data is of the type of signed number, then the original weight data will be treated as signed weight data.

[0017] In the multiply-accumulate operation method provided in this application embodiment, the step of obtaining corresponding signed activation data according to the data type of the input activation data, and performing Booth encoding on the signed activation data to generate several encoded activation values ​​includes:

[0018] If the data type of the input activation data is an unsigned number, then the second offset is subtracted from the input activation data to obtain signed activation data;

[0019] If the data type of the input activation data is a signed number, then the input activation data will be treated as signed activation data.

[0020] The signed activation data is encoded using Booth coding to generate several encoded activation values, including both negative and non-negative numbers.

[0021] In the multiply-accumulate operation method provided in this application embodiment, the Booth encoding is Radix-4 Booth encoding. The process of encoding the signed activated data using Booth encoding to generate several encoded activation values, including negative and non-negative numbers, includes:

[0022] S1. Pad zeros after the least significant bit of the signed active data to form extended data;

[0023] S2. Starting from the least significant bit of the extended data, select Z consecutive bits of data to the left as the first encoding window;

[0024] S3. Use the most significant bit of the first encoding window as the least significant bit of the next encoding window;

[0025] S4. Continue to select Z-1 consecutive bits of data to the left from the extended data, and form a new encoding window with the least significant bit of the next encoding window;

[0026] S5. Repeat steps S3 and S4 until the bit stream of the extended data has been selected;

[0027] S6. According to the preset Booth coding rules, each coding window is mapped to a corresponding coding activation value, thereby obtaining several coding activation values ​​including negative and non-negative numbers.

[0028] In the multiply-accumulate operation method provided in this application embodiment, the step of the selection shift module selecting one of the received signed weighted data as target data according to the type of the encoded activation value received by the selection shift module includes:

[0029] When the encoded activation value received by the selection shift module is negative, the selection shift module selects negative weight data as the target data;

[0030] When the encoding activation value received by the selection shift module is non-negative, the selection shift module selects positive weight data as the target data.

[0031] In the multiplication and accumulation method provided in this application embodiment, the addition processing of the multiplication results of multiple rows in the same column to obtain the cumulative result of the row within the column includes:

[0032] The multiplication results of the same column are input into the first-stage adder for accumulation, and the highest bit of the accumulated result is inverted to obtain the intermediate result within the column;

[0033] The intermediate results within all columns are summed to obtain the cumulative result of the rows within the column.

[0034] In the multiply-accumulate operation method provided in this application embodiment, determining the compensation code based on the data type of the original weight data, the data type of the input activation data, and the encoded activation value includes:

[0035] Based on the data type of the original weight data, determine the first compensation component;

[0036] The second compensation component is determined based on the data type of the input activation data;

[0037] The third compensation component is determined based on the negative number in the encoded activation value;

[0038] The fourth compensation component is determined by inverting the highest bit of the accumulated result output by the first-stage adder.

[0039] The first compensation component, the second compensation component, the third compensation component, and the fourth compensation component are added together to obtain the compensation code.

[0040] In the multiply-accumulate operation method provided in this application embodiment, if the data type of the original weight data is an unsigned number, then the first compensation component is the product of the sum of all the input activation data and the first offset; if the data type of the original weight data is a signed number, then the first compensation component is zero.

[0041] If the data type of the input activation data is an unsigned number, then the second compensation component is the product of the sum of all the original weight data and the second offset; if the data type of the input activation data is a signed number, then the second compensation component is zero.

[0042] The third compensation component is the sum of the compensation values ​​corresponding to all negative coded activation values ​​in each calculation cycle;

[0043] The fourth compensation component is the negative of the sum of fixed offsets introduced by inverting the highest bit of the accumulated result.

[0044] In the multiply-accumulate operation method provided in the embodiments of this application, the first offset is 2^(M-1), where M is the bit width of the original weight data.

[0045] In the multiply-accumulate operation method provided in the embodiments of this application, the second offset is 2^(N-1), where N is the bit width of the input activation data.

[0046] Secondly, embodiments of this application provide a multiply-accumulate arithmetic device, including:

[0047] The weight data encoding module is used to obtain the corresponding signed weight data according to the data type of the original weight data. The signed weight data includes two sets of weight data, positive and negative, and broadcasts the signed weight data to the selection shift module in the memory array.

[0048] The activation data encoding module is used to obtain the corresponding signed activation data according to the data type of the input activation data, perform Booz coding on the signed activation data to generate several encoded activation values, and broadcast the encoded activation values ​​to the selection shift module in the in-memory array.

[0049] The in-memory array includes an m-row, n-column selection shift module and several addition processing modules. The selection shift module selects one of the received signed weight data as target data based on the type of the received encoded activation value, and performs a shift operation on the target data to obtain the product of the signed weight data and the encoded activation value. The addition processing modules perform addition processing on the product results of multiple rows in the same column for each selection shift module in the in-memory array to obtain an intra-column row accumulation result.

[0050] The shift-accumulate module is used to shift and accumulate all the row-to-row accumulation results in the column to generate an initial multiplication-accumulation result;

[0051] The compensation calculation module is used to determine the compensation code based on the data type of the original weight data, the data type of the input activation data, and the encoded activation value, and to compensate the initial multiplication-accumulation result using the compensation code to obtain the final multiplication-accumulation result.

[0052] The multiply-accumulate arithmetic device provided in this application embodiment further includes:

[0053] An activation data caching module is used to cache the signed activation data obtained by the activation data encoding module;

[0054] The weight data caching module is used to cache the signed weight data obtained by the weight data encoding module.

[0055] In the multiply-accumulate arithmetic device provided in this application embodiment, the in-memory array is an array based on in-memory computing.

[0056] In the multiply-accumulate operation device provided in this application embodiment, the weight data encoding module is used for:

[0057] When the data type of the original weight data is unsigned, the first offset is subtracted from the original weight data to convert the unsigned original weight data into signed weight data that includes both positive and negative weight data.

[0058] When the data type of the original weight data is a signed number, the original weight data is treated as signed weight data;

[0059] The signed weight data is broadcast to the selection shift module in the in-memory array.

[0060] In the multiply-accumulate arithmetic apparatus provided in this application embodiment, the activation data encoding module is used for:

[0061] When the data type of the input activation data is an unsigned number, the second offset is subtracted from the input activation data to obtain signed activation data;

[0062] When the data type of the input activation data is a signed number, the input activation data is treated as signed activation data;

[0063] The signed activation data is encoded using Booth coding to generate several encoded activation values, including negative and non-negative numbers.

[0064] The encoded activation value is broadcast to the selection shift module in the in-memory array.

[0065] Thirdly, embodiments of this application provide a neural network processing apparatus, including the multiply-accumulate operation apparatus described above.

[0066] In the neural network processing apparatus provided in this application embodiment, the neural network processing apparatus includes a compiler, and the weight data encoding module reuses the compiler so that the weight encoding operation is completed when the compiler is offline.

[0067] Fourthly, embodiments of this application provide an electronic device including the multiply-accumulate operation device described above.

[0068] In summary, the multiply-accumulate operation method provided in this application obtains signed weight data based on the data type of the original weight data, and broadcasts the signed weight data containing both positive and negative weight data to each selection shift module of the in-memory array. Simultaneously, it obtains signed activation data based on the data type of the input activation data and performs Booth encoding, broadcasting the generated encoded activation value to the selection shift module. This transforms the complex multiplication operation into a selection and shift operation based on the type of encoded activation value, greatly simplifying the circuit structure and logic depth of the core computing unit, thereby directly reducing the energy consumption of each operation. Furthermore, for each selection shift module in the in-memory array, the results of multiplying multiple rows in the same column are added to obtain the column-level row accumulation result, and all column-level row accumulation results are shifted and accumulated to generate the initial multiply-accumulate result. This parallel computing and fusion architecture fully leverages spatial parallelism, significantly increasing the amount of computation that can be completed per unit time, effectively improving computing power. At the same time, through careful design of the computation process, it reduces dependence on complex computing units and unnecessary intermediate data handling, further optimizing computational power consumption. Finally, a compensation code is determined based on the data type of the original weight data, the data type of the input activation data, and the encoded activation value. This compensation code is then used to compensate for the initial multiply-accumulate result, yielding the final multiply-accumulate result. This compensation mechanism, with a very low additional computational overhead, ensures that the final multiply-accumulate result maintains high accuracy even after simplifying the computation process and hardware design. This allows for the achievement of high computing power and low power consumption advantages without sacrificing computational accuracy, thereby fundamentally improving the energy efficiency ratio of multiply-accumulate operations. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0070] Figure 1 This is a flowchart illustrating the multiplication and accumulation operation method provided in the embodiments of this application.

[0071] Figure 2 This is a schematic diagram of the process of performing Booth coding on signed activation data according to an embodiment of this application.

[0072] Figure 3 This is a schematic diagram of the structure of a storage array of 32 groups of 64 rows and 8 columns provided in the embodiments of this application.

[0073] Figure 4 This is a schematic diagram of the structure of the multiply-accumulate arithmetic device provided in the embodiments of this application.

[0074] Figure 5This is another schematic diagram of the structure of the in-memory array provided in the embodiments of this application.

[0075] Figure 6 This is another schematic diagram of the multiply-accumulate operation device provided in the embodiments of this application. Detailed Implementation

[0076] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of systems and methods consistent with some aspects of this application as detailed in the appended claims.

[0077] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0078] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0079] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0080] In the description of this application, it should be noted that the terms "upper," "lower," "left," "right," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and for simplifying the description, and do not indicate or imply that the system or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. In addition, terms such as "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0081] This application provides a multiplication-accumulation operation method, apparatus, neural network processing apparatus, and electronic device. Specifically, the multiplication-accumulation operation apparatus can be integrated into an electronic device, which can be a server or a terminal, etc. The terminal can include mobile phones, wearable smart devices, tablet computers, laptops, and personal computers (PCs), etc. The server can be a single server or a server cluster composed of multiple servers, and can be a physical server or a virtual server.

[0082] The technical solutions shown in this application will be described in detail below through specific embodiments. It should be noted that the order of description of the following embodiments is not intended to limit the priority of the embodiments.

[0083] Please see Figure 1 , Figure 1 This is a flowchart illustrating the multiplication-accumulation operation method provided in an embodiment of this application. The specific flow of this multiplication-accumulation operation method can be as follows:

[0084] 101. Based on the data type of the original weight data, obtain the corresponding signed weight data, which includes two sets of weight data, positive and negative, and broadcast the signed weight data to the selection shift module in the memory array.

[0085] The original weight data can be the model parameters obtained after training. The original weight data can have different bit widths (such as 8 bits or 16 bits) and sign types (signed or unsigned).

[0086] In practical implementation, if the original weight data is a signed number, it indicates that the original weight data itself is represented in two's complement form, containing positive and negative values ​​(i.e., it has two sets of weight data, positive and negative). In this case, no numerical conversion is required, and the original weight data can be directly used as signed weight data. If the original weight data is an unsigned number, it indicates that the numerical range of the original weight data is within [0, 2^M - 1], where M is the weight bit width (e.g., M=8, i.e., W8). In some embodiments, to make it suitable for subsequent processing, it needs to be converted to a signed two's complement form with the same bit width, that is, the numerical range is converted from the original [0, 255] to [-2^(M-1), 2^(M-1)-1].

[0087] In some embodiments, the above numerical conversion method can be performed offline by the compiler during the model compilation stage. In this case, the numerical range conversion method can be: subtract a first offset from the original weight data, where the first offset is 2^(M-1). For example, when M=8, the first offset is 128. Subtracting 128 from the unsigned number 123 (binary '01111011') yields the signed number -5 (two's complement representation '11111011').

[0088] In another embodiment, the above-described numerical conversion method can be performed online by hardware circuitry when writing the original weight data into the memory array. Specifically, the most significant bit (MSB) of the binary representation of the unsigned original weight data can be inverted. In two's complement representation, this operation is numerically equivalent to subtracting 2^(M-1). For example, when the weight bit width is 8 bits, the binary representation of the unsigned number 123 is 01111011. After inverting its MSB, it becomes 11111011, corresponding to -5 in the signed two's complement representation. For example, when the weight bit width is 8 bits, the binary representation of the unsigned number 230 is 11100110. After inverting its MSB, it becomes 01100110, corresponding to 102 in the signed two's complement representation.

[0089] That is, step 102 can be: if the data type of the original weight data is an unsigned number, then the first offset is subtracted from the original weight data so that the unsigned original weight data is converted into signed weight data including both positive and negative weight data; if the data type of the original weight data is a signed number, then the original weight data is used as signed weight data.

[0090] It is understandable that, regardless of the numerical conversion method used, the resulting data will be signed weight data including both positive and negative weights. After obtaining the signed weight data in the above manner, this signed weight data can be broadcast to the selection shift module in the in-memory array.

[0091] 102. Based on the data type of the input activation data, obtain the corresponding signed activation data, perform Booth encoding on the signed activation data to generate several encoded activation values, and broadcast the encoded activation values ​​to the selection shift module in the memory array.

[0092] In this embodiment, the step "obtaining the corresponding signed activation data according to the data type of the input activation data" is similar in logic to step 101. That is, if the data type of the input activation data is an unsigned number, the second offset is subtracted from the input activation data to obtain the signed activation data; if the data type of the input activation data is a signed number, the input activation data is used as the signed activation data.

[0093] The second offset is 2^(N-1), where N is the activation bit width (e.g., N=8). After obtaining the signed activation data, Booth coding can be performed on the signed activation data to generate several encoded activation values, including negative and non-negative numbers.

[0094] In some embodiments, the Booth encoding is Radix-4 Booth encoding, as shown in Table 1 below, which is a preset Booth encoding rule table. Radix-4 Booth encoding can map a continuous encoding window to encoded values ​​in the set {-2, -1, 0, 1, 2}. Figure 2 As shown, the specific encoding process includes (for example, taking an activation bit width of N=8 (i.e., A8) as an example):

[0095] S1. Add zeros after the least significant bit of the signed active data to form extended data.

[0096] For example, for an 8-bit (N=8) signed activation data, a 0 is added after its least significant bit (LSB) to form a 9-bit extended data.

[0097] S2. Starting from the least significant bit of the extended data, select Z consecutive bits of data to the left as the first encoding window.

[0098] For example, starting from the least significant bit of the 9-bit extended data (i.e. the bit where the padded 0 is located), select consecutive Z bits of data to the left (i.e. the high bit direction) as the first encoding window. The first encoding window is taken from bits [2:0].

[0099] S3. Use the most significant bit of the first encoding window as the least significant bit of the next encoding window.

[0100] In this embodiment, after obtaining the first encoding window, the most significant bit of the first encoding window can be used as the least significant bit of each subsequent encoding window.

[0101] S4. Continue to select M-1 consecutive bits of data to the left from the extended data, and combine them with the least significant bit of the next encoding window to form a new encoding window.

[0102] Specifically, data selection can continue from the position where data was previously selected from the extended data. That is, the position after the end position of the previous data selection is taken as the starting position of the new data selection, and M-1 consecutive bits of data are selected from the extended data, thus forming a new encoding window with the least significant bit of the next encoding window mentioned in S3.

[0103] S5. Repeat steps S3 and S4 until the bit stream of extended data has been selected.

[0104] Then, repeat steps S3 and S4 until the bit stream of extended data is selected, resulting in several consecutive encoding windows.

[0105] For example, an 8-bit signed active data can be divided into four 3-bit data segments.

[0106] It should be noted that the length of the encoding window is related to the radix characteristic of the Radix-4 Buses coding used. For signed active data with a bit width of N bits, after padding the least significant bit with zeros in step 1021, it becomes N+1 bits. Then, by selecting the bit data as described above, the following can be generated: N / 2 Each encoding window is independently mapped to an encoding activation value. This process decomposes an N-bit multiplication operation into approximately N / 2 suboperations based on simple encoded values, thereby improving computational throughput.

[0107] S6. According to the preset Booth coding rules, each coding window is mapped to the corresponding coding activation value, thereby obtaining several coding activation values ​​including negative and non-negative numbers.

[0108] For example, each encoding window can be mapped to a corresponding encoding activation value according to the preset Booth encoding rules in Table 1. This encoding activation value is a two-bit signed number with a value range of {-2, -1, 0, 1, 2}.

[0109]

[0110] Table 1

[0111] In Table 1, (Bi+1, Bi, Bi-1) refers to a coding window captured during Booth coding. The Booth coding value is the coding activation value.

[0112] For example, suppose the 8-bit signed activation data is 01101110, whose decimal value is 110. First, add a 0 to the end to obtain 9-bit extended data 011011100. Then, take 3 bits of data starting from the LSB of the extended data as the first encoding window, and use the MSB of the first encoding window as the LSB of the subsequent encoding windows. Then, continue to select consecutive 2 bits of data from the extended data and combine them with the MSB of the first encoding window to form the current encoding window, thus obtaining four 3-bit encoding windows: 100, 111, 101, and 011. According to the preset Booth coding rules in Table 1, 011 maps to 2, 101 maps to -1, 111 maps to 0, and 100 maps to -2. Therefore, the four generated encoding activation values ​​are: {2, -1, 0, -2}.

[0113] Furthermore, the decimal values ​​of the four encoded activation values ​​{2, -1, 0, -2} are: 2*2^6+(-1)*2^4+0*2^2+(-2)*2^0=128-16+0-2=110, which is consistent with the original decimal values. Therefore, Booth encoding itself does not change the numerical value of the activation value. Thus, for Booth encoding directly derived from signed activation data, the process is numerically equivalent, and no compensation code is needed for correction.

[0114] 103. Based on the type of the encoded activation value received by the selection shift module, the selection shift module selects one of the received signed weighted data as the target data; and performs a shift operation on the target data to obtain the result of multiplying the signed weighted data and the encoded activation value.

[0115] This embodiment uses Figure 3 Taking a storage array with 64 rows, 8 columns, and 32 groups as an example, the specific process is as follows:

[0116] Within each selection shift module, for each encoded activation value, one of the received signed weighted data can be selected as the target data based on the type of the received encoded activation value. Specifically, when the encoded activation value received by the selection shift module is negative, the selection shift module selects negative weighted data as the target data; when the encoded activation value received by the selection shift module is non-negative (i.e., zero or positive), the selection shift module selects positive weighted data as the target data.

[0117] like Figure 3 As shown, each selection shift module in the in-memory array can receive broadcast signed weight data, including two sets of weight data (positive and negative), namely positive weight data (W8c) and negative weight data (W8Cb), through a selector (MUX). By controlling the selector according to the type of the encoded activation value (i.e., negative or non-negative), the selection of target data can be completed.

[0118] It should be noted that in standard two's complement arithmetic, multiplying weight data by -1 requires two steps: "inverting" and "adding one". However, in this embodiment, when the encoded activation value is negative, the "inverting" operation can be achieved simply by selecting the negative weight data using a selector in the shift module within the in-memory array. The "adding one" operation is separated and left for subsequent correction using compensation codes. This eliminates the need for a dedicated adder for each negative weight multiplication on the core computation path of the in-memory array, significantly saving circuit area and dynamic power consumption. Furthermore, the selection operation itself is more efficient than arithmetic operations.

[0119] Then, the selected target data can be shifted based on the absolute value of the encoded activation value to perform a multiplication operation between the signed weighted data and the encoded activation value, generating a partial product.

[0120] In some embodiments, the selected target data can be shifted according to the absolute value of the encoded activation value to achieve the effect of "multiplying by 1" or "multiplying by 2", thereby obtaining the result of multiplying the signed weighted data with the encoded activation value (i.e., partial product).

[0121] It should be noted that this shift operation includes both shifting and non-shifting. For example, when the absolute value is 2, the target data is shifted left by one bit (equivalent to multiplying by 2); when the absolute value is 1 or 0, the target data is not shifted (equivalent to multiplying by 1 or 0). Since the absolute value is limited to only three cases: 0, 1, and 2, the target data can be shifted to achieve the desired "multiply by 1" or "multiply by 2" operation without calling any dedicated multiplier or programmable shifter circuitry. This routing-based shift mechanism not only saves valuable chip area and power consumption but also shortens the critical path latency by avoiding complex arithmetic units, thus providing a foundation for achieving high-efficiency, highly parallel multiply-accumulate calculations in in-memory computing architectures.

[0122] 104. For each selection shift module in the storage array, perform addition on the results of multiplication of multiple rows in the same column to obtain the cumulative result of the row within the column.

[0123] Specifically, the multiplication results of the same column can be input into the first-level adder for accumulation, and the highest bit of the accumulated result can be inverted to obtain the intermediate result within the column; then all intermediate results within the column are accumulated to obtain the cumulative result of the row within the column (mac_results).

[0124] Taking the processing of an 8-bit input activation data as an example, the four encoded activation values ​​obtained after Radix-4 Buss coding need to be processed in four calculation cycles. For example... Figure 3 As shown, for each selection shift module in the in-memory array, the multiplication result of the same column in each calculation cycle can be input to the first-stage adder (adder 1). The multiplication results of each adjacent pair of rows in that column are accumulated to obtain a cumulative result. For example, the multiplication result of row 0 and the multiplication result of row 1 are added through the first-stage adder to obtain the first cumulative result; then the multiplication result of row 2 and the multiplication result of row 3 are added through the adder to obtain the second cumulative result, and so on. The cumulative result output by the first-stage adder is usually a 12-bit signed number.

[0125] Next, the most significant bit of each accumulated result is inverted. This inversion is numerically equivalent to adding a fixed value (e.g., 1024) to each accumulated result, thus converting it from signed two's complement representation to unsigned representation. After this conversion, all subsequent accumulation operations can be performed in the unsigned number field, completely eliminating the need for dynamic sign bit extension, simplifying circuit design, and improving timing performance.

[0126] The output of this step is the intermediate result within the column. Subsequently, all intermediate results within the column are accumulated using the corresponding second-level adder (adder 2), and the outputs of the second-level adder are then accumulated using an addition tree to obtain the cumulative row summation result within the column for each calculation cycle. Taking the processing of an 8-bit input activation data as an example, this embodiment can obtain the cumulative row summation result within the column for 4 calculation cycles.

[0127] 105. Shift and sum all the row sums within each column to generate the initial multiplication and summation result.

[0128] Since complete input activation data requires multiple computation cycles to process (e.g., 8-bit input activation data requires 4 computation cycles), multiple in-column row accumulation results corresponding to the number of computation cycles will be obtained. Specifically, the in-column row accumulation results obtained from all computation cycles can be input into the shift accumulation module for cross-cycle accumulation. In this shift accumulation module, the in-column row accumulation results corresponding to different computation cycles are shifted and aligned according to the bit weight of the processed encoded activation value, and then accumulated to finally generate the initial multiplication accumulation result.

[0129] Because Radix-4 Buses coding is used, an input activation data with a bit width of N is encoded as K = N / 2 K consecutive encoded activation values ​​are processed sequentially over K consecutive computation cycles. Each encoded activation value corresponds to a specific 2 bits in the original activation data, so the cycle number k (counting from 0 or 1) directly determines the bit weight of the column-to-row accumulation result in the initial multiplication-accumulation result for that computation cycle.

[0130] Specifically, the encoded activation value processed in the k-th calculation cycle corresponds to the (2k)th and (2k+1)th bits from low to high in the original activation data. Therefore, the in-column row accumulation result obtained in this calculation cycle should actually be multiplied by 2 during the final accumulation. 2k (That is, shift left by 2k bits) to ensure correct alignment with the results of other calculation cycles. For example, the cumulative result of the column in cycle 0 is not shifted, the cumulative result of the column in cycle 1 is shifted left by 2 bits, the cumulative result of the column in cycle 2 is shifted left by 4 bits, and so on.

[0131] This embodiment utilizes a shift-accumulation module to perform cross-cycle fusion of column-to-row accumulation results across multiple computation cycles, achieving efficient and accurate complete multiplication-accumulation and reconstruction. Based on the inherent bit weight relationship of the encoded activation values, this shift-accumulation module performs regular shifting and alignment of the column-to-row accumulation results generated in different computation cycles before accumulation, ensuring the numerical correctness of the final generated initial multiplication-accumulation result. This embodiment can be implemented using simple, fixed shift control logic, eliminating the need for complex dynamic scheduling or arithmetic units, significantly simplifying hardware design and reducing circuit area and power consumption. Simultaneously, this shift-accumulation module, as a crucial link connecting cycle-based parallel computation and final result output, ensures the computational accuracy and throughput of the in-memory computing architecture in highly parallel, pipelined processing.

[0132] In this embodiment of the application, the in-memory array includes g groups of m rows and n columns of selection shift modules. Figure 3 In the configuration described above, the storage array can be specifically configured to include 64 selection shift modules, each of which performs 64 rows of accumulation, with a total of 8 columns of weight calculation channels.

[0133] The original weight data for each column (i.e., each W8(n), where n is the column index, 0≤n<8) is broadcast to the corresponding column selection shift module in all 64 selection shift modules.

[0134] For each selection shift module's input activation data A8(g) (where g is the selection shift module group index, 0 ≤ g < 64), it first undergoes the aforementioned encoding process, converting it into multiple encoded activation values. These encoded activation values ​​are broadcast to all 8 columns of selection shift modules within that group, serving as input for the multiplication operation.

[0135] Under the aforementioned data broadcasting and parallel computing mechanism, the in-memory array can perform massively parallel operations within each computation cycle. Specifically, in each computation cycle, the in-memory array processes an encoded activation value obtained from the input activation data. After four clock cycles, the in-memory array has cumulatively performed 64 (number of groups of selection shift modules) × 8 (number of columns of weight calculation channels) = 512 basic "encoded activation value × signed weight data" multiplication operations in parallel. The multiplication results generated by these operations are then accumulated and inverted within multiple rows in the column, and then accumulated through an addition tree to obtain the row-level accumulation result within the column. Finally, through shift accumulation, 8 columns × 64 groups = 512 independent initial multiplication accumulation results are output, where each initial multiplication accumulation result corresponds to the multiplication accumulation sum of 64 rows of original data.

[0136] Each initial multiplication-accumulation result corresponds to a specific weight calculation channel n and a specific selection shift module g. The corresponding final multiplication-accumulation result... The calculation formula is as follows:

[0137] ;

[0138] Where k ranges from 0 to 63, representing the accumulation of 64 rows of original data, A8 k (g) and W8 k (n) represents the input activation data and original weight data of the g-th group and the k-th row, respectively.

[0139] As can be seen, this in-memory array works in conjunction with extremely high spatial parallelism (64 groups × 8 columns) and temporal parallelism (Radix-4 Booth coding), efficiently producing 512 initial multiply-accumulate results per computation batch (e.g., 4 cycles), thus greatly improving computational throughput. These initial multiply-accumulate results are then combined with compensation codes determined in subsequent steps to obtain accurate final multiply-accumulate results.

[0140] 106. Based on the data type of the original weight data, the data type of the input activation data, and the encoded activation value, determine the compensation code, and use the compensation code to compensate the initial multiplication and accumulation result to obtain the final multiplication and accumulation result.

[0141] Specifically, the first compensation component can be determined based on the data type of the original weight data; the second compensation component can be determined based on the data type of the input activation data; the third compensation component can be determined based on the negative number in the encoded activation value; the fourth compensation component can be determined based on the inversion operation performed on the highest bit of the accumulated result output by the first-stage adder; the first compensation component, the second compensation component, the third compensation component and the fourth compensation component are added together to obtain the compensation code.

[0142] In some embodiments, the compensation code can be determined based on the following principle: Let the theoretical result obtained by directly performing standard multiply-accumulate operations on the original weight data and the original input activation data be the final multiply-accumulate result, and the result processed by the aforementioned steps in the embodiments of this application and output by the in-memory array be the initial multiply-accumulate result. The relationship between the two satisfies: Final multiply-accumulate result = Initial multiply-accumulate result + Compensation code. Therefore, Compensation code = Final multiply-accumulate result - Initial multiply-accumulate result. Based on this relationship, the compensation component corresponding to the deviation introduced by various transformations and simplification operations can be derived.

[0143] by Figure 3 Taking the in-memory array shown as an example, the calculation method for each compensation component is as follows:

[0144] The first compensation component is used to compensate for the bias introduced when the original weighted data is unsigned, as it is converted to signed weighted data. Its calculation formula is as follows:

[0145]

[0146] It is understandable that the final result of multiplication and summation is... After the original weight data is transformed, the initial multiplication and accumulation result is... Therefore, the first compensation component The settlement results are shown below:

[0147]

[0148] In the above formula, g is the group index of the selected shift module. In this embodiment, the memory array contains 64 selected shift modules, therefore the value of g ranges from 0 to g < 64. A8 k (g) represents the input activation data of the g-th group selecting the shift module and the k-th row. 128 is the first offset, with a value of 2. (M 1) Where M is the bit width of the original weight data. In this embodiment, M=8, so the first offset is 2. (8 1) =128.

[0149] Based on the above formula, when the original weight data is unsigned, converting it to signed weight data requires subtracting the first offset (128). Therefore, in the multiply-accumulate operation, the product of the signed weight data and the encoded activation value is relative to the product of the original weight data and the input activation data. Each row will introduce... A8 k The deviation is (g)×128. After summing over 64 rows, the total deviation is - To compensate for this deviation, the first compensation component takes the negative of the total deviation, i.e. If the original weight data is a signed number, no conversion is needed, and the first compensation component is zero.

[0150] That is, for each selection shift module, if the data type of the original weight data is an unsigned number, then the first compensation component is the product of the sum of all input activation data and the first offset; if the data type of the original weight data is a signed number, then the first compensation component is zero.

[0151] The second compensation component is used to compensate for the deviation introduced when the input activation data is an unsigned number, due to its conversion to signed activation data. Its calculation formula is as follows:

[0152]

[0153] Similarly, each second compensation component The calculation results can be seen as follows:

[0154] ;

[0155] In the above formula, n is the index of the weight calculation channel. In this embodiment, the in-memory array has 8 columns of weight calculation channels, so the value of n ranges from 0 to n < 8. k is the row index, indicating that 64 rows of data are accumulated, and k ranges from 0 to 63. W8 k (n) represents the original weight data in the nth column and kth row. 128 is the second offset, with a value of 2. (N 1) Where N is the bit width of the input activation data. In this embodiment, N=8, so the second offset is 2. (8 1) =128.

[0156] Based on the above formula, when the input activation data is an unsigned number, converting it to signed activation data requires subtracting the second offset (128). Therefore, in the multiply-accumulate operation, the product of the converted encoded activation value and the signed weight data is relative to the product of the input activation data and the original weight data. Each row will introduce... W8 k The deviation is (n)×128. After summing over 64 rows, the total deviation is - To compensate for this deviation, the second compensation component takes the negative of the total deviation, i.e. If the input activation data is a signed number, no conversion is needed, and the second compensation component is zero.

[0157] That is, if the data type of the input activation data is an unsigned number, then the product of the sum of all the original weight data of the second compensation component and the second offset is used; if the data type of the input activation data is a signed number, then the second compensation component is zero.

[0158] The third compensation component is used to compensate for the deviation introduced by the "add one" operation in the two's complement arithmetic when the encoded activation value is negative, by only inverting the signed weight data. Therefore, it is also necessary to compensate for the absolute value of each negative encoded activation value, i.e., the third compensation component. The formula for calculating it is as follows:

[0159]

[0160] In the above formula, g is the group index for selecting the shift module. In this embodiment, the in-memory array contains 64 groups of selection shift modules, so the value of g ranges from 0 to g < 64. cy0, cy1, cy2, and cy3 represent the 0th, 1st, 2nd, and 3rd calculation cycles, respectively. k is the row index, indicating that 64 rows of data are accumulated, and k ranges from 0 to 63. A2 k,cy0 (g), A2 k,cy1 (g), A2k,cy2 (g), A2 k,cy3 (g) represents the encoded activation value processed in the g-th group of the shift module, the k-th row, and the 0th, 1st, 2nd, and 3rd calculation cycles. Its value is one of the values ​​in the set {-2, -1, 0, +1, +2}. `neg[·]` is the absolute value extraction function. It is defined as follows: if A2 is negative, it returns its absolute value (i.e., |A2|, which may be 1 or 2); if A2 is non-negative (0 or positive), it returns 0. 0 2 2 2 4 2 6 These are the bit weight coefficients, corresponding to the bit positions of the encoded activation values ​​processed in the original data during the 0th, 1st, 2nd, and 3rd calculation cycles, respectively.

[0161] That is, the third compensation component is the sum of the compensation values ​​corresponding to all negative coded activation values ​​in each calculation cycle.

[0162] The fourth compensation component is used to compensate for the fixed offset (e.g., 1024) introduced by inverting the most significant bit of the accumulated result output from the first-stage adder. For example, ... Figure 3 Regarding the storage array shown, the fourth compensation component The calculation formula is as follows:

[0163] ;

[0164] In the above formula, k is the row index, indicating that the summation is performed by traversing the rows in the memory array. In this embodiment, the memory array contains 64 rows, so k is from 0 to 63. 1024 is the single-row single-cycle offset (Δ). This is the inversion of the most significant bit (MSB) of each accumulated result output by the first-stage adder, which is numerically equivalent to adding a fixed value to each accumulated result.

[0165] For any given row, in each of the four computation cycles, an accumulated result is generated and the MSB is inverted. Therefore, the total positive offset introduced into this row during the entire computation process is 1024 × 85 = 87040. Total offset for all rows: The storage array has 64 rows, so the total positive offset introduced into the initial multiply-accumulate result is 64 × 87040 = 5570560. To correct (i.e. subtract) this offset added to the initial multiply-accumulate result, the fourth compensation component must be its opposite, i.e., -5570560. That is, the fourth compensation component is the opposite of the sum of the fixed offsets introduced by inverting the most significant bit of the accumulated result.

[0166] It is understandable that this fourth compensation component is independent of the specific input data, depending only on the array size (number of rows), the single offset value (Δ=1024), the number of processing cycles, and its bit weights. Therefore, this fourth compensation component can be calculated and stored once during the chip design phase or system initialization, and applied to the entire calculation batch with extremely low overhead during the compensation correction phase.

[0167] Finally, the compensation code used to correct a specific initial multiplication-accumulation result can be obtained by algebraically combining the aforementioned related compensation components according to its corresponding original data type. For example, when both the original weight data and the original input activation data in the processed task are unsigned numbers, the compensation code is: first compensation component + second compensation component + third compensation component + fourth compensation component.

[0168] After obtaining the initial multiply-accumulate result and the corresponding compensation code, the compensation operation can be performed. Specifically, the initial multiply-accumulate result and the compensation code can be algebraically added to eliminate systematic biases introduced by data format conversion, calculation simplification, and symbol processing, thus obtaining an accurate final multiply-accumulate result. This final multiply-accumulate result is numerically equivalent to the theoretical result obtained by directly using the original weight data and input activation data to perform standard multiply-accumulate operations, thereby ensuring the absolute accuracy of the final multiply-accumulate result while guaranteeing extremely high computational throughput and energy efficiency.

[0169] Furthermore, in practical implementation, introducing compensation code calculation does indeed increase additional circuit area and power consumption. However, through reasonable design of the computational scale, this overhead is negligible and manageable relative to the entire multiply-accumulate operation system.

[0170] Therefore, the initial multiplication and accumulation result of the output can be corrected by adding corresponding compensation codes according to different calculation modes. The compensation code circuit introduces additional area and power consumption overhead. However, in the in-memory computing architecture of this scheme, with... Figure 3 Taking the in-memory array shown as an example, its core computing section is enormous, containing 64 sets of selection and shift modules. Each module performs 64 rows of accumulation, and the entire column has 8 weight calculation channels. When completing a complete computing task (processing 64 sets of 8-bit input activation data), the actual amount of target data executed by the core computing section is extremely large: a total of 64 (rows) × 8 (columns) × 64 (sets) × 4 (cycles) = 131,072 basic multiplication operations and 8 (columns) × 64 (sets) × 4 (cycles) = 2048 accumulations across 64 rows are completed.

[0171] In contrast, the computational load required for compensation code calculation is much smaller. Its main tasks can be reduced to accumulating the partial product of the original input activation data and the original weight data and multiplying it by a fixed offset, as well as performing statistical weighting on negative encoded activation values. Figure 3 The configuration shown is estimated to have an equivalent computational load of approximately n + 2 × g = 8 + 2 × 64 = 136 times of accumulation over 64 rows of data.

[0172] Evaluations show that the area and power consumption overhead of compensation code calculation accounts for only about 3% to 5% of the total overhead of the entire in-memory computing system. This proportion is acceptable in typical large-scale integrated circuit designs. Therefore, the embodiments of this application can, while utilizing the same or similar computing hardware resources as traditional bit-serial computing (A1W8), increase the encoded activation value processed in a single computing cycle from 1 bit to multiple bits through encoding and compensation mechanisms. This significantly reduces the total number of clock cycles required to complete the same bit-width multiplication and accumulation operation proportionally to the encoding base, thereby greatly improving the theoretical computing throughput at the same clock frequency and significantly reducing the dynamic power consumption caused by data handling and computing cycles. Even considering the small overhead of compensation code calculation, this solution still achieves significant performance improvement and energy efficiency optimization at the system level, demonstrating outstanding overall advantages.

[0173] In summary, the multiply-accumulate operation method provided in this application obtains signed weight data based on the data type of the original weight data, and broadcasts the signed weight data containing both positive and negative weight data to each selection shift module of the in-memory array. Simultaneously, it obtains signed activation data based on the data type of the input activation data and performs Booth encoding, broadcasting the generated encoded activation value to the selection shift module. This transforms the complex multiplication operation into a selection and shift operation based on the type of encoded activation value, greatly simplifying the circuit structure and logic depth of the core computing unit, thereby directly reducing the energy consumption of each operation. Furthermore, for each selection shift module in the in-memory array, the results of multiplying multiple rows in the same column are added to obtain the column-level row accumulation result, and all column-level row accumulation results are shifted and accumulated to generate the initial multiply-accumulate result. This parallel computing and fusion architecture fully leverages spatial parallelism, significantly increasing the amount of computation that can be completed per unit time, effectively improving computing power. At the same time, through careful design of the computation process, it reduces dependence on complex computing units and unnecessary intermediate data handling, further optimizing computational power consumption. Finally, a compensation code is determined based on the data type of the original weight data, the data type of the input activation data, and the encoded activation value. This compensation code is then used to compensate for the initial multiply-accumulate result, yielding the final multiply-accumulate result. This compensation mechanism, with a very low additional computational overhead, ensures that the final multiply-accumulate result maintains high accuracy even after simplifying the computation process and hardware design. This allows for the achievement of high computing power and low power consumption advantages without sacrificing computational accuracy, thereby fundamentally improving the energy efficiency ratio of multiply-accumulate operations.

[0174] It should be noted that the multiply-accumulate operation method provided in this application can be applied to various scenarios that require multiply-accumulate operations. It is particularly suitable for digital in-memory computing systems with stringent requirements for energy efficiency and throughput, such as artificial intelligence inference acceleration chips, high-performance digital signal processors (DSPs), neural network processing units (NPUs) in mobile devices, and customized acceleration cores in edge computing devices.

[0175] To facilitate better implementation of the multiplication-accumulation operation method provided in the embodiments of this application, the embodiments of this application also provide a multiplication-accumulation operation device. The meanings of the terms used are the same as in the multiplication-accumulation operation method described above, and specific implementation details can be found in the descriptions in the method embodiments.

[0176] Please see Figures 4-5 , Figure 4 This is a schematic diagram of the structure of the multiplication and accumulation operation method system provided in the embodiments of this application. Figure 5 This is a schematic diagram of the structure of the in-memory array provided in an embodiment of this application. The multiply-accumulate operation method system may include a weight data encoding module 201, an activation data encoding module 202, an in-memory array 203, a shift-accumulate module 204, and a compensation calculation module 205. Among them,

[0177] The weight data encoding module 201 is used to obtain the corresponding signed weight data according to the data type of the original weight data. The signed weight data includes two sets of weight data, positive and negative, and broadcasts the signed weight data to the selection shift module in the storage array.

[0178] The activation data encoding module 202 is used to obtain the corresponding signed activation data according to the data type of the input activation data, perform Booz encoding on the signed activation data to generate several encoded activation values, and broadcast the encoded activation values ​​to the selection shift module in the memory array.

[0179] The in-memory array 203 includes an m-row, n-column selection and shift module 2031 and several addition processing modules 2032. The selection and shift module 2031 selects one of the received signed weight data as the target data based on the type of the received encoded activation value, and performs a shift operation on the target data to obtain the product of the signed weight data and the encoded activation value. The addition processing modules 2032 perform addition processing on the multiplication results of multiple rows in the same column for each selection and shift module 2031 in the in-memory array 203, obtaining the cumulative result of the row within the column.

[0180] The shift-accumulation module 204 shifts and accumulates all the row-to-row accumulation results in the column to generate the initial multiplication-accumulation result;

[0181] The compensation calculation module 205 is used to determine the compensation code based on the data type of the original weight data, the data type of the input activation data, and the encoded activation value, and to compensate the initial multiplication and accumulation result using the compensation code to obtain the final multiplication and accumulation result.

[0182] In some embodiments, such as Figure 6 As shown, the multiply-accumulate operation device further includes an activation data cache module 206 and / or a weight data cache module 207. The activation data cache module 206 is used to cache the signed activation data obtained by the activation data encoding module; the weight data cache module 207 is used to cache the signed weight data obtained by the weight data encoding module.

[0183] In this embodiment of the application, the in-memory computing array is an array based on in-memory computing.

[0184] In some embodiments, the weight data encoding module 201 is specifically used to: when the data type of the original weight data is an unsigned number, subtract a first offset from the original weight data to convert the unsigned original weight data into signed weight data including both positive and negative weight data; when the data type of the original weight data is a signed number, use the original weight data as signed weight data; and broadcast the signed weight data to the selection shift module in the memory array.

[0185] In some embodiments, the activation data encoding module 202 is specifically used for: when the data type of the input activation data is an unsigned number, subtracting a second offset from the input activation data to obtain signed activation data; when the data type of the input activation data is a signed number, using the input activation data as signed activation data; performing Booth encoding on the signed activation data to generate several encoded activation values ​​including negative and non-negative numbers; and broadcasting the encoded activation values ​​to the selection shift module in the memory array.

[0186] For specific implementation methods of each of the above units, please refer to the embodiments of the multiplication and accumulation operation method described above, which will not be repeated here.

[0187] In summary, the multiply-accumulate operation device provided in this application obtains signed weight data based on the data type of the original weight data, and broadcasts the signed weight data containing both positive and negative weight data to each selection shift module of the in-memory array. Simultaneously, it obtains signed activation data based on the data type of the input activation data and performs Booth encoding, broadcasting the generated encoded activation value to the selection shift module. This transforms the complex multiplication operation into a selection and shift operation based on the type of encoded activation value, greatly simplifying the circuit structure and logic depth of the core computing unit, thereby directly reducing the energy consumption of each operation. Furthermore, for each selection shift module in the in-memory array, the results of multiplying multiple rows in the same column are added to obtain the column-level row accumulation result, and all column-level row accumulation results are shifted and accumulated to generate the initial multiply-accumulate result. This parallel computing and fusion architecture fully leverages spatial parallelism, significantly increasing the amount of computation that can be completed per unit time, effectively improving computing power. At the same time, through careful design of the computation process, it reduces dependence on complex computing units and unnecessary intermediate data handling, further optimizing computational power consumption. Finally, a compensation code is determined based on the data type of the original weight data, the data type of the input activation data, and the encoded activation value. This compensation code is then used to compensate for the initial multiply-accumulate result, yielding the final multiply-accumulate result. This compensation mechanism, with a very low additional computational overhead, ensures that the final multiply-accumulate result maintains high accuracy even after simplifying the computation process and hardware design. This allows for the achievement of high computing power and low power consumption advantages without sacrificing computational accuracy, thereby fundamentally improving the energy efficiency ratio of multiply-accumulate operations.

[0188] This application also provides a neural network processing device, which includes the aforementioned multiply-accumulate operation device. By integrating the multiply-accumulate operation device, this neural network processing device can efficiently perform calculations on core layers such as convolutional layers and fully connected layers in a neural network model, thereby significantly improving the energy efficiency and speed of model inference.

[0189] In some embodiments, the neural network processing apparatus further includes a compiler, and the weight data encoding module can reuse the compiler so that the weight encoding operation (i.e., converting unsigned weight data into signed weight data) is performed offline by the compiler, thereby further reducing the overhead of real-time hardware computation.

[0190] This application also provides an electronic device that may integrate the multiply-accumulate arithmetic unit of this application. This electronic device can achieve the following functions through the multiply-accumulate arithmetic unit:

[0191] Based on the data type of the original weight data, the corresponding signed weight data is obtained. The signed weight data includes two sets of weight data, positive and negative, and the signed weight data is broadcast to the selection shift module in the memory array.

[0192] Based on the data type of the input activation data, the corresponding signed activation data is obtained, and the signed activation data is encoded by Booz, generating several encoded activation values, which are then broadcast to the selection shift module in the memory array.

[0193] Based on the type of the encoded activation value received by the selection shift module, the selection shift module selects one of the received signed weighted data as the target data; and performs a shift operation on the target data to obtain the result of multiplying the signed weighted data and the encoded activation value.

[0194] For each selection shift module in the in-memory array, the results of multiplication of multiple rows in the same column are added to obtain the cumulative result of the row within the column.

[0195] Shift and sum all the row sums within each column to generate the initial multiplication and summation result;

[0196] Based on the data type of the original weight data, the data type of the input activation data, and the encoded activation value, the compensation code is determined, and the initial multiplication-accumulation result is compensated using the compensation code to obtain the final multiplication-accumulation result.

[0197] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0198] Therefore, embodiments of this application provide a storage medium storing multiple instructions that can be loaded by a processor to execute the steps of any of the methods provided in this application. The storage medium may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0199] Since the instructions stored in the storage medium can execute the steps of any method provided in the embodiments of this application, the beneficial effects that any method provided in the embodiments of this application can achieve can be realized. For details, please refer to the previous embodiments, which will not be repeated here.

[0200] The multiplication and accumulation operation method, apparatus, neural network processing apparatus, and electronic device provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for multiplication and accumulation operations, characterized in that, include: Based on the data type of the original weight data, the corresponding signed weight data is obtained. The signed weight data includes two sets of weight data, positive and negative. The signed weight data is then broadcast to the selection shift module in the in-memory array. The in-memory array includes the selection shift module, which has m rows and n columns. Based on the data type of the input activation data, the corresponding signed activation data is obtained, and the signed activation data is encoded by Booz, generating several encoded activation values, and the encoded activation values ​​are broadcast to the selection shift module in the in-memory array. Based on the type of the encoded activation value received by the selection shift module, the selection shift module selects one of the received signed weight data as the target data; and performs a shift operation on the target data to obtain the product of the signed weight data and the encoded activation value. For each of the selection shift modules in the storage array, the multiplication results of multiple rows in the same column are added to obtain the cumulative result of the row in the column; Shift and sum all the row sums in the columns to generate an initial multiplication and summation result; Based on the data type of the original weight data, the data type of the input activation data, and the encoded activation value, a compensation code is determined, and the initial multiply-accumulate result is compensated using the compensation code to obtain the final multiply-accumulate result.

2. The multiplication and accumulation method as described in claim 1, characterized in that, The process of obtaining the corresponding signed weight data based on the data type of the original weight data includes: If the original weight data is of unsigned type, then the first offset is subtracted from the original weight data to convert the unsigned original weight data into signed weight data that includes both positive and negative weight data. If the original weight data is of the type of signed number, then the original weight data will be treated as signed weight data.

3. The multiplication and accumulation operation method as described in claim 1, characterized in that, The step involves obtaining corresponding signed activation data based on the data type of the input activation data, and then performing Booth encoding on the signed activation data to generate several encoded activation values, including: If the data type of the input activation data is an unsigned number, then the second offset is subtracted from the input activation data to obtain signed activation data; If the data type of the input activation data is a signed number, then the input activation data will be treated as signed activation data. The signed activation data is encoded using Booth coding to generate several encoded activation values, including both negative and non-negative numbers.

4. The multiplication and accumulation operation method as described in claim 3, characterized in that, The Booth encoding is Radix-4 Booth encoding. The Booth encoding of the signed activation data to generate several encoded activation values, including negative and non-negative numbers, includes: S1. Pad zeros after the least significant bit of the signed active data to form extended data; S2. Starting from the least significant bit of the extended data, select Z consecutive bits of data to the left as the first encoding window; S3. Use the most significant bit of the first encoding window as the least significant bit of the next encoding window; S4. Continue to select Z-1 consecutive bits of data to the left from the extended data, and form a new encoding window with the least significant bit of the next encoding window; S5. Repeat steps S3 and S4 until the bit stream of the extended data has been selected; S6. According to the preset Booth coding rules, each coding window is mapped to a corresponding coding activation value, thereby obtaining several coding activation values ​​including negative and non-negative numbers.

5. The multiplication and accumulation operation method as described in claim 4, characterized in that, The step of selecting one of the received signed weighted data as target data based on the type of the encoded activation value received by the selection shift module includes: When the encoded activation value received by the selection shift module is negative, the selection shift module selects negative weight data as the target data; When the encoding activation value received by the selection shift module is non-negative, the selection shift module selects positive weight data as the target data.

6. The multiplication and accumulation operation method as described in claim 1, characterized in that, The addition of the multiplication results of multiple rows in the same column to obtain the cumulative sum of rows within the column includes: The multiplication results of the same column are input into the first-stage adder for accumulation, and the highest bit of the accumulated result is inverted to obtain the intermediate result within the column; The intermediate results within all columns are summed to obtain the cumulative result of the rows within the column.

7. The multiplication and accumulation method according to any one of claims 1-6, characterized in that, The step of determining the compensation code based on the data type of the original weight data, the data type of the input activation data, and the encoded activation value includes: Based on the data type of the original weight data, determine the first compensation component; The second compensation component is determined based on the data type of the input activation data; The third compensation component is determined based on the negative number in the encoded activation value; The fourth compensation component is determined by inverting the highest bit of the accumulated result output by the first-stage adder. The first compensation component, the second compensation component, the third compensation component, and the fourth compensation component are added together to obtain the compensation code.

8. The multiplication and accumulation operation method as described in claim 7, characterized in that, If the original weight data is of unsigned number type, then the first compensation component is the product of the sum of all input activation data and the first offset; if the original weight data is of signed number type, then the first compensation component is zero. If the data type of the input activation data is an unsigned number, then the second compensation component is the product of the sum of all the original weight data and the second offset; if the data type of the input activation data is a signed number, then the second compensation component is zero. The third compensation component is the sum of the compensation values ​​corresponding to all negative coded activation values ​​in each calculation cycle; The fourth compensation component is the negative of the sum of fixed offsets introduced by inverting the highest bit of the accumulated result.

9. The multiplication and accumulation method as described in claim 2 or 8, characterized in that, The first offset is 2^(M-1), where M is the bit width of the original weight data.

10. The multiplication and accumulation method as described in claim 3 or 8, characterized in that, The second offset is 2^(N-1), where N is the bit width of the input activation data.

11. A multiply-accumulate arithmetic device, characterized in that, include: The weight data encoding module is used to obtain the corresponding signed weight data according to the data type of the original weight data. The signed weight data includes two sets of weight data, positive and negative, and broadcasts the signed weight data to the selection shift module in the memory array. The activation data encoding module is used to obtain the corresponding signed activation data according to the data type of the input activation data, perform Booz coding on the signed activation data to generate several encoded activation values, and broadcast the encoded activation values ​​to the selection shift module in the in-memory array. The in-memory array includes an m-row n-column selection shift module and several addition processing modules; The selection shift module is used to select one of the received signed weight data as the target data according to the type of the received encoded activation value; and to perform a shift operation on the target data to obtain the multiplication result of the signed weight data and the encoded activation value; the addition processing module is used to perform addition processing on the multiplication results of multiple rows in the same column for each selection shift module in the storage array to obtain the cumulative result of the row in the column; The shift-accumulate module is used to shift and accumulate all the row-to-row accumulation results in the column to generate an initial multiplication-accumulation result; The compensation calculation module is used to determine the compensation code based on the data type of the original weight data, the data type of the input activation data, and the encoded activation value, and to compensate the initial multiplication-accumulation result using the compensation code to obtain the final multiplication-accumulation result.

12. The multiply-accumulate arithmetic device as described in claim 11, characterized in that, Also includes: An activation data caching module is used to cache the signed activation data obtained by the activation data encoding module; And / or, The weight data caching module is used to cache the signed weight data obtained by the weight data encoding module.

13. The multiply-accumulate arithmetic device as described in claim 12, characterized in that, The in-memory computing array is an array based on in-memory computing.

14. The multiply-accumulate arithmetic device as described in claim 11, characterized in that, The weighted data encoding module is used for: When the data type of the original weight data is unsigned, the first offset is subtracted from the original weight data to convert the unsigned original weight data into signed weight data that includes both positive and negative weight data. When the data type of the original weight data is a signed number, the original weight data is treated as signed weight data; The signed weight data is broadcast to the selection shift module in the in-memory array.

15. The multiply-accumulate arithmetic device as claimed in claim 11, characterized in that, The activation data encoding module is used for: When the data type of the input activation data is an unsigned number, the second offset is subtracted from the input activation data to obtain signed activation data; When the data type of the input activation data is a signed number, the input activation data is treated as signed activation data; The signed activation data is encoded using Booth coding to generate several encoded activation values, including negative and non-negative numbers. The encoded activation value is broadcast to the selection shift module in the in-memory array.

16. A neural network processing device, characterized in that, Includes the multiply-accumulate arithmetic device as described in any one of claims 11-15.

17. The neural network processing apparatus as claimed in claim 16, characterized in that, The neural network processing device includes a compiler, and the weight data encoding module reuses the compiler so that the weight encoding operation is performed when the compiler is offline.

18. An electronic device, characterized in that, Includes the multiply-accumulate arithmetic device as described in any one of claims 11-15.

Images