Hybrid In-Memory Computing Architecture and Device Based on Remainder System

Abstract

This application relates to a hybrid in-memory computing architecture and device based on a remainder system. The in-memory computing architecture includes: multiple computing cores, a global cache module, a data transmission module, and a top-level control module. The data transmission module, under the control of the top-level control module, acquires the input matrix and weight matrix, and sends them to each computing core through the global cache module. The computing cores, under the control of the top-level control module, perform multiplication and accumulation operations on the input matrix and weight matrix using a modulo-array-based remainder system to obtain output data, which is then stored in the global cache module. The global cache module, under the control of the top-level control module, obtains the target computation result based on the output data. The data transmission module is also used to output the target computation result under the control of the top-level control module. This architecture improves computational efficiency.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a hybrid in-memory computing architecture and apparatus based on a remainder system. Background Technology

[0002] As artificial neural networks continue to grow in scale, large model architectures have achieved significant performance improvements in fields such as natural language processing and computer vision. The complexity and large number of parameters of large models bring huge computational and storage requirements. FP4 (4-bit Floating-Point), as a low-precision floating-point number format, reduces computational complexity and energy consumption by reducing the bit width of floating-point numbers, and has significant energy efficiency advantages in the inference stage of large models.

[0003] Existing floating-point in-memory computation schemes are typically based on back-alignment or pre-alignment methods. Back-alignment methods involve exponent alignment and integer multiplication and accumulation of mantissas, interleaving these tightly coupled operations, which introduces significant design complexity and circuit overhead in in-memory computation architectures. Pre-alignment methods, on the other hand, pre-align the exponents of the inputs and weights to achieve local maxima, ensuring that the exponent of the product and the exponent of the partial sum are always the same.

[0004] However, these methods suffer from high complexity and low overall computational performance when dealing with low-precision floating-point numbers. Summary of the Invention

[0005] Therefore, it is necessary to provide a hybrid in-memory computing architecture and device based on a remainder system that can improve computing energy efficiency in response to the above-mentioned technical problems.

[0006] In a first aspect, this application provides a hybrid in-memory computing architecture based on a remainder system. The in-memory computing architecture includes: multiple computing cores, a global cache module, a data transmission module, and a top-level control module. The data transmission module is connected to the global cache module, and the global cache module is also connected to the computing cores. The top-level control module is connected to each computing core, the global cache module, and the data transmission module respectively.

[0007] The data transmission module is used to acquire the input matrix and weight matrix under the control of the top-level control module, and send the input matrix and weight matrix to each computing core through the global cache module;

[0008] The computation kernel, under the control of the top-level control module, uses a modulo array-based remainder system to perform multiplication and accumulation operations on the input matrix and weight matrix to obtain output data, which is then stored in the global cache module.

[0009] The global cache module is used to obtain the target calculation result based on the output data under the control of the top-level control module.

[0010] The data transmission module is also used to output the target calculation results under the control of the top-level control module.

[0011] In one embodiment, the computing core includes multiple computing modules, a local cache module, and a local control module, with the local cache module and the local control module respectively connected to each computing module;

[0012] The local caching module is used to split the input matrix into multiple input vectors and send the multiple input vectors to each calculation module respectively, and to split the weight matrix into multiple weight vectors and send the multiple weight vectors to each calculation module respectively;

[0013] The local control module is used to generate periodic signals;

[0014] The calculation module is used to perform multiplication and accumulation operations on the received input vector and weight vector based on the periodic signal to obtain the output data, and then send the output data to the global cache module.

[0015] In one embodiment, the calculation module includes multiple calculation rows, multiple first converters, an accumulation module, a second converter, and a normalization module. Each calculation row is connected to a local cache module, and each calculation row is also connected to a corresponding first converter. The multiple first converters are also connected to the accumulation module, the accumulation module is also connected to the second converter, the second converter is also connected to the normalization module, and the normalization module is also connected to the global cache module.

[0016] The computation row is used to perform multiplication on the input vector and the weight vector to obtain the product data;

[0017] The first converter is used to convert the product data output from each calculation row into remainder data;

[0018] The accumulation module is used to perform partial sum accumulation calculations on multiple remainder data to obtain the cumulative remainder sum;

[0019] The second converter is used to convert the sum of remainders into binary fixed-point numbers;

[0020] The normalization module is used to convert binary fixed-point numbers to floating-point numbers to obtain output data.

[0021] In one embodiment, the computation row includes a storage unit and a logic unit. A first end of the storage unit is connected to a local cache module, a second end of the storage unit is connected to the input end of the logic unit, the input end of the logic unit is also connected to the local cache module, and the output end of the logic unit is connected to the input end of a first converter.

[0022] The storage unit is used to receive and store the weight vector;

[0023] The logic unit is used to perform multiplication on the input vector and the weight vector to obtain the product data.

[0024] In one embodiment, the storage unit includes a plurality of memories, each memory being used to store weight data in a weight vector.

[0025] In one embodiment, the product data includes: a product exponent, a product mantissa, and a product sign, wherein the product exponent includes the least significant bit, the second least significant bit, and the most significant bit; the logic unit includes: a first full adder, a second full adder, a first AND gate, a first XOR gate, and a first XNOR gate;

[0026] The first XNOR gate is used to perform an XNOR operation on the sign bit of the weight vector and the sign bit of the input vector to obtain the product sign.

[0027] The first XOR gate is used to perform an XOR operation on the mantissa of the weight vector and the mantissa of the input vector to obtain the mantissa of the product.

[0028] The first AND gate is used to perform a bitwise AND operation between the mantissa of the weight vector and the mantissa of the input vector to obtain the first exponential carry.

[0029] The first full adder is used to accumulate the first exponent of the weight vector, the first exponent of the input vector, and the first exponent carry to obtain the second exponent carry and the least significant bit of the product exponent.

[0030] The second full adder is used to accumulate the second exponent of the weight vector, the second exponent of the input vector, and the second exponent carry to obtain the second lowest and highest bits of the product exponent.

[0031] In one embodiment, the accumulation module includes: a first addition tree, a second addition tree, a first adder, and a subtractor. The input terminals of the first and second addition trees are respectively connected to the output terminals of the first converter. The input terminals of the first adder are respectively connected to the output terminals of the first and second addition trees. The input terminals of the subtractor are respectively connected to the output terminals of the first and second addition trees.

[0032] The first converter is also used to convert the product exponent into binary exponent data. If the binary exponent data is less than or equal to a preset threshold, the product data is sent to the first addition tree; if the binary exponent data is greater than the threshold, the product data is sent to the second addition tree.

[0033] The first addition tree is used to perform addition calculations on the received product data to obtain the first accumulated data; the second addition tree is used to perform addition calculations on the received product data to obtain the second accumulated data.

[0034] The first adder and the subtractor are used to calculate and process the first accumulated data and the second accumulated data, respectively, to obtain the sum of the remainders.

[0035] In one embodiment, the first addition tree includes a plurality of second adders, the second adders including:

[0036] An AND gate group, which includes multiple AND gates, wherein the first input of the AND gate is used to receive the first target bit of the addend, and the second input of the AND gate is used to receive the corresponding bit of the first target bit in the augend;

[0037] An XOR gate group, which includes multiple XOR gates, has its first input terminal used to receive the second target bit of the addend, and its second input terminal used to receive the corresponding bit of the second target bit in the addend;

[0038] An OR gate group includes multiple OR gates. The first input of an OR gate is used to receive the carry data output by an AND gate, the second input of an OR gate is used to receive the intermediate data output by an XOR gate, and the OR gate is used to output one bit of the sum of the addend and the augend.

[0039] In one embodiment, the global cache module includes a floating-point adder connected to the computation core. The floating-point adder is used to accumulate the output data with the data stored in the global cache module to obtain the target computation result.

[0040] Secondly, this application also provides a hybrid in-memory computing device based on a remainder system, the device including the hybrid in-memory computing architecture based on a remainder system as described in the first aspect.

[0041] The aforementioned hybrid in-memory computing architecture and device based on the remainder system supports multiply-accumulate calculations on FP4 format data using the remainder system. The in-memory architecture significantly reduces hardware complexity and memory access overhead, and overcomes the limitations of existing technologies in low-precision floating-point calculations. At the same time, by utilizing the characteristics of the remainder system to optimize the multiply-accumulate calculation logic, it can avoid the problems of precision loss and storage redundancy in traditional solutions while ensuring calculation accuracy. This significantly improves the energy efficiency of in-memory computing and can also meet the needs of low-precision calculations in the large model inference stage, achieving high-energy-efficiency computing. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a schematic diagram of a hybrid in-memory computing architecture based on a remainder system in one embodiment.

[0044] Figure 2 This is a schematic diagram of the computing module in one embodiment;

[0045] Figure 3 This is a schematic diagram of the structure of the calculation row in one embodiment;

[0046] Figure 4 This is a schematic diagram of the structure for obtaining the remainder sum in one embodiment;

[0047] Figure 5 This is a schematic diagram of the structure of the second adder in one embodiment;

[0048] Figure 6 This is a schematic diagram of the logic gate circuit A obtained in one embodiment;

[0049] Figure 7 This is a schematic diagram of the logic gate circuit B obtained in one embodiment;

[0050] Figure 8 This is a schematic diagram of the structure of an 8-bit inverse code adder in one embodiment;

[0051] Figure 9 This is a flowchart illustrating a multiplication and accumulation calculation method in one embodiment.

[0052] Figure label:

[0053] 1. Calculation core; 11. Calculation module; 111. Calculation row; 1111. Storage unit; 1112. Logic unit; 112. First converter; 113. Accumulation module; 1131. First addition tree; 1132. Second addition tree; 114. Second converter; 115. Normalization module; 12. Local cache module; 13. Local control module; 2. Global cache module; 3. Data transmission module; 4. Top-level control module. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0055] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0056] In-memory computing is a technology that embeds computing units directly within memory, aiming to minimize the energy consumption and latency of data transfer. By performing computations within memory, in-memory computing architectures can significantly improve computational efficiency and energy efficiency.

[0057] Figure 1 A schematic diagram of a hybrid in-memory computing architecture based on a remainder system is shown in one embodiment of this application. (See also...) Figure 1 The hybrid in-memory computing architecture based on the remainder system provided in this application includes: multiple computing cores 1, a global cache module 2, a data transmission module 3, and a top-level control module 4. The data transmission module 3 is connected to the global cache module 2, and the global cache module 2 is also connected to the computing cores 1. The top-level control module 4 is connected to each computing core 1, the global cache module 2, and the data transmission module 3 respectively.

[0058] The data transmission module 3 is used to acquire the input matrix and weight matrix under the control of the top-level control module 4, and send the input matrix and weight matrix to each computing core 1 through the global cache module 2; the computing core 1 is used to perform multiplication and accumulation operations on the input matrix and weight matrix using a modulo array-based remainder system under the control of the top-level control module 4 to obtain output data, and store it in the global cache module 2; the global cache module 2 is used to obtain the target calculation result based on each output data under the control of the top-level control module 4; the data transmission module 3 is also used to output the target calculation result under the control of the top-level control module 4.

[0059] The data transmission module 3 coordinates data interaction between the in-memory computing chip and the external environment. Since the bandwidth between the chip and the external environment is much smaller than the internal bandwidth of the chip, the data transmission module 3 ensures transmission efficiency and data integrity by aggregating and buffering data. During the data input phase, the data transmission module 3 aggregates the data transmitted from the external environment to reach the bandwidth required by the chip before inputting it into the chip for processing. During the data output phase, the calculation results are aggregated and buffered to match the external transmission bandwidth before being output in an orderly manner. This data aggregation and buffering mechanism not only optimizes the communication efficiency between the chip and the external environment but also ensures the stability and continuity of the data flow, supporting efficient computing and data transmission.

[0060] In some embodiments, the data transmission module 3 may be a dual-port memory or register array based on a first-in-first-out (FIFO) data buffer mechanism.

[0061] The global cache module 2 is responsible for temporarily storing the input matrices and weight matrices required for computation, and distributing these data to the corresponding computing cores 1 for processing under the control of the top-level control module 4. The global cache module 2 not only ensures that each computing core 1 can obtain the necessary computing resources in a timely manner, but also temporarily stores the intermediate results of the computation. After the computing core 1 completes its computation task, the global cache module 2 organizes the output data obtained by each computing core 1 into the target computation result, and transmits the target computation result to the data transmission module 3 so that the target computation result can be transmitted outside the chip, ensuring the efficiency and data integrity of the entire computation process.

[0062] The top-level control module 4 is responsible for coordinating and controlling the operation of all modules within the chip. By decoding the chip's configuration information, the top-level control module 4 ensures that each module operates correctly according to its predetermined configuration and timing. Specifically, the top-level control module 4 is responsible for generating and distributing control signals to ensure that the data transmission module 3, the global cache module 2, and the computing core 1 perform their respective operations at the correct times. This centralized control mechanism not only guarantees the coordination and consistency among the modules within the chip but also optimizes the efficiency and accuracy of the overall computation process, ensuring that the hybrid in-memory computing architecture based on the remainder system can operate stably and efficiently under various operating conditions.

[0063] Among them, the Residue Number System (RNS) based on modulo arrays represents a number as the remainder of multiple modulo numbers, thereby splitting the input matrix and weight matrix into multiple independent computational tasks. The modulo array-based remainder system has the characteristics of easy remainder calculation and no carry-free addition, which allows each computational task to be executed simultaneously and the calculation result to be obtained quickly, thus greatly improving the calculation speed and reducing the computational energy consumption.

[0064] For example, the modulo array can be based on 2 n Modular arrays in the form of ±1, such as In this form, or based on a modulo array of prime numbers, such as {3,5,7}.

[0065] The aforementioned hybrid in-memory computing architecture based on the remainder system supports multiply-accumulate calculations on FP4 format data using the remainder system. The in-memory architecture significantly reduces hardware complexity and memory access overhead, and overcomes the limitations of existing technologies in low-precision floating-point calculations. At the same time, by utilizing the characteristics of the remainder system to optimize the multiply-accumulate calculation logic, it can avoid the problems of precision loss and storage redundancy in traditional solutions while ensuring computational accuracy. This significantly improves the energy efficiency of in-memory computing and can also meet the needs of low-precision calculations in the inference stage of large models, achieving high-energy-efficiency computing.

[0066] In an exemplary embodiment, the computing core 1 includes a plurality of computing modules 11, a local cache module 12, and a local control module 13, wherein the local cache module 12 and the local control module 13 are respectively connected to each computing module 11.

[0067] The local cache module 12 is used to split the input matrix into multiple input vectors and send the multiple input vectors to each calculation module 11 respectively, and to split the weight matrix into multiple weight vectors and send the multiple weight vectors to each calculation module 11 respectively; the local control module 13 is used to generate a periodic signal; the calculation module 11 is used to perform multiplication and accumulation operations on the received input vectors and weight vectors based on the periodic signal to obtain output data, and send the output data to the global cache module 2.

[0068] In one exemplary embodiment, see [reference] Figure 2 The calculation module 11 includes multiple calculation rows 111, multiple first converters 112, an accumulation module 113, a second converter 114, and a normalization module 115. Each calculation row 111 is connected to the local cache module 12. Each calculation row 111 is also connected to a corresponding first converter 112. The multiple first converters 112 are also connected to the accumulation module 113. The accumulation module 113 is also connected to the second converter 114. The second converter 114 is also connected to the normalization module 115. The normalization module 115 is also connected to the global cache module 2.

[0069] The calculation line 111 is used to perform multiplication on the input vector and the weight vector to obtain product data; the first converter 112 is used to convert the product data output by each calculation line 111 into remainder data; the accumulation module 113 is used to perform partial sum accumulation on multiple remainder data to obtain the remainder accumulation sum; the second converter 114 is used to convert the remainder accumulation sum into a binary fixed-point number; the normalization module 115 is used to perform floating-point conversion on the binary fixed-point number to obtain the output data.

[0070] For example, the hybrid in-memory computing architecture based on the remainder system provided in this application embodiment may include 8 computing cores 1, each computing core 1 includes 128 computing modules 11, each computing module 11 contains 128 computing rows 111 and 128 first converters 112, each computing module 11 performs a multiplication and accumulation operation on a row of the input matrix and the weight matrix to obtain output data.

[0071] See Figure 2 and Figure 3 In one possible implementation, the computing line 111 includes a storage unit 1111 and a logic unit 1112. The first end of the storage unit 1111 is connected to the local cache module 12, the second end of the storage unit 1111 is connected to the input end of the logic unit 1112, the input end of the logic unit 1112 is also connected to the local cache module 12, and the output end of the logic unit 1112 is connected to the input end of the first converter 112.

[0072] The storage unit 1111 is used to receive and store the weight vector; the logic unit 1112 is used to perform multiplication calculation on the input vector I and the weight vector W to obtain the product data.

[0073] In some embodiments, the storage unit 1111 includes: a plurality of memories, each memory being used to store a weight data W[i] in the weight vector.

[0074] For example, the memory can be SRAM (Static Random-Access Memory) or DRAM (Dynamic Random Access Memory).

[0075] Continue reading Figure 3 In some embodiments, the product data includes: product exponent E, product mantissa M, and product sign S, wherein the product exponent E includes the least significant bit E[0], the second least significant bit E[1], and the most significant bit E[2]. Figure 3From the perspective of the logic unit 1112, from left to right, are: a first full adder, a second full adder, a first AND gate, a first XOR gate, and a first XNOR gate. The first XOR gate is used to perform an XOR operation on the sign bit W[3] of the weight vector and the sign bit I[3] of the input vector to obtain the product sign S; the first XOR gate is used to perform an XOR operation on the mantissa bit W[0] of the weight vector and the mantissa bit I[0] of the input vector to obtain the product mantissa M; the first AND gate is used to perform an AND operation on the mantissa bit W[0] of the weight vector and the mantissa bit I[0] of the input vector to obtain the first exponent carry; the first full adder is used to accumulate the first exponent bit W[1] of the weight vector, the first exponent bit I[1] of the input vector and the first exponent carry to obtain the second exponent carry and the least significant bit E[0] of the product exponent; the second full adder is used to accumulate the second exponent bit W[2] of the weight vector, the second exponent bit I[2] of the input vector and the second exponent carry to obtain the second least significant bit E[1] and the most significant bit E[2] of the product exponent.

[0076] In this embodiment, the logic unit 1112 in each calculation row 111 performs multiplication calculation on the 4b input vector and the 4b weight vector to obtain the 5b product data, and transmits the product data to the first converter 112. The part and exponent of the product data are both integer powers of 2, and the mantissa part is only 1 bit, which can significantly reduce the computational complexity and reduce the computational energy consumption, thereby improving the computational efficiency.

[0077] In one exemplary embodiment, a method based on The RNS of the modulus array is partially summed and accumulated. As shown in Table 1, integer powers of 2 are used to sum the modulus array. The remainders exhibit a certain regularity: when the product exponent E is less than or equal to 3, the remainder is always 3. When the product exponent E is greater than 3, for 2 4 The remainder after -1 is , for 2 4 The remainder is 0, and the remainder is 0. 4 +1 and take the remainder. .

[0078]

[0079] See Figure 4 The first converter 112 is used to divide the product data into two parts and take the remainder, that is:

[0080]

[0081] Both of these parts are also integer powers of 2. Among them, E0 represents the data corresponding to the lowest bit E[0] of the product exponent; E1 represents the data corresponding to the second lowest bit E[1] of the product exponent; and E2 represents the data corresponding to the highest bit E[2] of the product exponent.

[0082] Multiple product data generated from multiple calculation rows are accumulated over two clock cycles. The cycle signal is generated by the local control module 13. In the first clock cycle, the RNS part after taking the modulo-array remainder is accumulated. Accumulate in the second clock cycle .

[0083] In one possible implementation, the accumulation module 113 includes: a first addition tree 1131, a second addition tree 1132, a first adder, and a subtractor. The input terminals of the first addition tree 1131 and the second addition tree 1132 are respectively connected to the output terminal of the first converter 112. The input terminal of the first adder is connected to the output terminals of the first addition tree 1131 and the second addition tree 1132, respectively. The input terminal of the subtractor is connected to the output terminals of the first addition tree 1131 and the second addition tree 1132, respectively. Figure 4 From the perspective of [the user], the symbol "+" indicates the first adder; "-" indicates the subtractor.

[0084] The first converter 112 is also used to convert the product exponent into binary exponent data. If the binary exponent data is less than or equal to a preset threshold, the product data is sent to the first addition tree 1131; if the binary exponent data is greater than the threshold, the product data is sent to the second addition tree 1132. The first addition tree 1131 is used to perform addition calculations on the received product data to obtain the first accumulated data. The second addition tree 1132 is used to perform addition calculations on the received product data to obtain the second accumulated data. The first adder and the subtractor are respectively used to perform calculation processing on the first accumulated data and the second accumulated data to obtain the remainder sum.

[0085] In some embodiments, the product exponent can be converted into binary exponent data using a 2-to-4 decoder in the first converter 112. The threshold can be 3. If the binary exponent data is less than or equal to 3, the product data is sent to the first addition tree 1131; if it is greater than 3, it is sent to the second addition tree 1132. In this way, the first addition tree 1131 and the second addition tree 1132 operate in parallel, simultaneously performing efficient partial sum accumulation on multiple product data. Furthermore, the product data received by each addition tree is the RNS partial sum that has been preprocessed and aligned by the first converter 112, enabling fast and convenient completion of addition calculations and improving computation speed.

[0086] The first addition tree 1131 and the second addition tree 1132 have the same structure. Taking the first addition tree 1131 as an example, the structure of the first addition tree 1131 can be divided into multiple layers. Each layer consists of a set of parallel second adders. Each layer receives all the data to be added, pairs these data into addends and augends, and sends them to the second adders of that layer for addition to obtain the sum. The sum is used as the data received by the next layer or as the first accumulated data of the final output.

[0087] See Figure 5 In one possible implementation, the first addition tree 1131 includes a plurality of second adders, each second adder comprising: an AND gate group, which includes a plurality of AND gates, wherein the first input of each AND gate is used to receive a first target bit of the addend, and the second input of each AND gate is used to receive a corresponding bit of the first target bit in the augend; an XOR gate group, which includes a plurality of XOR gates, wherein the first input of each XOR gate is used to receive a second target bit of the addend, and the second input of each XOR gate is used to receive a corresponding bit of the second target bit in the augend; and an OR gate group, which includes a plurality of OR gates, wherein the first input of each OR gate is used to receive carry data output by the AND gate, the second input of each OR gate is used to receive intermediate data output by the XOR gate, and the OR gate is used to output one bit of the sum of the addend and the augend.

[0088] exist Figure 5 From the perspective of the AND gate group, from right to left, the AND gates are the second AND gate, the third AND gate, the fourth AND gate, and the fifth AND gate; from right to left, the XOR gate group is the second XOR gate, the third XOR gate, the fourth XOR gate, and the fifth XOR gate; from right to left, the OR gate group is the first OR gate, the second OR gate, and the third OR gate. The first target bit corresponding to the second AND gate is a[0] in the addend a, that is, the first input terminal of the second AND gate receives a[0], and the second input terminal receives b[0] in the addend b; the second target bit corresponding to the second XOR gate is a[0] in the addend a, that is, the first input terminal of the second XOR gate receives a[0], and the second input terminal receives b[0] in the addend b, and the least significant bit o[0] of the output sum o. The third AND gate and the third XOR gate both receive a[1] and b[1]; the fourth AND gate and the fourth XOR gate both receive a[2] and b[2]; the fifth AND gate and the fifth XOR gate both receive a[3] and b[3]. The first input of the first OR gate receives the carry data output by the second AND gate, and the second output receives the intermediate data output by the third XOR gate. The first OR gate outputs the sum of o[1]. The first input of the second OR gate receives the carry data output by the third AND gate, and the second output receives the intermediate data output by the fourth XOR gate. The second OR gate outputs the sum of o[2]. The first input of the third OR gate receives the carry data output by the fourth AND gate, and the second output receives the intermediate data output by the fifth XOR gate. The third OR gate outputs the sum of o[3]. The fifth AND gate outputs the sum of o[4].

[0089] In some embodiments, the first adder is used to perform addition calculations on the first accumulated data and the second accumulated data to obtain the modulus 2 in the remainder sum. 4 -1 corresponds to the data; the subtractor is used to perform subtraction on the first and second accumulated data to obtain the modulus 2 in the sum of the remainders. 4 +1 corresponds to the data; based on the first accumulated data, the modulus 2 in the remainder sum can be directly obtained. 4 The corresponding data.

[0090] The first addition tree 1131 in this embodiment can perform addition calculations on multiple product data layer by layer to obtain the first accumulated data. Compared with the addition tree based on full adders in the prior art, the addition tree based on second adders in this embodiment only requires an average of 1 AND gate, 1 XOR gate and 1 OR gate per bit to add two input numbers, which can effectively reduce the computational energy consumption. Furthermore, by using the first addition tree 1131 and the second addition tree 1132 to perform parallel accumulation calculations on multiple product data, and using a carry-free adder in each addition tree to further accelerate the process, the remainder accumulation sum under different moduli is finally obtained through a simple combination of addition and subtraction. This can overcome the alignment and carry propagation delay problems in traditional floating-point accumulation, thereby achieving extremely high-efficiency matrix multiplication accumulation in the in-memory computing architecture.

[0091] In an exemplary embodiment, a second converter 114 is used to convert the remainder sum into a binary fixed-point number. According to the remainder theorem, the binary number... It can be represented as:

[0092]

[0093] Where n is the exponent of 2 in the modulus array, and in this embodiment, n is 4. They are {RNS_2 4 -1,RNS_2 4 , RNS_2 4 +1}. Binary number This can be further represented in a form that is easily implemented using an adder:

[0094]

[0095] in, It can be represented as:

[0096]

[0097] like Figure 6 As shown, A1 and A2 can be calculated using four full adders, one XOR gate, and one NOR gate, and then A can be obtained.

[0098] It can be represented as:

[0099]

[0100] like Figure 7 As shown, four full adders, one half adder, and one XOR gate can be used to calculate 4 bits of B1 and B2 and 1 bit of C. B [4], thus obtaining B.

[0101] like Figure 8 As shown, an 8-bit one's complement adder consisting of 8 full adders and 8 half adders can be used to calculate the result. . and After combining, we get the binary fixed-point number corresponding to the sum of the remainders.

[0102] Finally, the normalization module 115 processes the binary fixed-point number according to the IEEE 754 standard to handle special cases such as overflow and denormalized numbers, converts it into standard FP4 format output data and stores it in the global cache module 2, thus completing the calculation of the calculation module 11.

[0103] In this embodiment, a logic gate circuit is constructed to convert the remainder sum into a binary fixed-point number, which can quickly and accurately convert the remainder sum, reduce the calculation latency, save hardware resources and improve conversion efficiency.

[0104] In an exemplary embodiment, the global cache module 2 includes a floating-point adder connected to the computing core 1. The floating-point adder is used to accumulate the output data with the stored data in the global cache module 2 to obtain the target calculation result.

[0105] In this embodiment, a floating-point adder is used to ensure the efficiency and data integrity of the entire calculation process, so as to support larger-scale computing scenarios and complex requirements.

[0106] In one exemplary embodiment, a hybrid in-memory computing architecture based on a remainder system is provided, including: multiple computing cores 1, a global cache module 2, a data transmission module 3, and a top-level control module 4.

[0107] Data transmission module 3 is used to acquire the input matrix and weight matrix under the control of top-level control module 4, and send the input matrix and weight matrix to each computing core 1 through global cache module 2;

[0108] The computational core 1 includes multiple computational modules 11, a local cache module 12, and a local control module 13. The local cache module 12 is used to split the input matrix into multiple input vectors and send the multiple input vectors to each computational module 11 respectively, and to split the weight matrix into multiple weight vectors and send the multiple weight vectors to each computational module 11 respectively. The local control module 13 is used to generate a periodic signal. The computational module 11 is used to perform multiplication and accumulation operations on the received input vectors and weight vectors based on the periodic signal to obtain output data, and send the output data to the global cache module 2.

[0109] The calculation module 11 includes multiple calculation rows 111, multiple first converters 112, an accumulation module 113, a second converter 114, and a normalization module 115.

[0110] The calculation line 111 includes a storage unit 1111 and a logic unit 1112. The first end of the storage unit 1111 is connected to the local cache module 12, and the second end of the storage unit 1111 is connected to the input end of the logic unit 1112. The storage unit 1111 includes multiple memories, each memory being used to store one weight data in the weight vector. The input end of the logic unit 1112 is also connected to the local cache module 12, and the output end of the logic unit 1112 is connected to the input end of the first converter 112. The storage unit 1111 is used to receive and store the weight vector. The logic unit 1112 is used to perform multiplication calculation on the input vector and the weight vector to obtain the product data.

[0111] The product data includes: product exponent, product mantissa, and product sign. The product exponent includes the least significant bit, the second least significant bit, and the most significant bit. The logic unit 1112 includes: a first full adder, a second full adder, a first AND gate, a first XOR gate, and a first XNOR gate. The first XNOR gate is used to perform an XOR operation on the sign bit of the weight vector and the sign bit of the input vector to obtain the product sign. The first XOR gate is used to perform an XOR operation on the mantissa bit of the weight vector and the mantissa bit of the input vector to obtain the product mantissa. The first AND gate is used to perform an AND operation on the mantissa bit of the weight vector and the mantissa bit of the input vector to obtain the first exponent carry. The first full adder is used to accumulate the first exponent bit of the weight vector, the first exponent bit of the input vector, and the first exponent carry to obtain the second exponent carry and the least significant bit of the product exponent. The second full adder is used to accumulate the second exponent bit of the weight vector, the second exponent bit of the input vector, and the second exponent carry to obtain the second least significant bit and the most significant bit of the product exponent.

[0112] Multiple first converters 112 are also connected to an accumulation module 113, which is also connected to a second converter 114. The second converter 114 is also connected to a normalization module 115, which is also connected to a global cache module 2. The first converters 112 are used to convert the product data output from each calculation row 111 into remainder data. The accumulation module 113 is used to perform partial sum accumulation calculation on multiple remainder data to obtain the remainder sum. The second converter 114 is used to convert the remainder sum into a binary fixed-point number. The normalization module 115 is used to perform floating-point conversion on the binary fixed-point number to obtain the output data.

[0113] The accumulation module 113 includes a first addition tree 1131, a second addition tree 1132, a first adder, and a subtractor. The inputs of the first addition tree 1131 and the second addition tree 1132 are respectively connected to the output of the first converter 112. The input of the first adder is connected to the outputs of the first addition tree 1131 and the second addition tree 1132. The input of the subtractor is connected to the outputs of the first addition tree 1131 and the second addition tree 1132. The first converter 112 is also used to convert the product exponent into a binary exponent. If the binary exponent data is less than or equal to a preset threshold, the product data is sent to the first addition tree 1131; if the binary exponent data is greater than the threshold, the product data is sent to the second addition tree 1132. The first addition tree 1131 is used to perform addition calculations on the received product data to obtain the first accumulated data; the second addition tree 1132 is used to perform addition calculations on the received product data to obtain the second accumulated data; the first adder and the subtractor are respectively used to perform calculations on the first accumulated data and the second accumulated data to obtain the remainder sum.

[0114] The first addition tree 1131 includes multiple second adders, each of which includes: an AND gate group, comprising multiple AND gates, wherein the first input of each AND gate is used to receive the first target bit of the addend, and the second input of each AND gate is used to receive the corresponding bit of the first target bit in the augend; an XOR gate group, comprising multiple XOR gates, wherein the first input of each XOR gate is used to receive the second target bit of the addend, and the second input of each XOR gate is used to receive the corresponding bit of the second target bit in the augend; and an OR gate group, comprising multiple OR gates, wherein the first input of each OR gate is used to receive the carry data output by the AND gate, the second input of each OR gate is used to receive the intermediate data output by the XOR gate, and the OR gate is used to output one bit of the sum of the addend and the augend.

[0115] Global cache module 2, under the control of top-level control module 4, is used to obtain the target calculation result based on the output data. Global cache module 2 includes a floating-point adder connected to computation core 1. The floating-point adder is used to accumulate the output data with the data stored in global cache module 2 to obtain the target calculation result.

[0116] Data transmission module 3 is also used to output the target calculation results under the control of top-level control module 4.

[0117] In one exemplary embodiment, a hybrid in-memory computing device based on a remainder system is provided, the hybrid in-memory computing device based on a remainder system as described above.

[0118] See Figure 9 In one exemplary embodiment, a multiply-accumulate calculation method is provided. This method is applied to the hybrid in-memory computing architecture based on the remainder system in the above embodiment. The method includes steps 902 to 908, wherein:

[0119] Step 902: Obtain the input matrix and weight matrix.

[0120] Step 904: Perform multiplication and accumulation operations on the input matrix and weight matrix according to the remainder system based on the modulus array to obtain the output data.

[0121] Step 906: Obtain the target calculation result based on the output data.

[0122] Step 908: Output the target calculation result.

[0123] In one embodiment, step 904 may further include: splitting the input matrix into multiple input vectors and sending the multiple input vectors to each computing module 11 respectively; splitting the weight matrix into multiple weight vectors and sending the multiple weight vectors to each computing module 11 respectively; and performing multiplication and accumulation operations on the received input vectors and weight vectors based on the periodic signal to obtain output data.

[0124] In one embodiment, step 904 may further include: performing multiplication on the input vector and the weight vector to obtain product data; converting each product data into remainder data; performing partial sum accumulation on multiple remainder data to obtain a remainder sum; converting the remainder sum into a binary fixed-point number; and performing floating-point conversion on the binary fixed-point number to obtain output data.

[0125] In one embodiment, the process of obtaining the product data may further include: performing a XOR operation on the sign bit of the weight vector and the sign bit of the input vector to obtain the product sign; performing an XOR operation on the mantissa bits of the weight vector and the mantissa bits of the input vector to obtain the product mantissa; performing a AND operation on the mantissa bits of the weight vector and the mantissa bits of the input vector to obtain the first exponent carry; accumulating the first exponent bits of the weight vector, the first exponent bits of the input vector, and the first exponent carry to obtain the second exponent carry and the least significant bit of the product exponent; accumulating the second exponent bits of the weight vector, the second exponent bits of the input vector, and the second exponent carry to obtain the second least significant bit and the most significant bit of the product exponent.

[0126] In one embodiment, the process of obtaining the remainder sum may further include: converting the product exponent into binary exponent data; if the binary exponent data is less than or equal to a preset threshold, sending the product data to a first addition tree 1131, performing addition calculation on the received product data to obtain first accumulated data; if the binary exponent data is greater than the threshold, sending the product data to a second addition tree 1132, performing addition calculation on the received product data to obtain second accumulated data; and performing calculation processing on the first accumulated data and the second accumulated data to obtain the remainder sum.

[0127] In one embodiment, obtaining the target calculation result may include accumulating the output data with the stored data in the global cache module 2 to obtain the target calculation result.

[0128] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0129] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a non-volatile computer-readable storage medium, and when executed, the computer program may include the processes of the embodiments of the methods described above.

[0130] Any references to memory, databases, or other media used in the embodiments provided in this application may include at least one of non-volatile memory and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application may include at least one of relational databases and non-relational databases. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0131] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0132] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A hybrid in-memory computing architecture system based on a remainder system, characterized in that, The hybrid in-memory computing architecture based on the remainder system includes: multiple computing cores, a global cache module, a data transmission module, and a top-level control module. The data transmission module is connected to the global cache module, and the global cache module is also connected to the computing cores. The top-level control module is connected to each computing core, the global cache module, and the data transmission module. Each computing core includes multiple computing modules, a local cache module, and a local control module. The local cache module and the local control module are respectively connected to the computing modules. Each computing module includes multiple computing rows, multiple first converters, an accumulation module, a second converter, and a normalization module. Each computing row is connected to the local cache module, and each computing row is also connected to a corresponding first converter. The multiple first converters are also connected to the accumulation module, the accumulation module is also connected to the second converter, the second converter is also connected to the normalization module, and the normalization module is also connected to the global cache module. The data transmission module is used to acquire the input matrix and the weight matrix under the control of the top-level control module, and send the input matrix and the weight matrix to each of the computing cores through the global cache module; The local caching module is used to split the input matrix into multiple input vectors and send the multiple input vectors to each of the computing modules respectively, and to split the weight matrix into multiple weight vectors and send the multiple weight vectors to each of the computing modules respectively. The local control module is used to generate periodic signals; The calculation row is used to perform multiplication on the input vector and the weight vector to obtain product data; The first converter is used to convert the product data output by each of the calculation rows into remainder data; The accumulation module is used to perform partial sum accumulation calculation on multiple remainder data to obtain the remainder accumulation sum; The second converter is used to convert the sum of the remainders into a binary fixed-point number; The normalization module is used to perform floating-point conversion on the binary fixed-point number to obtain output data, and store it in the global cache module; The global cache module is used to obtain the target calculation result based on the output data under the control of the top-level control module. The data transmission module is also used to output the target calculation result under the control of the top-level control module.

2. The hybrid in-memory computing architecture system based on the remainder system according to claim 1, characterized in that, The computation row includes a storage unit and a logic unit. A first end of the storage unit is connected to the local cache module, a second end of the storage unit is connected to the input end of the logic unit, the input end of the logic unit is also connected to the local cache module, and the output end of the logic unit is connected to the input end of the first converter. The storage unit is used to receive and store the weight vector; The logic unit is used to perform multiplication calculations on the input vector and the weight vector to obtain the product data.

3. The hybrid in-memory computing architecture system based on the remainder system according to claim 2, characterized in that, The storage unit includes multiple memories, each memory being used to store one weight data in the weight vector.

4. The hybrid in-memory computing architecture system based on the remainder system according to claim 3, characterized in that, The memory is a static random access memory or a dynamic random access memory.

5. The hybrid in-memory computing architecture system based on the remainder system according to claim 2, characterized in that, The product data includes: product exponent, product mantissa, and product sign, wherein the product exponent includes least significant bit, second least significant bit, and most significant bit; the logic unit includes: a first full adder, a second full adder, a first AND gate, a first XOR gate, and a first XNOR gate; The first XOR gate is used to perform an XOR operation on the sign bit of the weight vector and the sign bit of the input vector to obtain the product sign; The first XOR gate is used to perform an XOR operation on the mantissa of the weight vector and the mantissa of the input vector to obtain the mantissa of the product. The first AND gate is used to perform an AND operation on the mantissa of the weight vector and the mantissa of the input vector to obtain the first exponential carry; The first full adder is used to accumulate the first exponent bit of the weight vector, the first exponent bit of the input vector, and the first exponent carry to obtain the second exponent carry and the least significant bit of the product exponent; The second full adder is used to accumulate the second exponent of the weight vector, the second exponent of the input vector, and the second exponent carry to obtain the second lowest and highest bits of the product exponent.

6. The hybrid in-memory computing architecture system based on the remainder system according to claim 5, characterized in that, The accumulation module includes: a first addition tree, a second addition tree, a first adder, and a subtractor. The input terminals of the first addition tree and the second addition tree are respectively connected to the output terminal of the first converter. The input terminal of the first adder is respectively connected to the output terminals of the first addition tree and the second addition tree. The input terminal of the subtractor is respectively connected to the output terminals of the first addition tree and the second addition tree. The first converter is further configured to convert the product exponent into binary exponent data, and if the binary exponent data is less than or equal to a preset threshold, send the product data to the first addition tree; if the binary exponent data is greater than the threshold, send the product data to the second addition tree. The first addition tree is used to perform addition calculations on the received product data to obtain the first accumulated data; the second addition tree is used to perform addition calculations on the received product data to obtain the second accumulated data. The first adder and the subtractor are used to perform calculations on the first accumulated data and the second accumulated data, respectively, to obtain the sum of the remainders.

7. The hybrid in-memory computing architecture system based on the remainder system according to claim 6, characterized in that, The first addition tree includes a plurality of second adders, the second adders including: An AND gate group, comprising multiple AND gates, wherein the first input of each AND gate is used to receive a first target bit of the addend, and the second input of each AND gate is used to receive the corresponding bit of the first target bit in the addend; An XOR gate group, comprising multiple XOR gates, wherein the first input of each XOR gate is used to receive the second target bit of the addend, and the second input of each XOR gate is used to receive the corresponding bit of the second target bit in the addend; An OR gate group, comprising multiple OR gates, wherein the first input of each OR gate is used to receive carry data output by the AND gate, the second input of each OR gate is used to receive intermediate data output by the XOR gate, and the OR gate is used to output one bit of the sum of the addend and the augend.

8. The hybrid in-memory computing architecture system based on the remainder system according to claim 1, characterized in that, The global cache module includes a floating-point adder connected to the computation core. The floating-point adder is used to accumulate the output data with the data stored in the global cache module to obtain the target computation result.

9. The hybrid in-memory computing architecture system based on the remainder system according to claim 1, characterized in that, The data transmission module is a dual-port memory or register array based on a first-in-first-out (FIFO) data buffer mechanism.

10. A hybrid in-memory computing device based on a remainder system, characterized in that, The hybrid in-memory computing device based on the remainder system includes the hybrid in-memory computing architecture system based on the remainder system as described in any one of claims 1-9.

Images