Storage circuit, chip and related method

Abstract

The invention discloses a storage circuit, a chip and a related method, and the storage circuit comprises a storage array which comprises N pairs of first storage areas and second storage areas which are spatially adjacent, and N is greater than or equal to 1; one linear operation module is configured to be used for receiving data from the corresponding pair of storage areas and executing linear operation of the first data from the first storage area and the second data from the second storage area with externally input third data; wherein the linear operation module is configured on the data output side of the storage array and is close to the position of a pair of corresponding storage areas. According to the technical scheme, concurrent processing of linear transformation operation on the same input data by using different weight data is facilitated, and the execution efficiency of the linear calculation task is improved.

Description

Technical Field

[0001] This application generally relates to the field of artificial intelligence technology. More specifically, this application relates to a memory computing circuit, a chip, and related methods. Background Technology

[0002] The core computation commonly found in various layers of a neural network model is to perform a linear transformation operation on the data to be processed based on the weight data. The linear transformation operation can be broken down into matrix vector multiplication (MVM) operation, which is further composed of multiply and accumulate (MAC) operation.

[0003] Near-Memory Computing (NMC) architecture helps alleviate data handling problems in data-intensive applications by integrating computing units on or adjacent to memory chips. In neural network models, multiple weights are often needed to compute on the same input data. Because different weights are at different distances from the linear arithmetic logic unit (LAU), the time required for the LAUTO to retrieve different weights from the storage matrix varies. This makes linear computation tasks with different weights on the same input data unsuitable for concurrent processing. Improving the execution efficiency of such tasks that utilize weights for linear computation on input data has become a significant technical challenge.

[0004] In view of this, this application proposes a memory computing circuit, chip, and related method to improve the efficiency of linear computation of input data based on weighted data. Summary of the Invention

[0005] In order to at least solve one or more of the technical problems mentioned above, this application proposes solutions for memory computing circuits, chips, and methods for performing calculations using memory computing circuits in several aspects.

[0006] In a first aspect, this application provides a memory computing circuit, comprising: a memory array including N pairs of spatially adjacent first memory regions and second memory regions, wherein N≥1; N linear operation modules, each linear operation module being configured to receive data from a corresponding pair of memory regions and to perform linear operations on first data from the first memory region and second data from the second memory region respectively with externally input third data; wherein the linear operation module is configured on the data output side of the memory array, near its corresponding pair of memory regions.

[0007] In a second aspect, this application provides a chip including the memory computing circuitry described in the first aspect.

[0008] In a third aspect, this application provides a method for performing calculations using a memory-based computing circuit, wherein the memory-based computing circuit includes a memory array and N linear operation modules, where N ≥ 1; the memory array includes N pairs of spatially adjacent first and second memory regions; the linear operation modules are configured on the data output side of the memory array, near their corresponding pair of memory regions; the method includes using the linear operation modules to: receive data from the corresponding pair of memory regions; and perform linear operations between the first data from the first memory region and the second data from the second memory region and the externally input third data, respectively.

[0009] By using the memory computing circuits, chips, and methods provided above, embodiments of this application, through alternating storage of first data and second data in a storage array and setting linear operation modules in adjacent positions of the storage areas of the first data and second data, can use the same linear operation module to execute linear calculation tasks involving the first data and the second data, thereby improving the consistency of the execution time of the two calculation tasks. This helps to concurrently process linear transformation operations on the same input data using data with different weights, thereby improving the execution efficiency of such linear calculation tasks. Attached Figure Description

[0010] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein: Figure 1 An exemplary structural diagram of a storage array in some embodiments of this application is shown; Figure 2 An exemplary structural diagram of a ReRAM memory cell in some embodiments of this application is shown; Figure 3 A schematic diagram illustrating an exemplary relationship between the resistance value of the resistive switching layer and the storage state in some embodiments of this application is shown; Figure 4 Exemplary structural diagrams of storage arrays in other embodiments of this application are shown; Figure 5 An exemplary schematic diagram of the peripheral circuitry of the storage array in some embodiments of this application is shown; Figure 6 An exemplary schematic diagram is shown in some embodiments of this application, illustrating the linear transformation of data to be processed based on weighted data; Figure 7 This illustration shows an exemplary diagram of calculating the same input data using multiple weight data in some embodiments of this application; Figure 8 Exemplary schematic diagrams of memory computing circuits in some embodiments of this application are shown; Figure 9 Exemplary schematic diagrams of memory computing circuits in some embodiments of this application are shown; Figure 10 An exemplary schematic diagram is shown in some embodiments of this application, illustrating linear calculation of input data based on weight data; Figure 11 Exemplary schematic diagrams of memory computing circuits in some embodiments of this application are shown; Figure 12 An exemplary schematic diagram of a memory computing circuit in some embodiments of this application is shown. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0012] It should be understood that the terms "comprising" and "including" as used in the specification and claims of this application indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0013] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0014] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0015] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise expressly specified. "Several" means one or more, unless otherwise expressly specified.

[0016] The specific embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0017] Near-memory computing architectures help alleviate data movement problems faced by data-intensive applications by integrating computing units on or adjacent to the storage array.

[0018] Figure 1 Exemplary structural diagrams of storage arrays in some embodiments of this application are shown. For example... Figure 1 As shown, the memory array 100 consists of multiple rows and columns of memory sections, accessed via horizontal and vertical wires, commonly referred to as word lines (WL), source lines (SL), and bit lines (BL). A memory section can be deployed between two source lines and between adjacent bit lines and word lines. Each memory section includes memory cells and switching units. The memory cells can be Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Resistive Random Access Memory (ReRAM), Spin-Transfer Torque Magnetoresistive Random Access Memory (STT-MRAM), Phase Change Memory (PCM), Ferroelectric Random Access Memory (FeRAM), or Flash Memory.

[0019] Figure 2 Exemplary structural diagrams of ReRAM memory cells in some embodiments of this application are shown. Figure 2As shown, a ReRAM memory cell includes an upper electrode, a resistive switching layer, and a lower electrode. The upper and lower electrodes can be metal electrodes, such as those made of aluminum (Al), titanium (Ti), or copper (Cu). The resistive switching layer can be a metal oxide material, such as zinc oxide (ZnO), magnesium oxide (MgO), or zirconium oxide (ZrO). x Titanium oxide (TiO) x Materials include aluminum oxide (Al2O3), tantalum oxide (Ta2O5), and hafnium oxide (HfO2). The resistive switching layer can change its resistance value according to the voltage across it.

[0020] Figure 3 This diagram illustrates an exemplary relationship between the resistance value of the resistive switching layer and the storage state in some embodiments of this application. For example... Figure 3 As shown, the positive direction of the resistance axis is the direction of increasing resistance. LR1 is the minimum resistance in the low-resistance state, LR2 is the maximum resistance in the low-resistance state, HR1 is the minimum resistance in the high-resistance state, and HR2 is the maximum resistance in the high-resistance state. Therefore, [LR1, LR2] is the low-resistance range of the resistive switching layer, and [HR1, HR2] is the high-resistance range of the resistive switching layer. In some embodiments, a high-voltage pulse can be applied to the resistive switching layer to form a conductive wire between the upper and lower electrodes, causing the resistive switching layer to exhibit a low-resistance state (LRS). In some embodiments, a low-voltage pulse can be applied to the resistive switching layer to break the conductive wire between the upper and lower electrodes, causing the resistive switching layer to become a high-resistance state (HRS). Moreover, even after the memory cell loses power, the resistive switching layer will maintain its resistance value. In a computing system, data can be represented using fixed-point numbers or floating-point numbers with multiple bits. In some embodiments, the storage array 100 uses each storage cell to store the value 0 / 1 of each bit, and multiple storage cells correspond to one data element; for example, when the storage cell is a ReRAM storage cell, the value 1 can be stored in the low resistance state of the resistive switching layer and the value 0 can be stored in the high resistance state of the resistive switching layer.

[0021] Back Figure 1 The switching unit can be a metal-oxide-semiconductor field-effect transistor (MOSFET). When the memory cell is a ReRAM memory cell and a MOSFET is used as the switching unit, the gate of the MOSFET is connected to the word line, and the current flowing to the memory cell can be controlled by adjusting the word line voltage. The source of the MOSFET is connected to the source line, and the drain is connected to the lower electrode of the memory cell. The upper electrode of the memory cell is connected to the bit line. The switching unit can be used to control access to the memory cell.

[0022] Figure 4 Exemplary structural diagrams of storage arrays from other embodiments of this application are shown. For example... Figure 4 As shown, the memory array 100 includes word lines and bit lines. A memory section can be deployed within the area formed by two adjacent word lines and two adjacent bit lines. The memory section includes memory cells and a selector. The selector can be a bidirectional diode-type selector or other nonlinear selection devices, such as a bidirectional threshold switch (OTS) or a metal-insulator transition (MIT). The selector is used to control access to the series-connected memory cells.

[0023] Figure 5 Exemplary schematic diagrams of peripheral circuitry for a memory array in some embodiments of this application are shown. In some embodiments, the memory array 100 stores weight data of a neural network, and the peripheral circuitry can retrieve data from the memory array 100 based on externally input address signals. Figure 5 As shown, the peripheral circuit of the storage array 100 includes a row address decoder, a column address decoder, a sensitive amplifier, and an output buffer. The externally input address signal is divided into a row address and a column address, which are used to locate the target storage cell where data needs to be read. The row address and column address are transmitted to the row address decoder and the column address decoder, respectively. The row address decoder decodes the row address to determine the row where the target storage cell is located. The column address decoder decodes the column address to determine the column where the target storage cell is located. The sensitive amplifier senses the information stored in the target storage cell and temporarily stores the sensing result in the data buffer.

[0024] A core computation prevalent in various layers of neural network models is the linear transformation of the data to be processed based on weight data. This linear transformation includes: performing a linear transformation on the vector to be processed using the weight matrix; and performing a linear transformation on the matrix to be processed using the weight matrix. Therefore, linear transformation operations can be broken down into matrix-vector multiplication operations. A linear transformation of the vector to be processed using the weight matrix is ​​equivalent to one matrix-vector multiplication operation; a linear transformation of the matrix to be processed using the weight matrix is ​​equivalent to multiple matrix-vector multiplication operations. Furthermore, matrix-vector multiplication consists of multiplication-accumulation operations.

[0025] Figure 6 Exemplary schematic diagrams are shown in some embodiments of this application, illustrating the linear transformation of data to be processed based on weighted data. For example... Figure 6 The fully connected layer shown includes three input neurons I1, I2, and I3, and four output neurons O1, O2, O3, and O4, with output neuron O... aWith input neuron I b There are neuron connection weights W a,b Therefore, the three input neurons form vector I, and the weights connecting the twelve neurons form matrix W. The linear transformation operation performed by the fully connected layer is to transform the 3-dimensional input neuron vector I into a 4-dimensional output neuron vector O. In other words, the linear transformation operation performed by the fully connected layer can be represented as a matrix-vector multiplication operation: WI = O. Furthermore, the multiplication operation between matrix W and vector I can be broken down into four multiplication-accumulation operations: the first row of W is multiplied and accumulated with I to obtain O1; the second row of W is multiplied and accumulated with I to obtain O2; the third row of W is multiplied and accumulated with I to obtain O3; and the fourth row of W is multiplied and accumulated with I to obtain O4. Even further, each multiplication-accumulation operation includes element-wise multiplication and accumulation. For example, when multiplying and accumulating the first row of W with I to obtain O1, it is necessary to multiply W... 1,1 Multiply by I1 to get O 1,1 W 1,2 Multiply by I2 to get O 1,2 W 1,3 Multiply by I3 to get O 1,3 Then O1=O 1,1 +O 1,2 +O 1,3 .

[0026] although Figure 6 Taking a fully connected layer as an example, this explains the operation of using weight matrices to perform linear transformations on the data being processed in a neural network model. It's understandable that other neural network layers can similarly include this operation. Using weights to perform linear transformations on the data being processed is a common operation in neural network models, and many AI processor optimization goals often include improving the processor's execution speed for such linear transformation operations.

[0027] In neural network models, multiple weights are often used to calculate the same input data. For example, in the context of the previous discussion... Figure 6 The matrix-vector multiplication operation WI=O mentioned above requires each row vector in W to be multiplied and accumulated with I. For example, in the Transformer's self-attention mechanism, during the linear transformation of the embedding matrix using query weight matrices W_Q, W_K, and W_V to obtain the query (Q) matrix, key (K) matrix, and value (V) matrix, W_Q, W_K, and W_V need to be multiplied by the same embedding matrix, thus involving data from different weight matrices being calculated with the same embedding vector.

[0028] Figure 7 This illustration shows an exemplary diagram of calculating the same input data using multiple weight data in some embodiments of this application. In this example, weight data WA and weight data WB are linearly calculated with external input data IC in a linear calculator, where WA and WB are stored in different rows of a storage array. The execution of the calculation task includes two steps: data retrieval and calculation. Due to the different distances of WA and WB to the sensitive amplifier and the linear calculator, the time required for the linear calculator to retrieve WA and WB from the storage matrix differs, resulting in a difference in the time required to execute the linear calculation task of WA and IC compared to the time required to execute the linear calculation task of WB and IC. Because in concurrent processing, among multiple concurrent calculation tasks, the calculation task with a shorter execution time is forced to wait for the other to finish execution, the difference in execution time between the linear calculation tasks of WA and IC and WB and IC makes the two calculation tasks unsuitable for concurrent processing. How to improve the execution efficiency of such tasks that use weight data to perform linear calculations on input data has become a technical problem. In view of this, this application proposes a memory computing circuit, chip, and related methods to improve the efficiency of linear calculations on input data based on weight data.

[0029] Figure 8 Exemplary schematic diagrams of memory computing circuits in some embodiments of this application are shown. For example... Figure 8 As shown, in some embodiments, the in-memory computing circuit includes: a memory array including N pairs of spatially adjacent first memory regions and second memory regions, where N≥1; N linear operation modules, one of which is configured to receive data from a corresponding pair of memory regions and perform linear operations on the first data from the first memory region and the second data from the second memory region with externally input third data; wherein the linear operation module is configured on the data output side of the memory array, near its corresponding pair of memory regions.

[0030] In some embodiments, a storage array is used to store weight data of a neural network model, wherein each storage unit in the storage array is used to store one bit, and a weight data element may occupy a different number of storage units depending on the data type of the weight data. For example, a single-precision floating-point number may occupy 32 storage units, and a double-precision floating-point number may occupy 64 storage units. In some embodiments, a single storage region is used to store several weight data elements, and thus a single storage region may include several storage units.

[0031] The first data and the second data can be different data blocks in the same weight matrix, or the first data can be a data block in the first weight matrix and the second data can be a data block in the second weight matrix. In some embodiments, the first data and the second data are two row vectors in the same weight matrix. The storage array includes N pairs of first storage areas and second storage areas; the N first storage areas can be used to store the first data, for example, by splitting the first data into N parts and storing one part in each first storage area; the N second storage areas can be used to store the second data, for example, by splitting the second data into N parts and storing one part in each second storage area. In some embodiments, corresponding to the first data and the second data being split into N parts, the third data is also split into N parts and provided to N linear operation modules.

[0032] In some embodiments, the first storage region and the second storage region are spatially adjacent; for example, the first storage region and the second storage region are adjacent to each other on the left and right, that is, there are no other storage units between them, so that the first storage region and the second storage region are alternately distributed in one or more rows of the storage array; as another example, the first storage region and the second storage region are adjacent to each other on the top and bottom, that is, there are no other storage units between them, so that the first storage region and the second storage region are alternately distributed in one or more columns of the storage array.

[0033] The linear operation module can perform operations including multiplication and addition on the input data. In some embodiments, each adjacent pair of first and second storage regions corresponds to one linear operation module; for example, such as... Figure 8 As shown, the first storage area M1 and the second storage area M2 are an adjacent pair of storage areas, corresponding to the linear operation module LC1. The data stored in M1 and M2 can be input into LC1 for computation. In some embodiments, the linear operation module obtains first data from the first storage area corresponding to the module, second data from the second storage area corresponding to the module, and third data input externally, so as to perform linear operations on the first data and the third data, and on the second data and the third data. The third data can be input into the linear operation module from a high-speed cache (e.g., the L1 cache closest to the linear operation module).

[0034] In some embodiments, the linear operation modules are configured on the data output side of the storage array to improve the speed of acquiring data stored in the storage array and reduce power consumption caused by data transmission. Furthermore, each linear operation module is located close to its corresponding pair of storage regions, thereby further improving data acquisition speed and reducing transmission power consumption.

[0035] This application, by alternately storing first data and second data in a storage array and setting a linear operation module in a position adjacent to the storage area of ​​the first data and the second data, allows the same linear operation module to be used to execute linear calculation tasks involving the first data and the second data, thereby improving the consistency of the execution time of the two calculation tasks. This helps to concurrently process linear transformation operations on the same input data using data with different weights, thus improving the execution efficiency of such linear calculation tasks.

[0036] Figure 9 Exemplary schematic diagrams of memory computing circuits in some embodiments of this application are shown. For example... Figure 9 As shown, in some embodiments, the linear operation module includes: an input register configured to store third data from an external input; a first linear operator configured to perform a linear operation between first data from a first storage area and third data from the input register; and a second linear operator configured to perform a linear operation between second data from a second storage area and third data from the input register.

[0037] In these embodiments, the input register stores externally input third data; the first linear arithmetic unit receives the third data from the input register and the first data from the first storage area, and then performs linear operations on the received first data and the third data; the second linear arithmetic unit receives the third data from the input register and the second data from the second storage area, and then performs linear operations on the received second data and the third data.

[0038] In some embodiments, the first linear arithmetic unit and the second linear arithmetic unit include multipliers.

[0039] In these embodiments, the first storage area can store multiple first data elements, the second storage area can store multiple second data elements, and the input register can store multiple third data elements. The multiplier in the first linear arithmetic unit can perform element-wise multiplication of the multiple first data elements stored in the first storage area with the multiple third data elements stored in the input register. The multiplier in the second linear arithmetic unit can perform element-wise multiplication of the multiple second data elements stored in the second storage area with the multiple third data elements stored in the input register. It is understood that the essence of multiplication is a binary shift and addition process. In other embodiments, the first and second linear arithmetic units include a shift module implemented by a shift register and an adder.

[0040] In some embodiments, N≥2, the first data A is split into multiple first data blocks Ai, the second data B is split into multiple second data blocks Bi, and the externally input third data C is split into multiple third data blocks Ci, i=1~N, and the first storage area in the i-th pair of storage areas is used to store the first data blocks Ai of the first data A, and the second storage area is used to store the second data blocks Bi of the second data B; the third data blocks Ci are provided to the linear operation module corresponding to the i-th pair of storage areas.

[0041] In these embodiments, the first data A is split into N first data blocks A1~AN, which are stored in N first storage areas respectively; the second data B is split into N second data blocks B1~BN, which are stored in N second storage areas respectively; and the third data C is split into N third data blocks C1~CN, which are provided to N linear operation modules respectively.

[0042] Figure 10 Exemplary schematic diagrams are shown in some embodiments of this application, illustrating linear calculations of input data based on weighted data. For example... Figure 10 As shown, the computational operation performed is W*C=D; where W is the weight matrix, including weight vectors A and B; and C is the input data. This computational operation involves performing linear calculations on the same input data C using weight vectors A and B. According to the technical solution of this application, A is divided into N first data blocks A1~AN, B is divided into N second data blocks B1~BN, and C is divided into N third data blocks C1~CN (in... Figure 10 In the example shown, N=8). In the storage array, A and B are stored in two storage rows; the first storage row stores A1-A4 and B1-B4, with A1 and B1 stored adjacently, A2 and B2 adjacently, A3 and B3 adjacently, and A4 and B4 adjacently; the second storage row stores A5-A8 and B5-B8, with A5 and B5 adjacently, A6 and B6 adjacently, A7 and B7 adjacently, and A8 and B8 adjacently. Further, a linear operation module LCI is set up near Ai (i=1, 2, ..., N) and Bi. LCI receives Ai, Bi, and Ci, and performs multiplication and accumulation calculations on Ai and Ci, and on Bi and Ci.

[0043] Figure 11 Exemplary schematic diagrams of memory computing circuits in some embodiments of this application are shown. For example... Figure 11 As shown, in some embodiments, the in-memory computing circuit further includes: a first reduction circuit configured to reduce the results of the first linear arithmetic units from each linear arithmetic module; and a second reduction circuit configured to reduce the results of the second linear arithmetic units from each linear arithmetic module.

[0044] Correspondingly, A, B, and C are all divided into N parts. The first linear computation task of applying weighted data A to external input data C is further divided into smaller first subtasks, which are executed in the first linear arithmetic units of each linear operation module. Similarly, the second linear computation task of applying weighted data B to external input data C is divided into smaller second subtasks, which are executed in the second linear arithmetic units of each linear operation module. It is understandable that the execution result of either the first or second subtask is a local result of either the first or second linear computation task. The results from the first linear arithmetic units of each linear operation module need to be reduced, for example, by accumulation, weighted summation, finding the maximum or minimum value, to obtain the final result of the first linear computation task. Likewise, the results from the second linear arithmetic units of each linear operation module need to be reduced to obtain the final result of the second linear computation task.

[0045] Combination Figure 10 and Figure 11 Let's look at it this way: The first linear arithmetic unit in LC1~LC8 performs the linear calculation task of Ai and Ci, and needs to sum the results of the linear calculation of Ai and Ci to obtain the final result D1 of the linear calculation of A and C; similarly, the second linear arithmetic unit in LC1~LC8 performs the linear calculation task of Bi and Ci, and needs to sum the results of the linear calculation of Bi and Ci to obtain the final result D4 of the linear calculation of B and C.

[0046] In some embodiments, the data output side of the storage array includes a single side, with N pairs of storage regions evenly distributed along the single side. For example, read circuits are provided on the left, right, top, or bottom side of the storage array, thereby forming a data output side on a single side of the storage array to facilitate the acquisition of stored data from the storage array and the linear calculation of external input data based on the read data.

[0047] In some embodiments, N pairs of storage regions are evenly distributed across a single storage row. In these embodiments, the data stored in N pairs of storage regions can be read at once by selecting a row of the storage array, thereby improving the efficiency of retrieving stored data.

[0048] In some embodiments, the data output side of the storage array includes two opposing sides, with N pairs of storage regions evenly distributed along both sides. For example, read circuits are respectively provided on the left and right sides of the storage array, or on the upper and lower sides of the storage array, thereby forming data output sides on opposing sides of the storage array to facilitate the acquisition of stored data from the storage array and the linear calculation of external input data based on the read data. In some embodiments, N / 2 pairs of storage regions are located near the first output side of the storage array and are evenly distributed, while the remaining N / 2 pairs of storage regions are located near the second output side of the storage array and are evenly distributed.

[0049] In some embodiments, the storage array includes a symmetrically distributed first storage row and a second storage row, with N storage regions evenly distributed across the first and second storage rows. In these embodiments, the first storage row is located near the first output side of the storage array, the second storage row is located near the second output side of the storage array, N / 2 pairs of storage regions are evenly distributed across the first storage row, and the remaining N / 2 pairs of storage regions are evenly distributed across the second storage row.

[0050] Figure 12 Exemplary schematic diagrams of memory computing circuits in some embodiments of this application are shown. For example... Figure 12 As shown, in some embodiments, the first reduction circuit includes a left first reduction circuit and a right first reduction circuit, used to reduce the results of the first linear arithmetic unit of the corresponding linear arithmetic module; the second reduction circuit includes a left second reduction circuit and a right second reduction circuit, used to reduce the results of the second linear arithmetic unit of the corresponding linear arithmetic module.

[0051] In these embodiments, a read circuit, a linear operation module, a left-side first reduction circuit, and a left-side second reduction circuit are arranged on the left side of the storage matrix. Similarly, a read circuit, a linear operation module, a right-side first reduction circuit, and a right-side second reduction circuit are arranged on the right side of the storage matrix. The left-side first / second reduction circuit reduces the results of the first / second linear operators in the left-side linear operation module, and the right-side first / second reduction circuit reduces the results of the first / second linear operators in the right-side linear operation module. In some embodiments, the reduction operation includes summing, weighted summing, finding the minimum value, or finding the maximum value of the results from multiple first / second linear operators.

[0052] continue Figure 12 In some embodiments, the memory computing circuit further includes: a third reduction circuit configured to reduce the results from the left first reduction circuit and the right first reduction circuit; and a fourth reduction circuit configured to reduce the results from the left second reduction circuit and the right second reduction circuit.

[0053] In these embodiments, the result of the first linear arithmetic unit in the left linear operation module is first reduced by the left first reduction circuit to obtain the left first reduction result, and the result of the first linear arithmetic unit in the right linear module is reduced by the right first reduction circuit to obtain the right first reduction result. Then, the left first reduction result and the right first reduction result are reduced by the third reduction circuit to obtain the linear calculation result of the first data and the third data. Similarly, the left second reduction circuit reduces the result of the second linear arithmetic unit in the left linear operation module to obtain the left second reduction result, and the right second reduction circuit reduces the result of the second linear arithmetic unit in the right linear module to obtain the right second reduction result. Then, the left second reduction result and the right second reduction result are reduced by the fourth reduction circuit to obtain the linear calculation result of the second data and the third data.

[0054] In some embodiments, the sizes of the first and second storage regions are determined based on the processing capacity of the linear operation module in a single cycle. In these embodiments, the sizes of the first and second storage regions can be determined based on the length of input data that the linear operation module can process in one cycle, in order to fully utilize the computing power provided by the linear operation module.

[0055] In some embodiments, the first and second storage regions in each pair of storage regions support simultaneous reading of first and second data within the same memory access cycle. In these embodiments, the first and second data stored in each pair of storage regions can be read simultaneously within the same memory access cycle, thereby facilitating the simultaneous completion of linear computation tasks in which the first and second data stored in each pair of storage regions are respectively involved.

[0056] In some embodiments, N pairs of storage regions support the simultaneous reading of first data and second data within the same memory access cycle. In these embodiments, the first data and second data stored in N pairs of storage regions can be read simultaneously within the same memory access cycle, thereby facilitating the simultaneous completion of linear computation tasks involving the first data and the second data respectively.

[0057] In some embodiments, the difference in data transmission latency from each pair of storage regions to the corresponding linear operation module is less than a specified threshold. In these embodiments, a corresponding linear operation module can be located adjacent to each pair of storage regions so that the difference in data transmission latency from each pair of storage regions to the corresponding linear operation module is less than a specified threshold, which can be set to 0.01 nanoseconds or 0.02 nanoseconds, for example.

[0058] In summary, by alternately storing first data and second data in the storage array and setting linear operation modules in adjacent positions of the storage areas of the first data and second data, the same linear operation module can be used to execute linear calculation tasks involving the first data and the second data, thereby improving the consistency of the execution time of the two calculation tasks. This helps to concurrently process linear transformation operations on the same input data using data with different weights, thus improving the execution efficiency of such linear calculation tasks.

[0059] This application also provides a chip that includes the memory computing circuit according to any of the preceding embodiments.

[0060] This application also discloses a method for performing calculations using a memory-based computing circuit, wherein the memory-based computing circuit includes a memory array and N linear operation modules, where N≥1; the memory array includes N pairs of spatially adjacent first memory regions and second memory regions; the linear operation modules are configured on the data output side of the memory array, near their corresponding pair of memory regions; the method includes using the linear operation modules to: receive data from the corresponding pair of memory regions; and perform linear operations between the first data from the first memory region and the second data from the second memory region and the externally input third data, respectively.

[0061] Corresponding to the memory computing circuit disclosed in this application, this application discloses the following embodiments of a method for performing calculations using the memory computing circuit: In some embodiments, the linear operation module includes an input register, a first linear arithmetic unit, and a second linear arithmetic unit; the method further includes: using the input register to store third data from external input; using the first linear arithmetic unit to perform a linear operation between first data from a first storage area and third data from the input register; and using the second linear arithmetic unit to perform a linear operation between second data from a second storage area and third data from the input register.

[0062] In some embodiments, the first linear arithmetic unit and the second linear arithmetic unit include multipliers.

[0063] In some embodiments, where N≥2, the method further includes: splitting first data A into multiple first data blocks Ai, i=1~N; splitting second data B into multiple second data blocks Bi; splitting externally input third data C into multiple third data blocks Ci; using a first storage area in the i-th pair of storage areas to store the first data blocks Ai of the first data A; using a second storage area in the i-th pair of storage areas to store the second data blocks Bi of the second data B; and providing the third data blocks Ci to the linear operation module corresponding to the i-th pair of storage areas.

[0064] In some embodiments, the in-memory circuit further includes a first reduction circuit and a second reduction circuit; the method further includes: using the first reduction circuit to reduce the results of the first linear arithmetic unit from each linear arithmetic module; and using the second reduction circuit to reduce the results of the second linear arithmetic unit from each linear arithmetic module.

[0065] In some embodiments, the data output side of the storage array includes a single side, and N pairs of storage regions are uniformly distributed along the single side.

[0066] In some embodiments, N pairs of storage regions are evenly distributed across a single storage row.

[0067] In some embodiments, the data output side of the storage array includes two opposing sides, with N pairs of storage regions evenly distributed along both sides.

[0068] In some embodiments, the storage array includes a symmetrically distributed first storage row and a second storage row, with N pairs of storage regions evenly distributed on the first and second storage rows.

[0069] In some embodiments, the first reduction circuit includes a left first reduction circuit and a right first reduction circuit; the second reduction circuit includes a left second reduction circuit and a right second reduction circuit; the method further includes: using the left first reduction circuit and the right first reduction circuit to reduce the result of the first linear arithmetic unit of the corresponding linear arithmetic module; and using the left second reduction circuit and the right second reduction circuit to reduce the result of the second linear arithmetic unit of the corresponding linear arithmetic module.

[0070] In some embodiments, the memory circuit further includes a third reduction circuit and a fourth reduction circuit; the method further includes: using the third reduction circuit to reduce the results from the left first reduction circuit and the right first reduction circuit; and using the fourth reduction circuit to reduce the results from the left second reduction circuit and the right second reduction circuit.

[0071] In some embodiments, the method further includes: determining the size of the first storage region and the second storage region based on the single processing volume of the linear operation module.

[0072] In some embodiments, the first and second storage regions in each pair of storage regions support simultaneous reading of the first and second data in the same memory access cycle.

[0073] In some embodiments, N pairs of storage regions support the simultaneous reading of first data and second data in the same memory access cycle.

[0074] In some embodiments, the difference in data transmission delay between each pair of storage regions and the corresponding linear operation module is less than a specified threshold.

[0075] While numerous embodiments of this application have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will arise for those skilled in the art without departing from the spirit and intent of this application. It should be understood that various alternatives to the embodiments of this application described herein may be employed in the practice of this application. The appended claims are intended to define the scope of protection of this application and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A memory computing circuit, comprising: A storage array comprising N pairs of spatially adjacent first and second storage regions, where N ≥ 1; N linear operation modules, each configured to receive data from a corresponding pair of storage areas, and to perform linear operations on the first data from the first storage area and the second data from the second storage area with the third data from an external input. The linear operation module is configured on the data output side of the storage array, near its corresponding pair of storage areas.

2. The memory computing circuit according to claim 1, wherein, The linear operation module includes: Input register, configured to store third data from external input; A first linear arithmetic unit, configured to perform linear operations on first data from a first storage region and third data from the input register; and The second linear arithmetic unit is configured to perform linear operations on second data from the second storage area and third data from the input register.

3. The memory computing circuit according to claim 2, wherein, The first and second linear arithmetic units include multipliers.

4. The memory computing circuit according to any one of claims 2-3, wherein, N≥2, the first data A is split into multiple first data blocks Ai, the second data B is split into multiple second data blocks Bi, and the externally input third data C is split into multiple third data blocks Ci, i=1~N, and The first storage area in the i-th pair of storage areas is used to store the first data block Ai of the first data A, and the second storage area is used to store the second data block Bi of the second data B. The third data block Ci is provided to the linear operation module corresponding to the i-th pair of storage regions.

5. The memory computing circuit according to claim 4 further includes: The first reduction circuit is configured to reduce the results of the first linear arithmetic unit from each linear arithmetic module. The second reduction circuit is configured to reduce the results of the second linear arithmetic unit from each linear arithmetic module.

6. The memory computing circuit according to any one of claims 1-5, wherein, The data output side of the storage array includes a single side, and N pairs of storage regions are evenly distributed along the single side.

7. The memory-based computing circuit according to claim 6, wherein the N pairs of storage regions are uniformly distributed on a storage row.

8. The memory computing circuit according to claim 5, wherein, The data output side of the storage array includes two opposing sides, and N pairs of storage areas are evenly distributed along the two sides.

9. The memory computing circuit according to claim 8, wherein the memory array comprises a symmetrically distributed first memory row and a second memory row, and the N memory regions are uniformly distributed on the first memory row and the second memory row.

10. The memory computing circuit according to any one of claims 8-9, wherein, The first reduction circuit includes a left first reduction circuit and a right first reduction circuit, which are used to reduce the results of the first linear arithmetic unit of the corresponding linear arithmetic module respectively; The second reduction circuit includes a left-side second reduction circuit and a right-side second reduction circuit, which are used to reduce the results of the second linear arithmetic unit of the corresponding linear arithmetic module.

11. The memory computing circuit according to claim 10, further comprising: The third reduction circuit is configured to reduce the results from the left first reduction circuit and the right first reduction circuit; The fourth reduction circuit is configured to reduce the results from the second reduction circuit on the left and the second reduction circuit on the right.

12. The memory computing circuit according to any one of claims 1-11, wherein, The size of the first storage area and the second storage area is determined based on the single processing capacity of the linear operation module.

13. The memory computing circuit according to any one of claims 1-12, wherein, The first and second storage regions in each pair of storage regions support simultaneous reading of the first and second data in the same memory access cycle.

14. The memory computing circuit according to any one of claims 1-13, wherein, N pairs of storage regions support the simultaneous reading of the first and second data in the same memory access cycle.

15. The memory computing circuit according to any one of claims 1-14, wherein, The difference in data transmission latency between each pair of storage areas and the corresponding linear operation module is less than a specified threshold.

16. A chip comprising a memory computing circuit according to any one of claims 1-15.

17. A method for performing calculations using in-memory circuits, wherein, The in-memory computing circuit includes a storage array and N linear operation modules, where N ≥ 1; the storage array includes N pairs of spatially adjacent first and second storage regions; the linear operation modules are configured on the data output side of the storage array, near their corresponding pair of storage regions; the method includes using the linear operation modules to: Receive data from the corresponding pair of storage areas; Perform linear operations on the first data from the first storage area and the second data from the second storage area, respectively, with the third data from the external input.

18. The method according to claim 17, wherein, The linear arithmetic module includes an input register, a first linear arithmetic unit, and a second linear arithmetic unit; the method further includes: The input register is used to store externally input third data; The first linear arithmetic unit is used to perform a linear operation between first data from the first storage area and third data from the input register; and The second linear arithmetic unit is used to perform a linear operation between the second data from the second storage area and the third data from the input register.

19. The method according to claim 18, wherein, The first and second linear arithmetic units include multipliers.

20. The method according to any one of claims 18-19, wherein, N≥2, the method further includes: The first data A is split into multiple first data blocks Ai, i=1~N; The second data B is split into multiple second data blocks Bi; The externally input third data C is split into multiple third data blocks Ci; Use the first storage region in the i-th pair of storage regions to store the first data block Ai of the first data A; Use the second storage area in the i-th pair of storage areas to store the second data block Bi of the second data B; The third data block Ci is provided to the linear operation module corresponding to the i-th pair of storage regions.

21. The method according to claim 20, wherein, The memory computing circuit further includes a first reduction circuit and a second reduction circuit; the method further includes: The first reduction circuit is used to reduce the results of the first linear arithmetic unit from each linear arithmetic module; The second reduction circuit is used to reduce the results of the second linear arithmetic unit from each linear arithmetic module.

22. The method according to any one of claims 17-21, wherein, The data output side of the storage array includes a single side, and N pairs of storage regions are evenly distributed along the single side.

23. The method according to claim 22, wherein, The N pairs of storage regions are evenly distributed on a single storage row.

24. The method according to claim 21, wherein, The data output side of the storage array includes two opposing sides, and N pairs of storage areas are evenly distributed along the two sides.

25. The method according to claim 24, wherein, The storage array includes a symmetrically distributed first storage row and a second storage row, and the N pairs of storage regions are evenly distributed on the first storage row and the second storage row.

26. The method according to any one of claims 24-25, wherein, The first reduction circuit includes a left first reduction circuit and a right first reduction circuit; The second reduction circuit includes a left-side second reduction circuit and a right-side second reduction circuit; the method further includes: The first reduction circuit on the left and the first reduction circuit on the right are used to reduce the results of the first linear arithmetic unit of the corresponding linear arithmetic module. The left-side second reduction circuit and the right-side second reduction circuit are used to reduce the results of the second linear arithmetic unit of the corresponding linear arithmetic module.

27. The method according to claim 26, wherein, The memory computing circuit further includes a third reduction circuit and a fourth reduction circuit; the method further includes: The third reduction circuit is used to reduce the results from the left first reduction circuit and the right first reduction circuit; The fourth reduction circuit is used to reduce the results from the left second reduction circuit and the right second reduction circuit.

28. The method according to any one of claims 17-27, further comprising: The size of the first and second storage areas is determined based on the single processing capacity of the linear operation module.

29. The method according to any one of claims 17-28, wherein, The first and second storage regions in each pair of storage regions support simultaneous reading of the first and second data in the same memory access cycle.

30. The method according to any one of claims 17-29, wherein, N pairs of storage regions support the simultaneous reading of the first and second data in the same memory access cycle.

31. The method according to any one of claims 17-30, wherein, The difference in data transmission latency between each pair of storage areas and the corresponding linear operation module is less than a specified threshold.

Images