Storage circuit, chip and related method

Abstract

The invention discloses a storage and calculation circuit, a chip and a related method, and the storage and calculation circuit comprises a storage array which comprises N storage areas, and N is greater than or equal to 2; one linear operation module is configured to be used for receiving data from the corresponding storage area and executing linear operation of the data from the corresponding storage area and external input data; and the input and output module is configured to receive external input data, copy the external input data into N copies, and respectively provide the N copies to the N linear operation modules for operation execution. According to the technical scheme, the linear transformation operation on the same external input data according to the multiple pieces of weight data can be accelerated with high concurrency.

Description

Technical Field

[0001] This application generally relates to the field of artificial intelligence technology. More specifically, this application relates to a memory computing circuit, a chip, and related methods. Background Technology

[0002] The core computation commonly found in various layers of a neural network model is to perform a linear transformation operation on the data to be processed based on the weight data. The linear transformation operation can be broken down into matrix vector multiplication (MVM) operation, which is further composed of multiply and accumulate (MAC) operation.

[0003] Near-Memory Computing (NMC) architecture helps alleviate data handling challenges in data-intensive applications by integrating computing units on or adjacent to memory chips. In neural network models, multiple weights are often used to compute on the same input data. In such cases, the computational pattern of reading one weight from the memory array each computation cycle and multiplying and summing it with the input data reduces the computational efficiency of the NMC architecture.

[0004] In view of this, this application proposes a memory computing circuit, chip, and related methods to improve the speed of linear operations on input data based on weighted data in near-memory computing systems. Summary of the Invention

[0005] In order to at least solve one or more of the technical problems mentioned above, this application proposes solutions for memory computing circuits, chips, and methods for performing calculations using memory computing circuits in several aspects.

[0006] In a first aspect, this application provides a memory computing circuit, comprising: a memory array including N memory regions, where N ≥ 2; N linear operation modules, each linear operation module configured to receive data from a corresponding memory region and perform linear operations between the data from the corresponding memory region and external input data; and an input / output module configured to receive the external input data and copy the external input data into N copies, respectively providing them to the N linear operation modules for execution of operations.

[0007] In a second aspect, this application provides a chip including the memory computing circuitry described in the first aspect.

[0008] In a third aspect, this application provides a method for performing calculations using a memory-based computing circuit, wherein the memory-based computing circuit includes a storage array, N linear operation modules, and an input / output module; N ≥ 2; the storage array includes N storage regions; the method includes: using one of the linear operation modules to: receive data from a corresponding storage region; perform a linear operation between the data from the corresponding storage region and external input data; using the input / output module to: receive the external input data; copy the external input data and provide it to each of the N linear operation modules for execution of the operation.

[0009] By using the memory computing circuits, chips, and methods provided above, embodiments of this application divide the memory array into N memory regions, set up a linear operation module for each memory region, and copy external input data into N copies and provide them to the N linear operation modules respectively. This allows linear operations to be performed on the same external input data based on the weight data stored in the N memory regions, thereby accelerating the linear transformation operation of the same external input data based on multiple weight data with a high degree of concurrency. Attached Figure Description

[0010] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily understood by reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of this application are illustrated by way of example and not limitation, and the same or corresponding reference numerals denote the same or corresponding parts, wherein: Figure 1 An exemplary structural diagram of a storage array in some embodiments of this application is shown; Figure 2 An exemplary structural diagram of a ReRAM memory cell in some embodiments of this application is shown; Figure 3 A schematic diagram illustrating an exemplary relationship between the resistance value of the resistive switching layer and the storage state in some embodiments of this application is shown; Figure 4 Exemplary structural diagrams of storage arrays in other embodiments of this application are shown; Figure 5 An exemplary schematic diagram is shown in some embodiments of this application, illustrating the linear transformation of data to be processed based on weighted data; Figure 6 This illustration shows an exemplary schematic diagram of using a near-memory computing system to perform a linear transformation operation on input data based on weight data in some embodiments of this application; Figure 7a Exemplary schematic diagrams of memory computing circuits in some embodiments of this application are shown; Figure 7b Exemplary schematic diagrams of memory computing circuits in some embodiments of this application are shown; Figure 7c Exemplary schematic diagrams of memory computing circuits in some embodiments of this application are shown; Figure 8 Exemplary schematic diagrams of memory computing circuits in some embodiments of this application are shown; Figure 9 Exemplary schematic diagrams of linear operation units in some embodiments of this application are shown; Figure 10 Exemplary schematic diagrams of memory computing circuits in some embodiments of this application are shown; Figure 11 An exemplary schematic diagram of a memory computing circuit in some embodiments of this application is shown. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0012] It should be understood that the terms "comprising" and "including" as used in the specification and claims of this application indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0013] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used in this specification and claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this specification and claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations.

[0014] As used in this specification and claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0015] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise expressly specified. "Several" means one or more, unless otherwise expressly specified.

[0016] The specific embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0017] Near-memory computing architectures help alleviate data movement problems faced by data-intensive applications by integrating computing units on or adjacent to the storage array.

[0018] Figure 1 Exemplary structural diagrams of storage arrays in some embodiments of this application are shown. For example... Figure 1 As shown, the memory array 100 consists of multiple rows and columns of memory sections, accessed via horizontal and vertical wires, commonly referred to as word lines (WL), source lines (SL), and bit lines (BL). A memory section can be deployed between two source lines and between adjacent bit lines and word lines. Each memory section includes memory cells and switching units. The memory cells can be Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Resistive Random Access Memory (ReRAM), Spin-Transfer Torque Magnetoresistive Random Access Memory (STT-MRAM), Phase Change Memory (PCM), Ferroelectric Random Access Memory (FeRAM), or Flash Memory.

[0019] Figure 2 Exemplary structural diagrams of ReRAM memory cells in some embodiments of this application are shown. Figure 2As shown, a ReRAM memory cell includes an upper electrode, a resistive switching layer, and a lower electrode. The upper and lower electrodes can be metal electrodes, such as those made of aluminum (Al), titanium (Ti), or copper (Cu). The resistive switching layer can be a metal oxide material, such as zinc oxide (ZnO), magnesium oxide (MgO), or zirconium oxide (ZrO). x Titanium oxide (TiO) x Materials include aluminum oxide (Al2O3), tantalum oxide (Ta2O5), and hafnium oxide (HfO2). The resistive switching layer can change its resistance value according to the voltage across it.

[0020] Figure 3 This diagram illustrates an exemplary relationship between the resistance value of the resistive switching layer and the storage state in some embodiments of this application. For example... Figure 3 As shown, the positive direction of the resistance axis is the direction of increasing resistance. LR1 is the minimum resistance in the low-resistance state, LR2 is the maximum resistance in the low-resistance state, HR1 is the minimum resistance in the high-resistance state, and HR2 is the maximum resistance in the high-resistance state. Therefore, [LR1, LR2] is the low-resistance range of the resistive switching layer, and [HR1, HR2] is the high-resistance range of the resistive switching layer. In some embodiments, a high-voltage pulse can be applied to the resistive switching layer to form a conductive wire between the upper and lower electrodes, causing the resistive switching layer to exhibit a low-resistance state (LRS). In some embodiments, a low-voltage pulse can be applied to the resistive switching layer to break the conductive wire between the upper and lower electrodes, causing the resistive switching layer to become a high-resistance state (HRS). Moreover, even after the memory cell loses power, the resistive switching layer will maintain its resistance value. In a computing system, data can be represented using fixed-point numbers or floating-point numbers with multiple bits. In some embodiments, the storage array 100 uses each storage cell to store the value 0 / 1 of each bit, and multiple storage cells correspond to one data element; for example, when the storage cell is a ReRAM storage cell, the value 1 can be stored in the low resistance state of the resistive switching layer and the value 0 can be stored in the high resistance state of the resistive switching layer.

[0021] Back Figure 1 The switching unit can be a metal-oxide-semiconductor field-effect transistor (MOSFET). When the memory cell is a ReRAM memory cell and a MOSFET is used as the switching unit, the gate of the MOSFET is connected to the word line, and the current flowing to the memory cell can be controlled by adjusting the word line voltage. The source of the MOSFET is connected to the source line, and the drain is connected to the lower electrode of the memory cell. The upper electrode of the memory cell is connected to the bit line. The switching unit can be used to control access to the memory cell.

[0022] Figure 4 Exemplary structural diagrams of storage arrays from other embodiments of this application are shown. For example... Figure 4 As shown, the memory array 100 includes word lines and bit lines. A memory section can be deployed within the area formed by two adjacent word lines and two adjacent bit lines. The memory section includes memory cells and a selector. The selector can be a bidirectional diode-type selector or other nonlinear selection devices, such as a bidirectional threshold switch (OTS) or a metal-insulator transition (MIT). The selector is used to control access to the series-connected memory cells.

[0023] A core computation prevalent in various layers of neural network models is the linear transformation of the data to be processed based on weight data. This linear transformation includes: performing a linear transformation on the vector to be processed using the weight matrix; and performing a linear transformation on the matrix to be processed using the weight matrix. Therefore, linear transformation operations can be broken down into matrix-vector multiplication operations. A linear transformation of the vector to be processed using the weight matrix is ​​equivalent to one matrix-vector multiplication operation; a linear transformation of the matrix to be processed using the weight matrix is ​​equivalent to multiple matrix-vector multiplication operations. Furthermore, matrix-vector multiplication consists of multiplication-accumulation operations.

[0024] Figure 5 Exemplary schematic diagrams are shown in some embodiments of this application, illustrating the linear transformation of data to be processed based on weighted data. For example... Figure 5 The fully connected layer shown includes three input neurons I1, I2, and I3, and four output neurons O1, O2, O3, and O4, with output neuron O... a With input neuron I b There are neuron connection weights W a,b Therefore, the three input neurons form vector I, and the weights connecting the twelve neurons form matrix W. The linear transformation operation performed by the fully connected layer is to transform the 3-dimensional input neuron vector I into a 4-dimensional output neuron vector O. In other words, the linear transformation operation performed by the fully connected layer can be represented as a matrix-vector multiplication operation: WI = O. Furthermore, the multiplication operation between matrix W and vector I can be broken down into four multiplication-accumulation operations: the first row of W is multiplied and accumulated with I to obtain O1; the second row of W is multiplied and accumulated with I to obtain O2; the third row of W is multiplied and accumulated with I to obtain O3; and the fourth row of W is multiplied and accumulated with I to obtain O4. Even further, each multiplication-accumulation operation includes element-wise multiplication and accumulation. For example, when multiplying and accumulating the first row of W with I to obtain O1, it is necessary to multiply W...1,1 Multiply by I1 to get O 1,1 W 1,2 Multiply by I2 to get O 1,2 W 1,3 Multiply by I3 to get O 1,3 Then O1=O 1,1 +O 1,2 +O 1,3 .

[0025] although Figure 5 Taking a fully connected layer as an example, this explains the operation of using weight matrices to perform linear transformations on the data being processed in a neural network model. It's understandable that other neural network layers can similarly include this operation. Using weights to perform linear transformations on the data being processed is a common operation in neural network models, and many AI processor optimization goals often include improving the processor's execution speed for such linear transformation operations.

[0026] Figure 6 Exemplary schematic diagrams are shown in some embodiments of this application, illustrating the use of a near-memory computing system to perform linear transformation operations on input data based on weight data. In some embodiments, the memory array 100 stores the weight data of the neural network, and peripheral circuitry can retrieve data from the memory array 100 based on externally input address signals. Figure 6 As shown, the peripheral circuit of the storage array 100 includes a row address decoder, a column address decoder, a sensitive amplifier, and an output buffer. The externally input address signal is divided into a row address and a column address, which are used to locate the target storage cell where data needs to be read. The row address and column address are transmitted to the row address decoder and the column address decoder, respectively. The row address decoder decodes the row address to determine the row where the target storage cell is located. The column address decoder decodes the column address to determine the column where the target storage cell is located. The sensitive amplifier senses the information stored in the target storage cell and temporarily stores the sensing result in the data buffer.

[0027] continue Figure 6The weight data can be stored row by row in the storage array 100, that is, a row of storage cells or multiple consecutive rows of storage cells in the storage array 100 can be used to store a row vector of the weight matrix. When performing a linear transformation on the input data according to the weight data, a row of storage cells in the storage array can be read at once, thereby accelerating the process of obtaining a row vector of the weight matrix. The calculation unit includes an adder and multiple multipliers. After obtaining the weight row vector from the output buffer and the input data from the cache memory (e.g., the L1 cache closest to the calculation unit), the multiplier in the calculation unit is used to multiply the weight row vector and the input data element by element to obtain multiple product results. The adder in the calculation unit is used to accumulate the multiple product results to obtain a multiply-accumulated result, thereby completing the multiply-accumulate operation of the weight row vector and the input data.

[0028] In neural network models, multiple weights are often used to calculate the same input data. For example, in the context of the previous discussion... Figure 5 The matrix-vector multiplication operation WI=O mentioned above requires each row vector in W to be multiplied and accumulated with I. For example, in the Transformer's self-attention mechanism, during the linear transformation of the embedding matrix using query weight matrices W_Q, W_K, and W_V to obtain the query (Q) matrix, key (K) matrix, and value (V) matrix, W_Q, W_K, and W_V need to be multiplied by the same embedding matrix, thus involving data from different weight matrices being calculated with the same embedding vector. Because... Figure 6 The calculation method shown allows only one weight data point to be read from the storage array per calculation cycle, followed by multiplication and summation of the read weight data and the input data. In scenarios where multiple weight data points are used to calculate the same input data, based on... Figure 6 The computational model shown requires multiple computation cycles to complete the calculation of multiple weight data with the same input data, thus reducing computational efficiency. In view of this, this application proposes a memory computing circuit, chip, and related methods to improve the speed of linear operations on input data based on weight data in near-memory computing systems.

[0029] Figure 7a Exemplary schematic diagrams of memory computing circuits in some embodiments of this application are shown. For example... Figure 7aAs shown, in some embodiments, the in-memory computing circuit includes: a storage array including N storage areas, where N≥2; N linear operation modules, each configured to receive data from a corresponding storage area and perform linear operations between the data from the corresponding storage area and external input data; and an input / output module configured to receive external input data and copy the external input data into N copies, providing each copy to one of the N linear operation modules for execution of operations.

[0030] In these embodiments, the storage array is divided into multiple storage regions, each of which can be used to store different blocks of the same weight matrix or blocks from different weight matrices. A one-to-one correspondence can be formed between the N linear operation modules and the N storage regions; that is, each storage region is configured with a corresponding linear operation module, and the linear operation between the weight data stored in a storage region and the external input data can be performed by the linear operation module corresponding to that storage region.

[0031] continue Figure 7a The input / output module receives external input data and copies it into N copies, providing each copy to one of the N linear operation modules for computation. In some embodiments, after acquiring external input data, the input / output module sends the external input data to each linear operation module, so that each linear operation module obtains a copy of the external input data. In other embodiments, the in-memory circuit further includes N register groups. After acquiring external input data, the input / output module stores the external input data into each of the N register groups, so that each register group stores a copy of the external input data. Furthermore, there is a one-to-one correspondence between the N linear operation modules and the N register groups, so that each linear operation module can obtain a copy of the external input data from the corresponding register group.

[0032] It is understood that this application divides the storage array into N storage areas, sets up a linear operation module for each storage area, and copies the external input data into N copies and provides them to the N linear operation modules respectively. This allows linear operations on the same external input data to be performed based on the weight data stored in the N storage areas, thereby accelerating the linear transformation operation of the same external input data based on multiple weight data with higher concurrency.

[0033] exist Figure 7a In the example shown, the storage array comprises N horizontally distributed storage regions. For example... Figure 7b As shown, in some other embodiments, the storage array includes N storage regions arranged vertically. For example... Figure 7cAs shown, in other embodiments, the storage array includes N storage regions distributed in a grid. In particular, the storage array may include two columns, each column having N / 2 vertically distributed storage regions.

[0034] In some embodiments, a storage array is used to store weight data of a neural network model, wherein each storage unit in the storage array is used to store one bit, and a weight data element may occupy a different number of storage units depending on the data type of the weight data. For example, a single-precision floating-point number may occupy 32 storage units, and a double-precision floating-point number may occupy 64 storage units. In some embodiments, a single storage region is used to store several weight data elements, and thus a single storage region may include several storage units.

[0035] Figure 8 Exemplary schematic diagrams of memory computing circuits in some embodiments of this application are shown. For example... Figure 8 As shown, in some embodiments, each storage region includes M sub-regions, where M ≥ 1; each linear operation module includes M linear operation units configured to perform linear operations on data from the corresponding M sub-regions and corresponding external input data.

[0036] exist Figure 8 In the example shown, the storage region includes M (M=4) sub-regions distributed vertically. In other embodiments, the storage region may include M sub-regions distributed horizontally or in a grid pattern; this application is not limited in this regard. In some embodiments, based on the one-to-one correspondence between the storage region and the linear operation module, there is also a one-to-one correspondence between the sub-regions in the storage region and the linear operation units in the linear operation module. That is, each sub-region is configured with a linear operation unit, and the linear operation between the data stored in the sub-region and the external input data is performed by the linear operation unit corresponding to that sub-region.

[0037] In some embodiments, data W in a single storage area is split into M data blocks and stored in M ​​sub-regions; external input data C provided to the linear operation module is split into M external data blocks and provided to M linear operation units respectively.

[0038] In these embodiments, the linear computation task of data W in the storage area and external input data C is broken down into smaller subtasks. Each subtask includes linear operations on data blocks of W and data blocks of C. Thus, the task of using the linear operation module to perform calculations on the data stored in the storage area and external input data can be executed through finer-grained computational operations, further improving computational concurrency.

[0039] Figure 9Exemplary schematic diagrams of linear operation units in some embodiments of this application are shown. For example... Figure 9 As shown, in some embodiments, the linear operation unit includes: an input register configured to store a corresponding external input data block; and a linear operator configured to perform linear operations on the data block from the corresponding sub-region and the external input data block from the input register.

[0040] In these embodiments, the external input data C provided to the linear operation module is divided into M external input data blocks, and the input register in each linear operation unit is used to store each external input data block; the data W in a single storage area is divided into M data blocks, and after each linear operation unit obtains the corresponding data block, the linear operation unit performs linear operations between the data block from the sub-region and the external input data block.

[0041] In some embodiments, the linear arithmetic unit includes a multiplier that can perform element-wise multiplication of data blocks from a subregion with external input data blocks. It is understood that multiplication is essentially a binary shift-and-add process. In other embodiments, the linear arithmetic unit includes a shift module implemented by a shift register and an adder.

[0042] Figure 10 Exemplary schematic diagrams of memory computing circuits in some embodiments of this application are shown. For example... Figure 10 As shown, in some embodiments, the in-memory circuit further includes: N reduction modules, each reduction module being configured to reduce the results of M linear operation units in a linear operation module.

[0043] As mentioned earlier, the task of using linear arithmetic modules to perform calculations on the data stored in the storage area and external input data is broken down into smaller-scale computational tasks, which are then executed by the linear arithmetic units. Each small-scale computational task can be considered a local computation result. To obtain the final computational result of the data stored in the storage area and the external input data, it is necessary to reduce the results of multiple linear arithmetic units. In some embodiments, the results of multiple linear arithmetic units are summed, weighted summed, maximized, or minimized to obtain the final computational result. In other embodiments, the results of multiple linear arithmetic units are concatenated to obtain the final computational result.

[0044] In some embodiments, the size of the sub-region is determined based on the amount of data processed in a single operation by a linear arithmetic unit.

[0045] In these embodiments, the size of the sub-region can be determined based on the length of the input data that the linear operation unit can process in one computation cycle, so as to make full use of the computing power provided by the linear operation unit. For example, if the linear operation unit can process input data with L data elements in one computation cycle, then the size of the sub-region can be set to be able to store L data elements.

[0046] Figure 11 Exemplary schematic diagrams of memory computing circuits in some embodiments of this application are shown. For example... Figure 11 As shown, in some embodiments, the storage array includes two opposing data output sides, with N storage regions symmetrically distributed on the two data output sides, where N is an even number.

[0047] As mentioned earlier, the storage array is divided into N storage regions. Figure 11 In the example shown, the storage array has data output sides on both the left and right sides. Of the N storage regions, N / 2 are located on the left side of the array, and the remaining N / 2 are located on the right side. Each data output side can be configured with a read circuit to read the data stored in the storage cells of the array. The read circuit includes a row address decoder, a column address decoder, a sensitive amplifier, and an output buffer. In some embodiments, each data output side is used to read the data stored in the storage cells closer to that side, thus allowing the two data output sides to read data from different regions of the array while maintaining high read speed and accuracy.

[0048] continue Figure 11 In some embodiments, the storage array includes a symmetrically distributed first storage row and a second storage row, wherein the N storage regions are symmetrically distributed on the first storage row and the second storage row.

[0049] like Figure 11 As further illustrated, the left and right data output sides of the in-memory computing circuit are respectively equipped with a left linear operation module and a right linear operation module. Each linear operation module includes multiple linear arithmetic units and a reduction module. External input data stored in the L1 cache is sent to the two linear operation modules and subjected to linear operations with the data stored in the first and second storage lines. Specifically, the external input data is split into input data blocks and input to the linear arithmetic units in the linear operation modules. The data in the first and second storage lines is split into storage data blocks and input to the linear arithmetic units in the linear operation modules. After the linear arithmetic units perform calculations on the input data blocks and storage data blocks to obtain intermediate results, the reduction module reduces the intermediate results.

[0050] In some embodiments, M sub-regions within a single storage region support simultaneous data reading in the same memory access cycle.

[0051] In these embodiments, data stored in M ​​sub-regions can be read simultaneously in the same memory access cycle, so that the M linear operation units in the linear operation module can simultaneously perform linear operations on external data blocks and sub-region data.

[0052] In some embodiments, the difference in data transmission latency between each sub-region in a single storage region and its corresponding linear operation unit is less than a specified threshold.

[0053] In these embodiments, a corresponding linear operation unit can be set in the vicinity of each sub-region so that the difference in data transmission delay from each sub-region to the corresponding linear operation unit in a single storage region is less than a specified threshold, which can be set to 0.01 nanoseconds or 0.02 nanoseconds, for example.

[0054] In some embodiments, N storage regions support simultaneous data reading in the same memory access cycle.

[0055] In these embodiments, data stored in N storage areas can be read simultaneously in the same memory access cycle, so that N linear operation modules can simultaneously perform linear operations on external data and data stored in each storage area.

[0056] In some embodiments, the difference in data transmission latency between each storage area and the corresponding linear operation module is less than a specified threshold.

[0057] In these embodiments, a corresponding linear operation module can be set in a location adjacent to each storage area so that the difference in data transmission delay between each storage area and the corresponding linear operation module is less than a specified threshold. For example, the threshold can be set to 0.01 nanoseconds or 0.02 nanoseconds.

[0058] In summary, this embodiment divides the storage array into N storage regions, sets up a linear operation module in each storage region, and copies the external input data into N copies and provides them to the N linear operation modules respectively. This allows for linear operations on the same external input data based on the weighted data stored in each of the N storage regions, accelerating the linear transformation operation on the same external input data based on multiple weighted data points with higher concurrency. Furthermore, the linear calculation task for data W in the storage region and external input data C is broken down into smaller subtasks. Each subtask includes linear operations on data blocks of W and C. Therefore, the task of calculating the data stored in the storage region and the external input data using the linear operation modules can be executed through finer-grained calculation operations, further improving computational concurrency.

[0059] This application also provides a chip that includes the memory computing circuit according to any of the preceding embodiments.

[0060] This application also discloses a method for computation using in-memory computing circuits, wherein the in-memory computing circuits include a storage array, N linear operation modules, and an input / output module; N ≥ 2; the storage array includes N storage areas; the method includes: using a linear operation module to: receive data from the corresponding storage area; perform linear operations between the data from the corresponding storage area and external input data; using the input / output module to: receive external input data; copy the external input data and provide it to the N linear operation modules for execution.

[0061] Corresponding to the memory computing circuit disclosed in this application, this application discloses the following embodiments of a method for performing calculations using the memory computing circuit: In some embodiments, each storage region includes M sub-regions, where M ≥ 1; each linear operation module includes M linear operation units; the method further includes using the linear operation units to perform linear operations between data from the corresponding M sub-regions and corresponding external input data.

[0062] In some embodiments, the method further includes: splitting data W in a single storage area into M data blocks and storing them in M ​​sub-regions; and splitting external input data C provided to the linear operation module into M external input data blocks and providing them to M linear operation units respectively.

[0063] In some embodiments, the linear operation unit includes an input register and a linear arithmetic unit; the method further includes: using the input register to store a corresponding external input data block; and using the linear arithmetic unit to perform a linear operation between the data block from the corresponding sub-region and the external input data block from the input register.

[0064] In some embodiments, the linear arithmetic unit includes a multiplier.

[0065] In some embodiments, the in-memory circuit further includes N reduction modules; the method further includes using each reduction module to reduce the results of M linear operation units in a linear operation module.

[0066] In some embodiments, the method further includes determining the size of the sub-region based on the amount of data processed in a single operation of a linear arithmetic unit.

[0067] In some embodiments, the storage array includes two opposing data output sides, with N storage regions symmetrically distributed on the two data output sides, where N is an even number.

[0068] In some embodiments, the storage array includes a symmetrically distributed first storage row and a second storage row, with N storage regions symmetrically distributed on the first and second storage rows.

[0069] In some embodiments, M sub-regions within a single storage region support simultaneous data reading within the same memory access cycle.

[0070] In some embodiments, the difference in data transmission latency between each sub-region in a single storage region and its corresponding linear operation unit is less than a specified threshold.

[0071] In some embodiments, N storage regions support simultaneous data reading in the same memory access cycle.

[0072] In some embodiments, the difference in data transmission latency between each storage region and the corresponding linear operation module is less than a specified threshold.

[0073] While numerous embodiments of this application have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will arise for those skilled in the art without departing from the spirit and intent of this application. It should be understood that various alternatives to the embodiments of this application described herein may be employed in the practice of this application. The appended claims are intended to define the scope of protection of this application and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. A memory computing circuit, comprising: A storage array comprising N storage regions, where N ≥ 2; There are N linear operation modules. Each linear operation module is configured to receive data from the corresponding storage area and perform linear operations between the data from the corresponding storage area and external input data. as well as The input / output module is configured to receive the external input data and to copy the external input data into N copies, which are then provided to the N linear operation modules for execution.

2. The memory computing circuit according to claim 1, wherein, Each storage region consists of M sub-regions, where M ≥ 1; Each linear operation module includes M linear operation units, configured to perform linear operations on data from the corresponding M sub-regions and corresponding external input data.

3. The memory computing circuit according to claim 2, wherein, Data W in a single storage area is split into M data blocks and stored in the M sub-areas; The external input data C provided to the linear operation module is divided into M external input data blocks, which are then provided to the M linear operation units respectively.

4. The memory computing circuit according to claim 3, wherein, The linear operation unit includes: Input registers, configured to store corresponding external input data blocks; and A linear arithmetic unit configured to perform linear operations on data blocks from corresponding subregions and external input data blocks from the input register.

5. The memory computing circuit according to claim 4, wherein, The linear arithmetic unit includes a multiplier.

6. The memory computing circuit according to any one of claims 2-5 further comprises: There are N reduction modules, each configured to reduce the results of M linear operation units in a linear operation module.

7. The memory-based computing circuit according to any one of claims 2-6, wherein the size of the sub-region is determined according to the single processing capacity of the linear arithmetic unit.

8. The memory-based computing circuit according to any one of claims 2-7, wherein the memory array comprises two opposing data output sides, and the N memory regions are symmetrically distributed on the two data output sides, where N is an even number.

9. The memory computing circuit according to claim 8, wherein the memory array comprises a first memory row and a second memory row that are symmetrically distributed, and the N memory regions are symmetrically distributed on the first memory row and the second memory row.

10. The memory computing circuit according to any one of claims 2-9, wherein, The M sub-regions within a single storage region support simultaneous data reading within the same memory access cycle.

11. The memory computing circuit according to any one of claims 2-10, wherein, The difference in data transfer latency between each sub-region and the corresponding linear operation unit in a single storage area is less than a specified threshold.

12. The memory computing circuit according to any one of claims 1-11, wherein, The N storage areas support simultaneous data reading within the same memory access cycle.

13. The memory computing circuit according to any one of claims 1-12, wherein, The difference in data transmission latency between each storage area and the corresponding linear operation module is less than a specified threshold.

14. A chip comprising a memory computing circuit according to any one of claims 1-13.

15. A method for performing calculations using in-memory circuits, wherein, The in-memory computing circuit includes a storage array, N linear arithmetic modules, and an input / output module; N≥2; the storage array comprises N storage regions; the method includes: Use one of the aforementioned linear operation modules to: Receive data from the corresponding storage area; Perform linear operations on data from the corresponding storage area and external input data; Use the input / output module to: Receive the external input data; The external input data is copied and provided to the N linear operation modules for execution.

16. The method according to claim 15, wherein, Each storage region includes M sub-regions, where M ≥ 1; each linear operation module includes M linear operation units; the method further includes using the linear operation units to perform linear operations between data from the corresponding M sub-regions and corresponding external input data.

17. The method of claim 16, further comprising: The data W in a single storage area is split into M data blocks and stored in the M sub-areas; The external input data C provided to the linear operation module is split into M external input data blocks, which are then provided to the M linear operation units respectively.

18. The method according to claim 17, wherein, The linear arithmetic unit includes an input register and a linear arithmetic unit; the method further includes: The input register is used to store the corresponding external input data block; The linear arithmetic unit is used to perform linear operations on data blocks from the corresponding sub-regions and external input data blocks from the input register.

19. The method according to claim 18, wherein, The linear arithmetic unit includes a multiplier.

20. The method according to any one of claims 16-19, wherein, The in-memory circuit further includes N reduction modules; the method further includes using each of the reduction modules to reduce the results of M linear operation units in a linear operation module.

21. The method according to any one of claims 16-20, further comprising: The size of the sub-region is determined based on the single processing capacity of the linear arithmetic unit.

22. The method according to any one of claims 16-21, wherein, The storage array includes two opposing data output sides, and the N storage regions are symmetrically distributed on the two data output sides, where N is an even number.

23. The method according to claim 22, wherein, The storage array includes a first storage row and a second storage row that are symmetrically distributed, and the N storage regions are symmetrically distributed on the first storage row and the second storage row.

24. The method according to any one of claims 16-23, wherein, The M sub-regions within a single storage region support simultaneous data reading within the same memory access cycle.

25. The method according to any one of claims 16-24, wherein, The difference in data transfer latency between each sub-region and the corresponding linear operation unit in a single storage area is less than a specified threshold.

26. The method according to any one of claims 15-25, wherein, The N storage areas support simultaneous data reading within the same memory access cycle.

27. The method according to any one of claims 15-26, wherein, The difference in data transmission latency between each storage area and the corresponding linear operation module is less than a specified threshold.

Images