Control method, internal memory and related chip system

Abstract

Disclosed in the embodiments of the present application are a control method, an internal memory and a related chip system which are provided in the embodiments of the present application, wherein the control method is applied to an internal memory, the internal memory is coupled to a processor, the internal memory comprises a plurality of banks, and each bank comprises M rows * N columns of storage units, M and N being integers greater than 0. The method comprises: receiving a first instruction sent by a processor, where the first instruction is used to indicate the address of a starting column j and the number k of columns, k being an integer greater than 1 but less than or equal to N; on the basis of the first instruction, determining a target bank and a target row; and in response to the first instruction, reading target data, wherein the target data is data stored in the j-th column to the (j+k-1)-th column of the target row in the target bank. By means of implementing the embodiments of the present application, the data reading efficiency and the computing performance can be improved, thereby improving the user experience.

Description

A control method, an internal memory and a related chip system

[0001] The present application claims priority to the Chinese patent application No. 202411343509.6, filed on September 24, 2024, entitled "A control method, an internal memory and a related chip system", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of intelligent terminals, and in particular to a control method, an internal memory and a related chip system. BACKGROUND

[0003] With the development of artificial intelligence technology (AI), generative AI is rising, such as ChatGPT, etc. Deploying large models on the terminal side for AI inference to improve user experience and productivity has become one of the main competitiveness of the terminal. There is a large amount of data in the large model inference process that needs to be moved from the internal memory to the processing units (PU), or central processing units (CPU), or neural processing units (NPU), or graphics processing units (GPU) for calculation, resulting in the main bottleneck of the current terminal side AI inference performance being concentrated in the data reading efficiency of the internal memory, i.e. when reading data from the storage area (bank) of the internal memory to the calculation unit for processing, the low data reading efficiency will reduce the computing performance of the terminal. Therefore, how to improve the data reading efficiency and improve the computing performance is a problem to be solved. SUMMARY

[0004] The embodiments of the present application provide a control method, an internal memory and a related chip system to improve the data reading efficiency, improve the computing performance, and further improve the user experience.

[0005] In a first aspect, the embodiments of the present application provide a control method applied to an internal memory, the internal memory being coupled with a processor, the internal memory comprising a plurality of storage areas (Banks), each Bank comprising M rows*N columns of storage units, M and N being integers greater than 0; the method comprising: receiving a first instruction sent by the processor, the first instruction being used to indicate a starting column address j and a column number k; k being an integer greater than 1 and less than or equal to N; determining a target Bank and a target row according to the first instruction; in response to the first instruction, reading target data, the target data being the data stored in the jth column to the (j+k-1)th column of the target row in the target Bank.

[0006] In the embodiment of the present application, the internal memory includes a plurality of Banks, each Bank can include M rows*N columns of storage units, and each storage unit can be used to store data. In some computing scenarios, in order to improve computing efficiency, the processor will place data into the storage units of the same row in computing order one by one. When data needs to be taken from the internal memory for computing, the processor will send a first instruction to the internal memory, the first instruction is a data reading instruction proposed in the present application, the first instruction can be used to indicate a starting column address j and a column number k in the M rows*N columns of storage units. After the internal memory receives the first instruction, the internal memory can read the data stored in the jth column storage unit to the (j+k-1)th column storage unit of the ith row (i.e. the target row) of the target Bank based on j and k. In the prior art, after activating a certain row of data in the Bank, the processor needs to issue multiple instructions (each instruction includes only one column address) to read multiple data for computing, which results in low data reading efficiency, and the instruction address bus is idle for a short time, making it difficult to perform multi-task concurrent operation. In the present application, since the first instruction can be used to indicate the starting column address j and the column number k, the internal memory can automatically read k columns of data starting from the jth column based on the starting column address j and the column number k, and the processor can read multiple columns of data by issuing one instruction, thereby improving data reading efficiency and computing performance.

[0007] In some embodiments, the internal memory further includes a mode register, the mode register includes a starting column address field and a column number field, j is stored in the starting column address field and k is stored in the column number field; j is obtained from the starting column address field and k is obtained from the column number field according to the first instruction.

[0008] In the embodiment of the present application, the first instruction can be a row activation instruction, and the data reading efficiency is improved by cooperation of the first instruction and the mode register. When the processor sends the first instruction to the internal memory, the internal memory identifies the first instruction and determines j and k from the starting column address field and the column number field of the mode register. Then, the internal memory can automatically read k columns of data starting from the jth column based on the starting column address j and the column number k in response to the first instruction, avoiding the processor needing to issue multiple instructions to read multiple columns of data in the same row of storage units, thereby improving data reading efficiency, improving computing performance, and further improving user experience.

[0009] In some embodiments, the mode register further includes a configuration mode field, if the configuration mode field is a first preset value, it is determined that j=0 and k=N-1; if the configuration mode field is a second preset value, j is obtained from the starting column address field and k is obtained from the column number field.

[0010] In the embodiment of the present application, the mode register can include a configuration mode field. When the configuration mode field is a first preset value, such as 0, the default mode can be entered, j=0 and k=N-1 are determined, and thus a whole row of data is automatically and continuously read out. When the configuration mode field is a second preset value, such as 1, the custom mode can be entered, j is obtained from the starting column address field, and k is obtained from the column number field, and thus the data stored from the jth column to the (j+k-1)th column is automatically and continuously read out.

[0011] In some embodiments, the mode register further includes a target Bank field; the target Bank and the target row are determined according to the first instruction, specifically: the target Bank is determined from the target Bank field according to the first instruction; and the target row is determined according to the first instruction.

[0012] In the embodiment of the present application, the mode register further includes a target Bank field, which can record the related information of the Bank to be activated, such as the Bank address or the index of the Bank to be activated, etc., so that the address of the Bank to be activated does not need to be carried in the first instruction, and the Bank to be activated is determined from the target Bank field of the mode register by identifying the first instruction.

[0013] In some embodiments, the target Bank field includes a plurality of target Banks.

[0014] In the embodiment of the present application, the target Bank field can include the information of a plurality of Banks to be activated, such as a plurality of Bank addresses or a plurality of indexes of Banks to be activated, etc. After the processor issues the first instruction, the first instruction can be identified, a plurality of Banks to be activated can be determined from the mode register, and a plurality of Banks can be activated at the same time.

[0015] In some embodiments, before receiving the first instruction, configuration information sent by the processor is received, the configuration information including j and k; j is stored in the starting column address field, and k is stored in the column number field.

[0016] In the embodiment of the present application, before receiving the first instruction, the processor receives a target task, can determine j and k according to the target task, and can first send configuration information to the internal memory to configure the mode register in the internal memory, so that j and k are pre-configured in the target field (i.e. the starting column address field and the column number field) of the mode register. When the target field of the mode register is pre-configured with j and k, the processor sends the first instruction to the internal memory, and the internal memory identifies the first instruction to determine j and k from the mode register.

[0017] In a second aspect, an embodiment of the present application provides a control method applied to an internal memory, the internal memory being coupled with a processor, the internal memory comprising a plurality of memory banks, each memory bank comprising M rows*N columns of memory cells, M and N being integers greater than 0; the method comprising: receiving a second instruction sent by the processor; determining a target memory bank and a target row according to the second instruction; receiving a third instruction sent by the processor, the third instruction being used to indicate a starting column address j and a column number k; k being an integer greater than 1 and less than or equal to N; and reading target data in response to the third instruction, the target data being data stored in the jth column to the (j+k-1)th column of the target row in the target memory bank.

[0018] In an embodiment of the present application, the internal memory comprises a plurality of memory banks, each memory bank can comprise M rows*N columns of memory cells, and each memory cell can be used to store data. In some computing scenarios, in order to improve computing efficiency, the processor will place data into memory cells in the same row one by one according to the computing order. When data needs to be taken from the internal memory for computing, the processor will first send a second instruction to the internal memory to activate a row in a target memory bank, and then the processor will send a third instruction to the internal memory, which can be used to indicate a starting column address j and a column number k. After the internal memory receives the third instruction, the internal memory can read data stored in the jth column memory cell to the (j+k-1)th column memory cell of the ith row (i.e., the target row) in the target memory bank based on j and k. In the prior art, after activating a row of data in a memory bank, the processor needs to issue multiple instructions (each instruction includes only one column address) to read multiple data for computing, which results in low data reading efficiency, less idle time of instruction address bus, and difficulty in multi-task concurrent operation. In the present application, since the third instruction can be used to indicate the starting column address j and the column number k, the internal memory can automatically read k columns of data starting from the jth column based on the starting column address j and the column number k, and the processor can read multiple columns of data by issuing one instruction, thereby improving data reading efficiency and computing performance.

[0019] In some embodiments, the third instruction comprises j and k; and j and k are determined according to the third instruction.

[0020] In an embodiment of the present application, the third instruction is a column strobe instruction, j and k are determined based on the third instruction by carrying j and k in the third instruction, and then k columns of data starting from the jth column can be automatically read based on j and k, so that the processor does not need to issue multiple instructions to read multiple columns of data, thereby improving instruction efficiency, data reading efficiency, and computing performance.

[0021] In some embodiments, the memory further includes a mode register, k is pre-configured in the mode register; the third instruction includes j, and the method further includes determining k from the mode register according to the third instruction.

[0022] In the embodiments of the present application, the third instruction is a column strobe instruction, and the data reading efficiency can be improved by cooperation of the third instruction and the mode register. When k is pre-configured in the mode register and j is included in the third instruction, the memory sends the third instruction to the processor, determines k from the mode register, and determines j from the third instruction, and then k columns of data starting from the jth column can be read automatically based on j and k, so that the processor does not need to issue multiple instructions to read multiple columns of data, the instruction efficiency and the data reading efficiency are improved, and the computing performance is improved.

[0023] In some embodiments, the memory further includes a mode register, j is pre-configured in the mode register; the third instruction includes k, and the method further includes determining j from the mode register according to the third instruction.

[0024] In the embodiments of the present application, the third instruction is a column strobe instruction, and the data reading efficiency can be improved by cooperation of the third instruction and the mode register. When k is pre-configured in the mode register and j is included in the third instruction, the memory sends the third instruction to the processor, determines k from the mode register, and determines j from the third instruction, and then k columns of data starting from the jth column can be read automatically based on j and k, so that the processor does not need to issue multiple instructions to read multiple columns of data, the instruction efficiency and the data reading efficiency are improved, and the computing performance is improved.

[0025] In some embodiments, the memory further includes a plurality of computing units, and each Bank corresponds to a computing unit, and the method further includes sending the target data to the computing units corresponding to the target Banks respectively.

[0026] In the embodiments of the present application, when the memory reads k columns of data starting from the jth column based on the starting column address j and the column number k, the read data can be sent to the PUs corresponding to the Banks for calculation, so that the processor does not need to issue multiple instructions to read multiple columns of data, the data reading efficiency is improved, and the computing performance of the computing units is improved.

[0027] In some embodiments, the method further includes sending the target data to the processor.

[0028] In the embodiments of the present application, after the memory automatically reads out k columns of data starting from the jth column based on the starting column address j and the column number k, the read-out data can be sent to the processor, avoiding the processor from issuing multiple instructions to read multiple columns of data, thereby improving the data reading efficiency and improving the computing performance of the processor.

[0029] In a third aspect, the embodiments of the present application provide a memory, which can be used to execute the control method of any one of the first aspect or execute the control method of any one of the second aspect.

[0030] In a fourth aspect, the embodiments of the present application provide a chip system, which includes a processor and the memory of the third aspect.

[0031] In a fifth aspect, the embodiments of the present application provide an electronic device, which includes the memory of the third aspect. BRIEF DESCRIPTION OF DRAWINGS

[0032] FIG. 1 is a schematic diagram of a chip system according to an embodiment of the present application.

[0033] FIG. 2A is a schematic diagram of a memory according to an embodiment of the present application.

[0034] FIG. 2B is a schematic diagram of a PIM-Die according to an embodiment of the present application.

[0035] FIG. 2C is a schematic diagram of a Bank according to an embodiment of the present application.

[0036] FIG. 2D is a schematic diagram of an instruction flow for reading data from the memory according to an embodiment of the present application.

[0037] FIG. 2E is a schematic diagram of an instruction flow for reading data from the memory and performing a computing task according to an embodiment of the present application.

[0038] FIG. 3 is a schematic diagram of a control method according to an embodiment of the present application.

[0039] FIG. 4 is a schematic diagram of a memory according to an embodiment of the present application.

[0040] FIG. 5 is a schematic diagram of reading data from the memory according to an embodiment of the present application.

[0041] FIG. 6 is a schematic diagram of another memory according to an embodiment of the present application.

[0042] FIG. 7 is a schematic diagram of an instruction bus according to an embodiment of the present application.

[0043] FIG. 8 is a schematic diagram of an instruction flow for reading data according to an embodiment of the present application.

[0044] Figure 9 is a flowchart illustrating another control method provided in an embodiment of this application.

[0045] Figure 10 is a schematic diagram of another internal memory structure provided in an embodiment of this application.

[0046] Figure 11 is a schematic diagram of another instruction bus provided in an embodiment of this application.

[0047] Figure 12 is a schematic diagram of another instruction bus provided in an embodiment of this application.

[0048] Figure 13 is a schematic diagram of another instruction flow for reading data provided in an embodiment of this application.

[0049] Figure 14 is a schematic diagram of a mode register provided in an embodiment of this application.

[0050] Figure 15 is a schematic diagram of another mode register provided in an embodiment of this application.

[0051] Figure 16 is a schematic diagram of another type of internal memory provided in an embodiment of this application.

[0052] Figure 17A is a schematic diagram of a multiply-accumulate calculation unit and a calculation pipeline provided in an embodiment of this application.

[0053] Figure 17B is a schematic diagram of another multiply-accumulate calculation unit and calculation pipeline provided in an embodiment of this application.

[0054] Figure 18 is a flowchart illustrating another control method provided in an embodiment of this application.

[0055] Figure 19 is a flowchart illustrating another control method provided in an embodiment of this application. Detailed Implementation

[0056] The embodiments of this application will now be described with reference to the accompanying drawings.

[0057] The terms "first," "second," "third," and "fourth," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0058] Reference to“an embodiment” herein means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase“in an embodiment” in various places in the specification are not necessarily referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. As will be apparent to those of ordinary skill in the art, embodiments described herein can be combined with other embodiments.

[0059] The chip system related to the embodiments of the application is introduced as follows:

[0060] An embodiment of the application provides a chip system. Referring to FIG. 1, FIG. 1 is a schematic diagram of a chip system provided by an embodiment of the application. The chip system can include, but is not limited to, a system on chip (Soc) 101, an internal memory 102, and an external memory 103. The chip system can be integrated in any electronic device, such as a computer, a mobile phone, a tablet, a personal digital assistant, a smart wearable device, a smart vehicle, or a smart home appliance, and the like, and the chip system can work under necessary software driving.

[0061] The system on chip (Soc) 101 refers to a technology of integrating a complete system on a single chip and grouping all or part of necessary electronic circuits. The system on chip 101 can include a plurality of processing units and a controller 1014. The plurality of processing units can include one or more of a central processing unit (CPU), a modem processing unit, a graphics processing unit (GPU), an image signal processor (ISP), a video coding unit, a digital signal processor (DSP), a baseband processing unit, and a neural-network processing unit (NPU), and the like. Optionally, the different processing units can communicate with each other through a primary bus, and each processing unit can also communicate with the controller 1014 through the primary bus.

[0062] The central processing unit 1011, which can also be referred to as a central processing unit (CPU) or a processor, can be a multi-core processor, i.e., a plurality of processor cores are integrated in one chip, or a single-core processor, which is not limited in the present application. The processor core is also referred to as a kernel, which is the most important component of the CPU. It is made of single-crystal silicon by a certain production process. All calculations, receiving instructions or storing instructions, and processing data of the CPU are performed by the processor core. The processor core can load instructions or data stored in the external memory 103 (such as a disk) into the internal memory 102 through the controller 1014, and transfer instructions or data that need to be operated from the internal memory 102 to the processor core through the controller 1014 for operation. When the operation is completed, the processor core can temporarily store the result in the internal memory 102, and store instructions or data that need to be stored for a long time in the external memory 103 through the controller 1014. The processor core can run an operating system, a file system (such as a flash file system F2FS) or an application program, etc., to control a plurality of hardware or software elements connected to the CPU, and can process various data and perform operations. Optionally, the central processing unit 1011 can include a memory, which can be a cache. The cache can include one or more of a level 1 cache (L1 Cache), a level 2 cache (L2 Cache), a level 3 cache (L3 Cache), etc. The cache can save instructions or data that have just been used or are frequently used by the central processing unit 1011. If the central processing unit 1011 needs to use the instructions or data again, it can be directly called from the cache. This avoids repeated access and reduces the waiting time of the processor core, thereby improving the efficiency of the central processing unit 1011.

[0063] The controller 1014 can be used to manage and control the communication between the central processing unit 1011 and the internal memory 102. The controller 1014 can also be used to manage and control the communication between the central processing unit 1011 and the external memory 103 (such as a disk), and to provide a standardized interface (such as a universal flash storage UFS standard) for communication between the central processing unit 1011 and the external storage device. It can also be understood that the central processing unit 1011 can send access instructions (such as read / write instructions) and control instructions to the internal memory 102 through the controller 1014, and the central processing unit 1011 can also send access instructions (such as read / write instructions) to the external memory 103 through the controller 1014.

[0064] The internal memory 102 (Memory) can be located outside the system on chip 101, and is usually a power-off volatile memory that loses the content stored thereon when power off, and can also be referred to as a main memory. The internal memory 102 in the present application includes a readable and writable running memory, which is used to temporarily store the operation data required by the central processing unit 1011, and interact with the external memory 103 or other external memories, and can be used as a temporary data storage medium for the operating system or other programs running.

[0065] The internal memory 102 can include one or more of a dynamic random access memory (DRAM), a static random access memory (SRAM), a synchronous dynamic random access memory (SDRAM), etc. Among them, the DRAM includes a double data rate synchronous dynamic random access memory (DDR SDRAM) referred to as DDR, a second generation double data rate synchronous dynamic random access memory (DDR2), a third generation double data rate synchronous dynamic random access memory (DDR3), a fourth generation low power double data rate synchronous dynamic random access memory (Low Power Double Data Rate 4, LPDDR4), and a fifth generation low power double data rate synchronous dynamic random access memory (Low Power Double Data Rate 5, LPDDR5), etc.

[0066] The external memory 103 is a non-volatile memory, and the content stored therein will not be lost after power off. The external memory 103 can be used to store instructions and data related to the operation of the central processing unit 1011 for a long time, such as startup programs, operating systems, application programs, and data, etc. Since the central processing unit 1011 cannot directly read instructions and data in the external memory 103, nor can it directly write instructions or data to the external memory 103, therefore, when executing a read (or load) instruction, the central processing unit 1011 actually temporarily loads the content to be read (including instructions and / or data) stored in the external memory 103 to the internal memory 102 through the controller 1014, and then reads it from the internal memory 102; and when executing a write (i.e. storage) instruction, the central processing unit 1011 actually temporarily writes the data to be stored (including instructions and / or data) to the internal memory 102, and then stores it from the internal memory 102 to the external memory 103 through the controller 1014.

[0067] The external memory 103 can include one or more of a Flash memory (e.g., a NAND Flash memory, a NOR Flash memory, etc.), a universal flash storage (UFS), an embedded Multi-Media Card (eMMC), a universal flash storage multi-chip package (uMCP) memory, an embedded Multi-Media Card multi-chip package (eMCP) memory, a solid state drive (SSD), etc.

[0068] It can be understood that the chip system in FIG. 1 is only some exemplary embodiments provided by the embodiments of the present application, and the chip system in the embodiments of the present application includes but is not limited to the above implementation.

[0069] With the development of artificial intelligence technology, generative AI is emerging, such as ChatGPT, etc. Deploying large models for AI inference on the terminal side to improve user experience and productivity has become one of the main competitiveness of the terminal. There is a large amount of data in the large model inference process that needs to be moved from the internal memory 102 to the central processing unit 1011, or the graphics processing unit 1012, or the neural network processing unit 1013 for calculation, which causes the current terminal side AI inference performance bottleneck to mainly concentrate on the memory bandwidth. At present, in order to improve the terminal side AI inference performance, processing in memory (PIM) has become the mainstream, that is, multiple independent computing units (Processing Units, PU) can be integrated in the internal memory 102, and part of the computing task is offloaded to the memory for calculation, which can improve the computing performance of the terminal and accelerate the large model inference. At the same time, using the processing in memory technology can reduce data movement, reduce terminal energy consumption, and improve terminal endurance. The structure of the internal memory 102 and the instruction flow of reading data from the internal memory 102 will be described in detail below in conjunction with FIGS. 2A-2E.

[0070] Referring to FIG. 2A, which is a structural diagram of a memory according to an embodiment of the present application, the memory 102 can include one or more ranks (Rank). Each Rank refers to a set of memory dies (Die), and memory dies in the same set share the same address and control signals. Each Rank is usually 64 bits wide, which means it can process 64 bits of data at the same time. The memory 102 can have one or more Ranks, such as a single Rank (1R) and a double Rank (2R). The double Rank can include two Ranks, and each Rank can be independently accessed, thereby improving the bandwidth and performance of the memory 102. For example, as shown in FIG. 2A, the Rank 1 can include a plurality of Dies, and the Rank 0 can be independently used to access data; the Rank 1 can include a plurality of PIM-Dies, and the PIM-Die is a Die with a computing unit (PU) integrated inside. The Rank 1 can be independently used to access data and perform computing tasks. It should be noted that the same structure of the Die can be included in the same Rank, or different structures of the Die can be included in the same Rank, which is not limited in the present application. The structure of the PIM-Die will be described in detail below in conjunction with FIG. 2B.

[0071] Please refer to FIG. 2B, which is a structural diagram of a PIM-Die according to an embodiment of the present application. Each PIM-Die can include a plurality of memory areas (Banks), such as Banks A-J shown in FIG. 2B. A Bank is a basic unit for storing and managing data. Each Bank can be regarded as a memory area and can process data requests independently. Different Banks allow parallel access and operation, which means that a memory controller can access multiple Banks simultaneously, thereby improving data read efficiency. For example, a PIM-Die includes 4 Banks, and data is stored in the 4 Banks. When data is accessed, the memory controller can activate data in the 4 Banks simultaneously. Each PIM-Die also integrates a plurality of computing units (PUs), such as PUs 1-5 shown in FIG. 2B. A PU is used to perform a computing task to achieve direct computation in the memory 102, instead of transmitting data to an external system-on-chip 101 for processing, thereby improving computing efficiency. Each PU can correspond to one or more Banks. For example, PU1 shown in FIG. 2B can correspond to Banks A and F, that is, data read from Banks A and F can be directly sent to PU1 for computation. Each PIM-Die can also integrate a plurality of mode registers (MRs), such as mode register 1 and mode register 2 shown in FIG. 2B. An MR can be used to configure the operation mode and functions of the memory 102, for example, to set parameters such as data width, delay, refresh rate, and the like. The presence of the MR enables the memory controller to configure and optimize the memory 102 according to the needs of the system, thereby improving performance and compatibility. By programming these registers, the system can adjust the behavior of the memory 102 to adapt to different application scenarios and performance requirements. Each PIM-Die can also include a global buffer that can be used as a cache to store frequently accessed data, thereby reducing access delay and improving system response speed. The structure of a Bank will be described in detail below with reference to FIG. 2C.

[0072] Please refer to FIG. 2C, which is a structural diagram of a Bank according to an embodiment of the present application. A Bank can include a memory array and a row buffer. The memory array is responsible for actual data storage and is composed of a plurality of memory cells, which are usually organized in the form of rows and columns, that is, the memory array can be composed of M rows and N columns of memory cells, allowing efficient data access. The row buffer is used to store row data of a row read from the memory array. The instruction flow of reading data from the memory 102 will be described in detail below with reference to FIG. 2D.

[0073] Please refer to FIG. 2D, which is a schematic diagram of an instruction flow for reading data from the internal memory according to an embodiment of the present application. Since the data is stored in the storage array of the Bank in rows and columns, and in order to improve the data reading efficiency, when reading data from the internal memory 102, a row of data in a Bank needs to be activated first, which can also be referred to as row activation. The purpose of row activation is to load the data in the specific row in the Bank to the temporary storage area of the row buffer, so that the data in the row can be accessed quickly. As shown in FIG. 2D, when it is needed to read data from the internal memory 102, the CPU needs to send a row gating instruction (which can also be referred to as ACTIVAE activation instruction) to the internal memory 102 to perform row activation. The row gating instruction includes a Bank address and a row (Row) address, such as the Bank address is Bank A and the Row address is Row M. The instruction can be used to activate the Mth row in the storage array of Bank A. It should be noted that the ACTIVAE activation instruction of the standard DRAM memory will only activate a row in a Bank each time. However, in order to save instruction overhead, the ACT4 and ACT16 activation instructions are proposed for the PIM-DRAM in-memory computing memory, which respectively represent activating a row in 4 or 16 Banks in the DRAM memory at the same time, thereby improving the data reading efficiency. Once the row is activated, the row data will be loaded to the row buffer of the corresponding Bank, and then the CPU can send a column gating instruction to the internal memory 102. The column gating instruction includes a column (col.) address, which is used to specify the specific column data of the row data stored in the row buffer, i.e., the CPU sends a column gating instruction to read the data in a column, such as the col. address is i, and the column gating instruction is used to read the data in the ith column. If it is needed to read the data in different columns in the same row, the CPU needs to send multiple column gating instructions to the internal memory 102 to read the data in multiple columns. In addition, the CPU can also send a pre-charge instruction to the internal memory 102 to pre-charge the specified Bank and close the activated row. After the pre-charge operation is performed, the CPU can continue to send a row gating instruction to the internal memory 102, so as to activate other rows in the Bank. The instruction flow for reading data from the internal memory 102 and performing a computing task will be described in detail below in combination with FIG. 2E.

[0074] Please refer to FIG. 2E, which is a schematic diagram of an instruction flow for reading data from the internal memory and performing a computing task according to an embodiment of the present application. The instruction flow for reading data from the internal memory 102 and performing a computing task is similar to the instruction flow for reading data from the internal memory 102. First, the CPU needs to send a row strobe instruction to the internal memory 102 to activate a row, and then the CPU sends a compute instruction (COMPUTE) to the internal memory 102. The compute instruction includes a column address for specifying the column data of the row data stored in the row buffer to be read and the column data is to be transferred to the PU corresponding to the bank for computing. That is, the CPU sends one compute instruction to read the data of a column to the PU for computing. For example, the compute instruction with the column address j is used to read the data of the jth column to the corresponding PU for computing. If the data of different columns in the same row needs to be read for computing, the CPU needs to send multiple compute instructions to the internal memory 102 to read the data of multiple columns for computing.

[0075] It is found through research that there are a large number of General Matrix Vector Multiplication (GEMV) operations in the large model inference process. In this process, the data of different columns in the same row is continuously read from the bank to the PU, or the CPU, or the NPU, or the GPU for computing. In the large model inference process, a large number of GEMV operations are involved, and the data of different columns in the same row needs to be continuously read from the bank to the PU for computing. However, at present, when the data of different columns in the same row is continuously read from the bank, after activating the data of a row in the bank, only one instruction (such as the column strobe instruction or the compute instruction) can be sent by the CPU to read the data of one column for computing. Therefore, the CPU needs to send multiple instructions to the internal memory 102 in sequence to read the data of multiple columns for computing. This is low in instruction efficiency, which leads to low data reading efficiency, and the idle time of the instruction address bus is short, making it difficult to perform multi-task concurrent operation. Therefore, in the present application, a high-concurrency instruction scheme is proposed, which can significantly reduce the number of column strobe instructions or compute instructions sent in the large model inference process, improve the instruction efficiency and computing pipeline, improve the data reading efficiency, and further improve the performance of the large model inference. The details will be described later, and will not be described here.

[0076] The flow of the control method provided by the embodiments of the present application will be described below.

[0077] FIG. 3 is a schematic diagram of a control method according to an embodiment of the present application. The control method is applied to the internal memory 102, which is coupled with a processor. The internal memory 102 includes a plurality of storage regions Bank, and each Bank includes M rows*N columns of storage units, where M and N are integers greater than 0.

[0078] As shown in FIG. 3, the control method can include:

[0079] S201: The internal memory receives a first instruction sent by the processor.

[0080] Specifically, the first instruction is used to indicate a starting column address j and a column number k of a storage unit corresponding to to-be-read data; k is an integer greater than 1 and less than or equal to N. Please refer to FIG. 4, which is a structural schematic diagram of an internal memory provided by an embodiment of the present application. In FIG. 4, the internal memory 102 includes a plurality of Banks, such as Bank A, Bank B, and Bank C. Each of the plurality of Banks includes a storage array, which can be composed of M rows*N columns of storage units. Each storage unit can be used to store data, such as 256 bits of data. In a large model inference process, there are a large number of matrix-vector multiplication operations. Generally, in order to improve the calculation efficiency, the processor will place the weight data into the storage units in the same row but different columns according to the calculation order, that is, the processor will first place the first group of weight data into the first column storage unit, then place the second group of weight data into the second column storage unit, and so on. The logical positions of these data in the storage space are adjacent, only the column positions are different. After storing a row of data, the data is stored in the storage unit of the next row, and so on. When data needs to be taken from the internal memory 102 for calculation, the processor will send a first instruction to the internal memory 102. The first instruction can be an activate instruction (ACTIVATE') proposed by the present application, which can be used to indicate a starting column address j and a column number k of a storage unit corresponding to to-be-read data in the M rows*N columns of storage units; j is an integer less than N, and k is an integer greater than 0 and less than or equal to N, so that the internal memory 102 can automatically read out k columns of data starting from the jth column based on the starting column address j and the column number k, thereby improving the data reading efficiency.

[0081] S202: The internal memory determines a target Bank and a target row according to the first instruction.

[0082] Specifically, the target Bank is a Bank in which to-be-read data is stored in the plurality of Banks, and the target row is in the to-be-activated Bank.

[0083] S203: The internal memory reads target data in response to the first instruction.

[0084] Specifically, the target data is data stored in the jth column to the (j+k-1)th column of the target row in the target Bank.

[0085] Specifically, the target Bank is a Bank storing the data to be read in the plurality of Banks; i is an integer less than M. When the memory 102 receives the first instruction, the memory 102 can read the data stored in the memory cells from the jth column to the (j+k-1)th column of the ith row in the target Bank based on j and k; when the target Bank is multiple, the data stored in the memory cells from the jth column to the (j+k-1)th column of the ith row in each target Bank can be read in parallel.

[0086] For example, as shown in FIG. 5, which is a schematic diagram of reading data from the memory according to an embodiment of the present application, the target Bank can include one or more, and one is shown in FIG. 5 as an example. It is assumed that the storage arrays in the target Bank are all composed of 8*8 memory cells, i=1, j=2, and k=5. When data needs to be taken from the memory 102 for calculation, the processor sends a first instruction to the memory 102, which can be used to indicate the starting column address j=2 and the column number k=5 of the memory cells corresponding to the data to be read. When the memory 102 receives the first instruction, the memory 102 can read the data stored in the memory cells from the 2nd column to the 5th column of the 1st row in the target Bank based on j=2 and k=5. Optionally, if the memory 102 includes multiple target Banks, the data stored in the memory cells from the 2nd column to the 5th column of the 1st row in each target Bank can be read in parallel.

[0087] In summary, since the instructions sent by the processor to the memory 102 cannot indicate the starting column address j and the column number k of the memory cells corresponding to the data to be read in the M rows*N columns of memory cells, when reading data in the same row but different columns from the Bank in succession, after activating the data in a certain row in the Bank, the processor can only read one column of data for calculation by issuing one instruction (which only includes one column address) at a time. Therefore, the processor needs to send multiple instructions to the memory 102 in sequence to read multiple columns of data for calculation in sequence, which is low in instruction efficiency, leads to low data reading efficiency, and has less idle time of the instruction address bus, making it difficult to perform multi-task concurrent operation. In the present application, since the first instruction can be used to indicate the starting column address j and the column number k of the memory cells corresponding to the data to be read in the M rows*N columns of memory cells, the memory 102 can automatically read k columns of data starting from the jth column in the target Bank in parallel based on the starting column address j and the column number k, avoiding the need for the processor to issue multiple instructions to read multiple columns of data, thereby improving the data reading efficiency, improving the computing performance, and further improving the user experience.

[0088] In some embodiments, referring to FIG. 6, which is a schematic diagram of another structure of the internal memory according to an embodiment of the present application, the internal memory 102 further includes a mode register, which includes a start column address field and a column number field, j is stored in the start column address field and k is stored in the column number field; j is obtained from the start column address field and k is obtained from the column number field according to the first instruction.

[0089] Specifically, the internal memory 102 includes one or more mode registers, which are pre-configured with j and k, i.e., j and k can be stored in one mode register, j can be stored in one mode register and k can be stored in another mode register, or information of j and k can be stored in more mode registers, which is not limited in the present application. It should be noted that FIG. 6 only shows the case that j and k are pre-configured in the mode register 1. When the processor sends the first instruction to the internal memory 102, the internal memory 102 identifies the first instruction and determines j and k from the start column address field and the column number field of the mode register. Further, the internal memory 102 can automatically read out k columns of data starting from the jth column based on the start column address j and the column number k in response to the first instruction, which avoids the processor from sending multiple instructions to read multiple columns of data of the same row of storage units, thereby improving the data reading efficiency, improving the computing performance, and further improving the user experience.

[0090] In an embodiment, the first instruction includes a Bank address of a target Bank and a row address of the ith row of storage units; the internal memory 102 identifies the first instruction and reads data stored by the ith row of storage units of the target Bank to the row buffer of the corresponding Bank based on the Bank address and the row address.

[0091] Specifically, the first instruction can include a Bank address of one or more target Banks and a row address of the ith row storage unit, so as to determine the target Banks from the plurality of Banks according to the Bank address, and perform row activation according to the row address. In this embodiment, the first instruction is a row activation instruction (also referred to as a high concurrency calculation activation instruction), and the data reading efficiency is improved by cooperation of the first instruction and the mode register. When the processor sends the first instruction to the internal memory 102, the internal memory 102 identifies the first instruction, and reads the data stored in the ith row storage unit of the target Banks into the row buffer of the corresponding Banks based on the Bank address of the target Banks and the row address. After identifying the first instruction, the internal memory 102 can also determine j and k from the mode register. Further, the internal memory 102 can read out k columns of data starting from the jth column based on the starting column address j and the column number k in response to the first instruction, so as to avoid the processor needing to issue multiple instructions to read multiple columns of data of the same row storage unit, thereby improving the data reading efficiency, improving the calculation performance, and further improving the user experience.

[0092] In some embodiments, j and k corresponding to one or more target Banks are pre-configured in the mode register, and j and k corresponding to other Banks can be stored in other mode registers.

[0093] In some embodiments, the internal memory 102 and the processor can be connected through an instruction bus, which can include L pins (PINs), and the first M bits can be used to indicate the instruction function, and the last L-M bits can be used to carry the Bank address and the row address. For example, referring to FIG. 7, which is a schematic diagram of an instruction bus provided in an embodiment of the present application, it is assumed that the instruction bus can include 8 pins (PINs), namely P0, P1, P2, P3, P4, P5, P6 and P7, and the first four bits can be used to indicate the instruction function, and the last four bits can be used to carry the Bank address and the row address. The first instruction (ACTIVATE’) is a row activation instruction, and different from the current row activation instruction, the first instruction can also be used to indicate the starting column address j and the column number k from the mode register, so the first four bits of the first instruction are different from the first four bits of the current row activation instruction. After receiving the instruction, the internal memory 102 can identify the first instruction through the first four bits, and then perform row activation and determine the starting column address j and the column number k from the starting column address field and the column number field in the mode register.

[0094] For example, as shown in FIG. 8, which is a schematic diagram of an instruction flow for reading data according to an embodiment of the present application, when data needs to be read from the memory 102, the processor can send a first instruction to the memory 102 to specify one or more Bank addresses and a row address i. After receiving the first instruction, the memory 102 identifies the first instruction, activates the ith row of one or more target Banks (for example, Bank A) corresponding to the one or more Bank addresses, and determines a starting column address j and a column number k from the mode register. For the target Banks, the memory 102 can automatically read out k columns of data starting from the jth column based on the starting column address j and the column number k, which can significantly reduce the number of column strobe instructions issued by the processor, thereby improving the data reading efficiency, improving the computing performance, and further improving the user experience. In addition, if data stored in other row storage units needs to be read, the processor needs to first send a PRECHARGE instruction to the memory 102 to close the activated row, and then send a first instruction to activate a new row.

[0095] For example, as shown in FIG. 9, which is a schematic diagram of a flow of another control method according to an embodiment of the present application, the control method can include:

[0096] S301: The memory receives a second instruction sent by the processor.

[0097] Specifically, when data needs to be read from the memory 102, the processor needs to first send a second instruction to the memory 102, and the second instruction is a row activation instruction (Activate). The second instruction specifies one or more target Banks and a row address of the ith row storage unit, so as to determine the target Banks from the plurality of Banks according to the Bank addresses, and activate the row according to the row address.

[0098] S302: The memory determines the target Banks and the target row according to the second instruction.

[0099] Specifically, the target Banks are Banks in the plurality of Banks that store the data to be read, and the target row is a row in the target Banks.

[0100] S303: The memory receives a third instruction sent by the processor.

[0101] Specifically, the third instruction is used to indicate a starting column address j and a column number k; k is an integer greater than 1 and less than or equal to N. The third instruction is a column strobe instruction (also referred to as a high-concurrency column strobe computing instruction), and the memory 102 activates the target data in response to the third instruction to indicate the starting column address j and the column number k, so as to improve the data reading efficiency.

[0102] S304: The memory reads the target data in response to the third instruction.

[0103] Specifically, the target data is data stored in the jth column to the (j+k-1)th column of the target row in the target Bank.

[0104] In summary, when the processor sends the second instruction to the internal memory 102, the internal memory 102 activates the target Bank based on the second instruction, and reads the data stored in the ith row storage unit of the target Bank to the row buffer of the corresponding Bank. The processor sends the third instruction to the internal memory 102, and the internal memory 102 can read out k columns of data starting from the jth column based on the starting column address j and the column number k in response to the third instruction. This avoids the processor from issuing multiple instructions to read multiple columns of data in the same row storage unit, thereby improving data reading efficiency, improving computing performance, and further improving user experience.

[0105] In some embodiments, referring to FIG. 10, which is a structural schematic diagram of another internal memory provided by an embodiment of the present application, each of the Banks further includes a row buffer. The internal memory 102 receives the second instruction sent by the processor, and the second instruction can include the Bank address of one or more target Banks and the row address of the ith row storage unit. Based on the Bank address of the one or more target Banks and the row address, the internal memory 102 reads the data stored in the ith row storage unit of the one or more target Banks to the row buffer of the corresponding Bank. In response to the third instruction, the internal memory 102 reads the data stored in the jth column storage unit to the (j+k-1)th column storage unit of the ith row from the row buffer of the one or more target Banks based on j and k.

[0106] In some embodiments, the third instruction includes j and k, and j and k are determined according to the third instruction.

[0107] Specifically, the third instruction is a column strobe instruction, and j and k are carried by the third instruction. j and k can be determined based on the third instruction, and then k columns of data starting from the jth column can be automatically read out based on j and k. This avoids the processor from issuing multiple instructions to read multiple columns of data, thereby improving instruction efficiency, improving data reading efficiency, and further improving computing performance.

[0108] In some embodiments, the internal memory 102 further includes a mode register, and k is preconfigured in the mode register. The third instruction includes j, and the method further includes determining k from the mode register according to the third instruction.

[0109] Specifically, the third instruction is a column strobe instruction, and the data reading efficiency can be improved by the third instruction and the mode register. When the mode register is pre-configured with k and the third instruction includes j, the processor sends the third instruction to the internal memory 102, the internal memory 102 identifies the third instruction, determines k from the mode register, and determines j from the third instruction, and then can automatically read out k columns of data starting from the jth column based on j and k, thereby improving the instruction efficiency, improving the data reading efficiency, and improving the computing performance.

[0110] In some embodiments, the internal memory 102 further includes a mode register pre-configured with j, and the third instruction includes k. The method further includes determining j from the mode register according to the third instruction.

[0111] Specifically, the third instruction is a column strobe instruction, and the data reading efficiency can be improved by the third instruction and the mode register. When the mode register is pre-configured with j and the third instruction includes k, the processor sends the third instruction to the internal memory 102, the internal memory 102 identifies the third instruction, determines j from the mode register, and determines k from the third instruction, and then can automatically read out k columns of data starting from the jth column based on j and k, thereby improving the instruction efficiency, improving the data reading efficiency, and improving the computing performance.

[0112] In summary, when the third instruction is a column strobe instruction, the data reading efficiency can be improved by the third instruction or by the third instruction and the mode register.

[0113] In some embodiments, the internal memory 102 and the processor can be connected through an instruction bus, which can include L pins (PINs). The first M bits can be used to indicate the instruction function, and the last L-M bits can be used to carry the starting column address j and the column number k. For example, referring to FIG. 11, which is a schematic diagram of another instruction bus provided in an embodiment of the present application, it is assumed that the instruction bus can include 8 pins (PINs), namely P0, P1, P2, P3, P4, P5, P6, and P7. The first four bits can be used to indicate the instruction function, and the last four bits can be used to carry the starting column address j and the column number k. The third instruction is a column strobe instruction, and the difference between the third instruction and the current column strobe instruction is that the last four bits of the third instruction can be used to carry the starting column address j and the column number k. After the internal memory 102 receives the third instruction, j and k can be determined through the last 4 bits.

[0114] In some embodiments, the first M bits can be used to indicate instruction function, and the last L-M bits can be used to carry the starting column address j. Since the number of pins of the instruction bus is limited, the number of columns k that can be represented is also limited, and therefore k can be configured first into the mode register of the memory 102 for storage, so as to improve the data reading efficiency by cooperating with the mode register through a third instruction. For example, referring to FIG. 12, which is a schematic diagram of another instruction bus provided by an embodiment of the present application, it is assumed that the instruction bus can include 8 pins (PIN), P0, P1, P2, P3, P4, P5, P6, and P7, wherein the first four bits can be used to indicate instruction function, and the last four bits can be used to carry the starting column address j. The third instruction is a column strobe instruction, and different from the current column strobe instruction, the first four bits of the third instruction can be used to indicate that k is determined from the mode register, and the last four bits can be used to carry the starting column address j. After the memory 102 receives the third instruction, k can be determined from the mode register through the first four bits, and j can be determined through the last four bits.

[0115] In some embodiments, the first M bits can be used to indicate instruction function, and the last L-M bits can be used to carry the column number k. Since the number of pins of the instruction bus is limited, the starting column address j that can be represented is also limited, and therefore j can be configured first into the mode register of the memory 102 for storage, so as to improve the data reading efficiency by cooperating with the mode register through a third instruction. It is assumed that the instruction bus can include 8 pins (PIN), P0, P1, P2, P3, P4, P5, P6, and P7, wherein the first four bits can be used to indicate instruction function, and the last four bits can be used to carry the column number k. The third instruction is a column strobe instruction, and different from the current column strobe instruction, the first four bits of the third instruction can be used to indicate that j is determined from the mode register, and the last four bits can be used to carry k. After the memory 102 receives the third instruction, j can be determined from the mode register through the first four bits, and k can be determined through the last four bits.

[0116] For example, as shown in FIG. 13, which is another instruction flow diagram for reading data provided by an embodiment of the present application, when data needs to be read from the internal memory 102, the processor can send a second instruction to the internal memory 102 to specify one or more Bank addresses and a row address i. After receiving the second instruction, the internal memory 102 identifies the second instruction and activates the ith row of one or more target Banks corresponding to the one or more Bank addresses. Then, the processor sends a third instruction to the internal memory 102, which includes a starting column address j. After identifying the third instruction, the internal memory 102 can determine j based on the third instruction and determine k from the mode register. The internal memory 102 can automatically read out k columns of data starting from the jth column based on the starting column address j and the column number k, which can significantly reduce the number of column strobe instructions issued by the processor, thereby improving data reading efficiency, improving computing performance, and further improving user experience. In addition, if data stored in other row storage units needs to be read, the processor needs to first send a PRECHARGE instruction to the internal memory 102 to close the activated row and then send a second instruction to activate a new row.

[0117] In some embodiments, for the above two control methods, the mode register includes a target field, and the method further includes: before receiving the first instruction and / or the third instruction, the processor receives a target task, determines configuration information according to the target task, and the configuration information includes j and / or k; the internal memory receives the configuration information sent by the processor, determines j and / or k according to the configuration information, and stores j and / or k in the target field of the mode register.

[0118] Specifically, the mode register can include a plurality of fields, which can be used to indicate different operation modes or information. Before receiving the first instruction or the third instruction, the processor can first send configuration information to the internal memory 102 to configure the mode register in the internal memory 102, so that j and / or k are pre-configured in the target field of the mode register. The target field includes a starting column address field and a column number field, the starting column address field is used to store j, and the column number field is used to store k. When the target field of the mode register is pre-configured with j and k, the processor sends the first instruction to the internal memory 102, and the internal memory 102 identifies the first instruction, determines j and k from the mode register; when the target field of the mode register is pre-configured with j and the third instruction includes k, the processor sends the third instruction to the internal memory 102, and the internal memory 102 identifies the third instruction, determines j from the mode register, and determines k from the third instruction; when the target field of the mode register is pre-configured with k and the third instruction includes j, the processor sends the third instruction to the internal memory 102, and the internal memory 102 identifies the third instruction, determines k from the mode register, and determines j from the third instruction.

[0119] In some embodiments, the mode register further comprises a target Bank field; the target Bank and the target row are determined according to the first instruction or the third instruction, specifically: the target Bank is determined from the target Bank field according to the first instruction or the third instruction; the target row is determined according to the first instruction or the second instruction.

[0120] Specifically, the mode register can further comprise a target Bank field, which can record the related information of the Bank to be activated, such as the Bank address or the index of the Bank to be activated, etc., so that the address of the Bank to be activated can not be carried in the first instruction or the second instruction, and the Bank to be activated can be determined from the target Bank field of the mode register by identifying the first instruction or the second instruction.

[0121] In some embodiments, the target Bank field comprises a plurality of target Banks.

[0122] In the embodiments of the present application, the target Bank field can comprise the information of a plurality of Banks to be activated, such as a plurality of Bank addresses or a plurality of indexes of Banks to be activated, etc., when the first instruction or the second instruction is issued by the processor, the plurality of Banks to be activated can be determined from the mode register by identifying the first instruction or the second instruction, and the plurality of Banks can be activated at the same time.

[0123] Exemplarily, as shown in FIG. 14, FIG. 14 is a schematic diagram of a mode register provided by an embodiment of the present application. The mode register can include a plurality of fields, respectively MR#, MA[6:0], Access, OP[7], OP[6], OP[5], OP[4], OP[3], OP[2], OP[1], and OP[0]. Among them, MR# represents the index of the mode register; MA[6:0] represents the address of the mode register; Access is R / W, indicating that the register can be read and written; OP[6]-OP[0] can be a target field, used to store j and k. OP[7] can be a configuration mode field. Optionally, when OP[7] is configured as 0, j=0 and k=N, that is, the default mode is entered; when OP[7] is configured as 1, j and k determined from OP[6]-OP[0] are entered, that is, the custom mode is entered. Optionally, in the custom mode, OP[6]-OP[3] are start column address fields, used to store the index start col.index of the start column address j. 16 different start column address indexes can be configured, such as 0, 3, 7, 11, 15, 19, 23, 27, 31, 35, 39, 43, 47, 51, 55, and 59. Optionally, OP[2]-OP[0] are column number fields, used to store the column number k in the custom mode. 8 different k can be configured, such as 4, 8, 12, 16, 32, 48, and 64.

[0124] Exemplarily, as shown in FIG. 15, FIG. 15 is a schematic diagram of another mode register provided by an embodiment of the present application. Since the types of k and j that can be configured by OP[7]-OP[0] are limited, k and j can be stored in two mode registers. Mode register MRy realizes the configuration of the start column address j, and mode register MRz realizes the configuration of the column number k (y and z are non-negative integers, representing the indexes of the registers). Among them, the MRy register can configure 64 different j through OP[5]-OP[0] (as start column address fields); the MRz register can configure 64 different k through OP[5]-OP[0] (as column number fields).

[0125] Optionally, the processor can send configuration information to the internal memory 102, and the configuration information includes the information of the Banks activated at the same time, so that the target Bank field in the mode register can be pre-configured with the information of one or more target Banks.

[0126] For example, as shown in FIG. 15, the OP[7]-OP[6] (for the target Bank field) of the MRz register can be configured to indicate the Banks to be activated simultaneously. Assuming that the memory 102 includes 16 Banks, the information can be used to indicate the status of the 16 Banks, such as using a 16-bit number to represent the status of the 16 Banks, each bit corresponding to a Bank, 0 indicating that the Bank does not need to be activated, and 1 indicating that the Bank needs to be activated. The Banks information (including the addresses or indexes of the Banks to be activated) is configured in the mode register in advance, and when the processor sends a row activation instruction to the memory 102, one or more target Banks can be activated simultaneously according to the Banks information in the mode register, so that the memory 102 can read data from multiple Banks simultaneously, improving the data reading efficiency.

[0127] In some embodiments, the first instruction or the second instruction is further used to indicate the activation of one or more target Banks, and the memory 102 can determine the one or more target Banks to be activated from the target Bank field in the mode register after receiving the first instruction; or the second instruction is further used to indicate the activation of one or more target Banks, and the memory 102 can determine the one or more target Banks to be activated from the target Bank field in the mode register after receiving the second instruction.

[0128] In some embodiments, the method further includes: sending the data stored in the jth column memory cell to the (j+k-1)th column memory cell of the ith row in the one or more target Banks to the processor.

[0129] Specifically, when the memory 102 automatically reads k columns of data starting from the jth column based on the starting column address j and the column number k, the read data can be sent to the processor, avoiding the processor from issuing multiple instructions to read multiple columns of data, thereby improving the data reading efficiency, improving the computing performance of the processor, and further improving the user experience.

[0130] In some embodiments, the memory further includes a plurality of computing units, each Bank corresponding to a computing unit, and the method further includes: sending the data stored in the jth column memory cell to the (j+k-1)th column memory cell of the ith row in the one or more target Banks to the computing unit corresponding to the one or more target Banks, respectively.

[0131] Specifically, as shown in FIG. 16, which is a schematic diagram of another internal memory provided by an embodiment of the present application, the internal memory 102 includes a plurality of banks, such as Bank A, Bank B, and Bank C, and a plurality of computing units, such as PU1, PU2, and PU3. Each bank can correspond to a computing unit, such as Bank A corresponding to PU1, Bank B corresponding to PU2, and Bank C corresponding to PU3. One computing unit can correspond to one or more banks, that is, the data read out from one bank can be put into the corresponding PU for calculation, and one PU can receive the data sent by one or more banks and perform calculation. When the internal memory 102 automatically reads out k columns of data starting from the jth column based on the starting column address j and the column number k, the read-out data can be sent to the PUs corresponding to the banks for calculation, avoiding the processor from issuing multiple instructions to read multiple columns of data, thereby improving the data reading efficiency, improving the computing performance of the computing unit, and further improving the user experience.

[0132] For example, as shown in FIG. 17A and FIG. 17B, FIG. 17A is a schematic diagram of a multiply-accumulate computing unit and a computing pipeline provided by an embodiment of the present application. Assuming that the time delay for completing the calculation operation of one column of data is T1, the prior art scheme completes the calculation of col.j~col.j+7 columns of data by issuing 8 computing instructions, and the time delay is 8*T1. Only three instructions are shown in the figure. FIG. 17B is a schematic diagram of a multiply-accumulate computing unit and a computing pipeline provided by an embodiment of the present application. As shown in FIG. 17B, by issuing a high-concurrency column strobe computing instruction, the processor can automatically read out multiple columns of data for calculation after issuing one instruction. Assuming that the time delay for completing the calculation operation of one column of data is T1, and the time delay for automatically reading out one column of data and completing the calculation operation is T2, T2 is less than T1, the scheme of the present application completes the calculation of col.j~col.j+7 columns of data with a time delay of T1+7*T2, (T1+7*T2)<8*T1, and therefore the scheme of the present application can effectively shorten the calculation time delay.

[0133] It should be noted that in the large model training scene, when computing in the internal memory 102, the weight parameters W of the large model are generally stored in the Bank, and the input vector V is transmitted by the processor to the internal memory 102 and stored in the global buffer. Therefore, the current technical solution needs to first issue a GBW instruction (Global Buffer Write, GBW) to transmit the input vector V to the global buffer in the internal memory 102, and then issue an activation instruction and multiple column strobe calculation instructions to complete the calculation. In the present application, after the processor issues the first instruction, such as COMP_SB_AC, the internal memory 102 automatically sequentially extracts multiple column data for calculation, and the process of writing the input vector to the global buffer can be mostly buried in the high-concurrency calculation process. The GBW instruction is transmitted to the internal memory 102 to input a small amount of input vector data, such as 64 bytes, to meet the input vector data required for calculating the weight data of the first few columns; then the activation instruction and the high-concurrency column strobe calculation instruction are issued to start sequentially extracting multiple column data from the specified starting column address for calculation; while calculating, the GBW instruction is issued to input the remaining input vector data to calculate with the weight parameters stored in the corresponding column.

[0134] In addition, the internal memory 102 can include multiple Ranks, each Rank corresponding to a memory chip set, which are connected to the same command address and data bus, and different memory chip sets need to be addressed by a chip select signal (chip select), that is, different Ranks are selected. At present, when performing a calculation operation, multiple column strobe calculation instructions need to be continuously issued to perform multiple data calculations, and the command address bus is always busy. In the present application, after issuing a high-concurrency calculation instruction to the memory corresponding to a certain Rank, the internal memory automatically extracts multiple column data for calculation, at which time the command address bus is idle and can be switched to access other Ranks. After issuing a high-concurrency calculation instruction to the PIM-DRAM corresponding to Rank0, the command address bus can be switched to Rank1 for read-write access, thereby realizing multi-task concurrency operation.

[0135] Please refer to FIG. 18, which is a flowchart of another control method provided by an embodiment of the present application, taking a first instruction for reading data from the internal memory 102 to the calculation unit of the internal memory 102 for calculation as an example.

[0136] As shown in FIG. 18, the control method can include:

[0137] S401: The processor sends configuration information to the memory.

[0138] Specifically, the configuration information includes a start column address j and a column number k, where the column number k can also be understood as a calculation length.

[0139] S402: The memory receives the configuration information and configures the mode register based on the configuration information to store j and k in the mode register.

[0140] Specifically, the memory 102 includes a plurality of mode registers, and j and k can be stored in the same mode register or in a plurality of mode registers. For details, please refer to the above description of FIG. 14 and FIG. 15, which will not be repeated here.

[0141] S403: The processor sends a first instruction to the memory.

[0142] Specifically, the first instruction can be represented by ACT_nB_COMP, where n represents an integer, and the value range is 1- the total number of Banks in the memory 102, such as ACT_4B_COMP, which means activating data in 4 Banks for calculation; ACT_SB_COMP and ACT_AB_COMP represent activating data in a single Bank and all Banks for calculation, respectively. The first instruction can include the Bank address of one or more target Banks and the row address of the i-th row storage unit. For details of the first instruction, please refer to the above description of FIG. 6-FIG. 8, which will not be repeated here.

[0143] S404: The memory identifies the first instruction and determines whether the data stored in the configuration mode field of the mode register is 1.

[0144] Specifically, when the data stored in the configuration mode field is not 1, S405 is executed; when the data stored in the configuration mode field is 1, S406 is executed. Assuming that the name of the mode register is MRx, and OP[7] is the configuration mode field.

[0145] It should be noted that when OP[7] of the MRx register = 1, it indicates entering the custom mode, in which, after the processor issues the ACT_SB_COMP command to activate the specified row, the memory 102 automatically starts to take out data from the column corresponding to the start col.index for calculation, and automatically sequentially executes until the data calculation of the calculation length is completed. When OP[7] of the MRx register = 0, it indicates entering the default mode, in which, after the processor issues the ACT_SB_COMP command to activate the specified row, the memory 102 automatically starts to take out data from the column corresponding to col.0 for calculation, and all col.data of a row is calculated (for example, PIM-LPDDR5 memory, 64 columns of data are calculated).

[0146] S405: The memory determines that j = 0 and k = N.

[0147] S406: The memory determines j and k from the target field in the mode register.

[0148] S407: The memory performs row activation on one or more target Banks.

[0149] For details of S407, please refer to the above description of FIG. 6, which is not repeated here.

[0150] S408: The memory takes out col.j into the calculation unit for calculation.

[0151] S409: The memory determines whether the data of col.j is calculated.

[0152] Specifically, if the calculation is completed, S410 is performed.

[0153] S410: The memory increments j and decrements k.

[0154] Specifically, the memory increments j and decrements k to continue reading the next column data.

[0155] S411: The memory determines whether k is equal to 0.

[0156] Specifically, if k = 0, S412 is performed to end the calculation; if k ≠ 0, the next column data is read for calculation.

[0157] S412: The memory ends the calculation.

[0158] It should be noted that the above is described by taking the ACT_SB_COMP instruction as an example. The ACT_4B_COMP, ACT_AB_COMP and the like have similar execution flows as the ACT_SB_COMP, and the difference is that the ACT_4B_COMP performs calculation after activating a certain row of data of 4 banks at the same time, the ACT_AB_COMP performs calculation after activating a certain row of data of all banks at the same time, that is, the ACT_nB_COMP instruction represents that a certain row of data of n banks is activated at the same time, and the corresponding calculation is performed according to the configuration of the high concurrency calculation mode register.

[0159] In summary, after the processor issues a high concurrency calculation activation instruction, the memory 102 internally automatically performs reading and calculation of multiple column data, reducing the number of column strobe instructions issued. The method of realizing self-defined high concurrency calculation through the register configuration. By configuring the start column address of calculation, the calculation length, the number of Banks activated at the same time and the like in the register, the user-defined high concurrency calculation granularity is realized.

[0160] Please refer to FIG. 19, which is a flowchart of another control method provided by an embodiment of the application. The third instruction is used to read data from the memory 102 to the calculation unit of the memory 102 for calculation, and the third instruction is a column strobe instruction (which can also be referred to as a high concurrency column strobe calculation instruction) as an example for description.

[0161] As shown in FIG. 19, the control method can include:

[0162] S501: The processor sends a second instruction to the memory.

[0163] The second instruction is a row activation command. For detailed description of the second instruction, please refer to the related description of FIG. 10 above, which is not repeated here.

[0164] S502: The memory activates a row of one or more target Banks.

[0165] S503: The processor sends a third instruction to the memory.

[0166] Specifically, the third instruction, which can also be referred to as a high concurrency column strobe calculation instruction. Optionally, the third instruction can carry j and k; or the third instruction can carry j, and k is stored in advance through the mode register; or the third instruction can carry k, and j is stored in advance through the mode register.

[0167] In some embodiments, high concurrent column strobe command configurations are implemented by third instructions (which can be denoted as COMP_nB_xC, where x denotes an integer, and the value of x ranges from 1 to the total number of columns included in a row in the memory 102) and by the mode register. For example, the instructions COMP_SB_SC, COMP_SB_4C, COMP_SB_8C, COMP_SB_16C, COMP_SB_32C, and COMP_SB_AC respectively represent that a certain Bank is selected to sequentially perform data calculation on 1 column, 4 columns, 8 columns, 16 columns, 32 columns, and all columns starting from a specified starting column address. The instructions COMP_4B_SC, COMP_4B_4C, COMP_4B_8C, COMP_4B_16C, COMP_4B_32C, and COMP_4B_AC respectively represent that 4 Banks are simultaneously selected to sequentially perform data calculation on 1 column, 4 columns, 8 columns, 16 columns, 32 columns, and all columns starting from a specified starting column address. The instructions COMP_AB_SC, COMP_AB_4C, COMP_AB_8C, COMP_AB_16C, COMP_AB_32C, and COMP_AB_AC respectively represent that all Banks are simultaneously selected to sequentially perform data calculation on 1 column, 4 columns, 8 columns, 16 columns, 32 columns, and all columns starting from a specified starting column address.

[0168] In some embodiments, before sending the third instruction to the memory 102, the processor can also send configuration information including j or k to the memory 102. Based on the configuration information, the memory stores j or k in the mode register.

[0169] S504: The memory identifies the third instruction, and determines j and k based on the third instruction.

[0170] Specifically, when the third instruction includes j and k, j and k are determined based on the third instruction. When the third instruction includes j and the mode register stores k, j is determined based on the third instruction, and k is determined from the mode register. When the third instruction includes k and the mode register stores j, k is determined based on the third instruction, and j is determined from the mode register.

[0171] S505: The memory fetches col.j to the calculation unit for calculation.

[0172] S506: The memory determines whether the data of col.j is calculated.

[0173] If the calculation is completed, S507 is performed. If the calculation is not completed, S506 is continued.

[0174] S507: The memory increases j by 1 and decreases k by 1.

[0175] Specifically, the internal memory increases j and decreases k to continue reading the next column of data.

[0176] S508: The internal memory determines whether k is equal to 0.

[0177] If yes, S509 is executed; if no, S505 is executed.

[0178] S509: The internal memory ends the calculation.

[0179] To sum up, after the processor issues a high-concurrency column strobe instruction, the internal memory 102 automatically performs reading and calculation of multiple columns of data, reducing the number of times of issuing the column strobe instruction. The internal pipeline can shorten the calculation delay.

[0180] The embodiment of the application provides an internal memory which can be used to execute any one of the control methods.

[0181] The embodiment of the application provides a chip system, which comprises a processor and the internal memory.

[0182] The embodiment of the application provides an electronic device comprising the internal memory.

[0183] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0184] It should be noted that, for the above-mentioned method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the application is not limited to the action sequence described, because according to the application, some steps can be performed in other sequences or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the application.

[0185] In several embodiments provided by the application, it should be understood that the disclosed device can be implemented by other manners. For example, the device embodiments described above are only schematic, for example, the division of the above units is only a logical function division, and actual implementation can be in another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical or other forms.

[0186] The units described as separate components above can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0187] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0188] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc., and specifically can be a processor in the computer device) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium can include: U disk, mobile hard disk, magnetic disk, optical disk, read-only memory (Read-Only Memory, abbreviated: ROM) or random access memory (Random Access Memory, abbreviated: RAM), and various program codes that can be stored in the medium.

[0189] The above-described embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A control method characterized by, The method is applied to a memory, the memory is coupled with a processor, the memory comprises a plurality of memory banks, each of the memory banks comprises M rows*N columns of memory cells, M and N are integers greater than 0; the method comprises: receiving a first instruction sent by the processor, the first instruction being used for indicating a starting column address j and a column number k; k is an integer greater than 1 and less than or equal to N; determining a target bank and a target row according to the first instruction; reading target data in response to the first instruction, the target data being data stored in the jth column to the (j+k-1)th column of the target row in the target bank.

2. The method of claim 1, wherein, The memory further comprises a mode register, the mode register comprising a starting column address field and a column number field, the j is stored in the starting column address field and the k is stored in the column number field; acquiring the j from the starting column address field and the k from the column number field according to the first instruction.

3. The method of claim 2, wherein, The mode register further comprises a configuration mode field, if the configuration mode field is a first preset value, determining that the j=0 and the k=N-1; if the configuration mode field is a second preset value, acquiring the j from the starting column address field and the k from the column number field.

4. The method of claim 2 or 3, wherein, The mode register further comprises a target bank field; The method according to any one of claims 2-5, wherein determining the target bank and the target row according to the first instruction comprises: acquiring the target bank from the target bank field according to the first instruction; determining the target row according to the first instruction.

5. The method according to claim 4, wherein the target bank field comprises a plurality of target banks.

6. The method according to any one of claims 2-5, further comprising: receiving configuration information sent by the processor before receiving the first instruction, the configuration information comprising the j and the k; storing the j in the starting column address field and storing the k in the column number field.

7. A control method characterized by, The method is applied to a memory, the memory is coupled with a processor, the memory comprises a plurality of memory banks, each of the memory banks comprises M rows*N columns of memory cells, M and N are integers greater than 0; the method comprises: receiving a second instruction sent by the processor; determining a target bank and a target row according to the second instruction; receiving a third instruction sent by the processor, the third instruction being used for indicating a starting column address j and a column number k; k is an integer greater than 1 and less than or equal to N; reading target data in response to the third instruction, the target data being data stored in the jth column to the (j+k-1)th column of the target row in the target bank.

8. The method of claim 7, wherein, The third instruction comprises the j and the k; determining the j and the k according to the third instruction.

9. The method of claim 7, wherein, The memory further comprises a mode register, the mode register being preconfigured with the k; the third instruction comprises the j, and the method further comprises: determining the k from the mode register according to the third instruction.

10. The method of claim 7, wherein, The internal memory further comprises a mode register, the mode register being pre-configured with the j; the third instruction comprises the k, and the method further comprises: determining the j from the mode register according to the third instruction.

11. The method according to any one of claims 7 to 10, characterized in that, The internal memory further comprises a plurality of computing units, each of the Banks corresponding to a computing unit, and the method further comprises: sending the target data to the computing units corresponding to the target Banks respectively.

12. The method according to any one of claims 7-10, characterized in that, The method further comprises: sending the target data to the processor.

13. An in-memory device, comprising: The internal memory is configured to perform the control method according to any one of claims 1-6 or the control method according to any one of claims 7-12.

14. A chip system, characterized by The chip system comprises a processor and the internal memory according to claim 13.

15. An electronic device, comprising: The internal memory according to claim 13.

Images