Control method, internal memory and related chip system

Abstract

The embodiment of the invention discloses a control method, an internal memory and a related chip system, the control method is applied to the internal memory, the internal memory is coupled with a processor, the internal memory comprises a plurality of storage areas Bank, each Bank comprises M rows * N columns of storage units, and M and N are integers greater than 0; the method comprises the following steps: receiving a first instruction sent by a processor, wherein the first instruction is used for indicating an initial column address j and a column number k; k is an integer greater than 1 and less than or equal to N; determining a target Bank and a target row according to the first instruction; in response to the first instruction, target data is read, and the target data is data stored from the jth column to the (j + k-1) th column of the target row in the target Bank. By implementing the embodiment of the invention, the data reading efficiency can be improved, the calculation performance is improved, and the user experience is further improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent terminal technology, and in particular to a control method, an internal memory and a related chip system. Background Art

[0002] With the development of artificial intelligence (AI), generative AI has emerged, such as ChatGPT. Deploying large models on the terminal side for AI reasoning and improving user experience and productivity has become one of the main competitive advantages of the terminal. In the large model reasoning process, a large amount of data needs to be moved from the memory to the processing units (PU) in the memory, or the central processing unit (CPU), or the neural network processor (NPU), or the graphics processing unit (GPU) for calculation. As a result, the bottleneck of the current terminal-side AI reasoning performance is mainly concentrated on the data reading efficiency of the memory, that is, when reading data from the storage area (bank) of the memory to the computing unit for processing, the data reading efficiency is low, which will reduce the computing performance of the terminal. Therefore, how to improve data reading efficiency and computing performance is an urgent problem to be solved. Summary of the invention

[0003] The embodiments of the present application provide a control method, an internal memory and a related chip system to improve data reading efficiency, improve computing performance, and thereby improve user experience.

[0004] In a first aspect, an embodiment of the present application provides a control method, which is applied to an internal memory, wherein the internal memory is coupled to a processor, and the internal memory includes multiple storage areas Bank, each Bank includes M rows and N columns of storage units, and M and N are integers greater than 0; the method includes: receiving a first instruction sent by the processor, the first instruction is used to indicate a starting column address j and a number of columns k; k is an integer greater than 1 and less than or equal to N; determining a target Bank and a target row according to the first instruction; in response to the first instruction, reading target data, the target data being data stored in the jth column to the (j+k-1)th column of the target row in the target Bank.

[0005] In an embodiment of the present application, the internal memory includes multiple Banks, each Bank may include M rows * N columns of storage units, and each storage unit may be used to store data. In some computing scenarios, in order to improve computing efficiency, the processor will put the data into the storage units of the same row one by one according to the calculation order. When it is necessary to fetch data from the internal memory for calculation, the processor will send a first instruction to the internal memory, which is a data reading instruction proposed in the present application, and the first instruction can be used to indicate the starting column address j and the number of columns k in the M rows * N columns of storage units. When the internal memory receives the first instruction, the internal memory can read the data stored in the jth column storage unit to the (j+k-1)th column storage unit of the i-th row (i.e., the target row) of the target Bank in sequence based on j and k. In the prior art, after activating a row of data in a Bank, the processor needs to issue multiple instructions (each instruction includes only one column address) to read multiple data for calculation, which results in low data reading efficiency, and the instruction address bus has little idle time, making it difficult to perform multi-task concurrent operations. In the present application, since the first instruction can be used to indicate the starting column address j and the number of columns k, the internal memory can automatically read out k columns of data starting from the jth column based on the starting column address j and the number of columns k. The processor can read multiple columns of data by issuing one instruction, thereby improving data reading efficiency and further improving computing performance.

[0006] In some embodiments, the internal memory also includes a mode register, the mode register includes a starting column address field and a column number field, j is stored in the starting column address field and k is stored in the column number field; according to the first instruction, j is obtained from the starting column address field and k is obtained from the column number field.

[0007] In an embodiment of the present application, the first instruction may be a row activation instruction, and the data reading efficiency is improved by cooperating with the mode register through the first instruction. When the processor sends the first instruction to the internal memory, the internal memory will recognize the first instruction and determine j and k from the starting column address field and the column number field of the mode register. Furthermore, the internal memory can respond to the first instruction, and the internal memory can automatically read out k columns of data starting from the jth column based on the starting column address j and the column number k, avoiding the processor from having to issue multiple instructions to read multiple columns of data in the same row of storage units, thereby improving data reading efficiency, improving computing performance, and thus improving user experience.

[0008] In some embodiments, the mode register also includes a configuration mode field. If the configuration mode field is a first preset value, j=0 and k=N-1 are determined; if the configuration mode field is a second preset value, j is obtained from the starting column address field and k is obtained from the column number field.

[0009] In an embodiment of the present application, the mode register may include a configuration mode field. When the configuration mode field is a first preset value, such as 0, the default mode can be entered, j=0 and k=N-1 can be determined, and a whole row of data can be automatically and continuously read out; when the configuration mode field is a second preset value, such as 1, the custom mode can be entered, j can be obtained from the starting column address field, k can be obtained from the column number field, and the data stored from the jth column to the (j+k-1)th column can be automatically and continuously read out.

[0010] In some embodiments, the mode register also includes a target Bank field; the target Bank and the target row are determined according to the first instruction, specifically: the target Bank is determined from the target Bank field according to the first instruction; the target row is determined according to the first instruction.

[0011] In an embodiment of the present application, the mode register also includes a target bank field, which can record relevant information of the bank to be activated, such as the bank address or the index of the bank to be activated, so that the first instruction does not need to carry the address of the bank to be activated, and the bank to be activated can be determined from the target bank field of the mode register by identifying the first instruction.

[0012] In some embodiments, the Target Bank field includes multiple Target Banks.

[0013] In an embodiment of the present application, the target bank field may include information of multiple banks to be activated, such as multiple bank addresses or indexes of multiple banks to be activated. When the processor issues the first instruction, it can recognize the first instruction, determine multiple banks to be activated from the mode register, and activate multiple banks at the same time.

[0014] In some embodiments, before receiving the first instruction, configuration information sent by the processor is received, the configuration information includes j and k; j is stored in the starting column address field, and k is stored in the column number field.

[0015] In the embodiment of the present application, before receiving the first instruction, after the processor receives the target task, j and k can be determined according to the target task. The processor can first send configuration information to the internal memory to configure the mode register in the internal memory, so that j and k are pre-configured in the target field of the mode register (i.e., the start column address field and the column number field storage). When the target field of the mode register is pre-configured with j and k, the processor sends the first instruction to the internal memory, and the internal memory will recognize the first instruction and determine j and k from the mode register.

[0016] In a second aspect, an embodiment of the present application provides a control method, which is applied to an internal memory, wherein the internal memory is coupled to a processor, and the internal memory includes multiple storage areas Bank, each Bank includes M rows and N columns of storage units, and M and N are integers greater than 0; the method includes: receiving a second instruction sent by the processor; determining a target Bank and a target row according to the second instruction; receiving a third instruction sent by the processor, the third instruction is used to indicate a starting column address j and a number of columns k; k is an integer greater than 1 and less than or equal to N; in response to the third instruction, reading target data, the target data being data stored in the jth column to the (j+k-1)th column of the target row in the target Bank.

[0017] In an embodiment of the present application, the internal memory includes multiple Banks, each Bank may include M rows * N columns of storage units, and each storage unit may be used to store data. In some computing scenarios, in order to improve computing efficiency, the processor will put the data into the storage units of the same row one by one according to the calculation order. When it is necessary to fetch data from the internal memory for calculation, the processor will first send a second instruction to the internal memory to activate the row of the target Bank, and then the processor will send a third instruction to the internal memory, and the third instruction can be used to indicate the starting column address j and the number of columns k. When the internal memory receives the third instruction, the internal memory can read the data stored in the jth column storage unit to the (j+k-1)th column storage unit of the i-th row (i.e., the target row) in the target Bank based on j and k. In the prior art, after activating a row of data in a Bank, the processor needs to issue multiple instructions (each instruction includes only one column address) to read multiple data for calculation, which results in low data reading efficiency, and the instruction address bus has little idle time, making it difficult to perform multi-task concurrent operations. In the present application, since the sth third instruction can be used to indicate the starting column address j and the number of columns k, the internal memory can automatically read out k columns of data starting from the jth column based on the starting column address j and the number of columns k. The processor can read multiple columns of data by issuing one instruction, thereby improving data reading efficiency and further improving computing performance.

[0018] In some embodiments, the third instruction includes j and k; j and k are determined according to the third instruction.

[0019] In an embodiment of the present application, the third instruction is a column selection instruction, which carries j and k. j and k can be determined based on the third instruction alone, and then based on j and k, k columns of data can be automatically read starting from the jth column, avoiding the processor from issuing multiple instructions to read multiple columns of data, thereby improving instruction efficiency, improving data reading efficiency, and then improving computing performance.

[0020] In some embodiments, the internal memory further includes a mode register, in which k is pre-configured; the third instruction includes j, and the method further includes: determining k from the mode register according to the third instruction.

[0021] In the embodiment of the present application, the third instruction is a column selection instruction, and the third instruction can be used in conjunction with the mode register to improve data reading efficiency. When the mode register is pre-configured with k, and the third instruction includes j, the processor sends the third instruction to the internal memory, and the internal memory will recognize the third instruction, determine k from the mode register, and determine j from the third instruction, and then can automatically read out k columns of data starting from the jth column based on j and k, avoiding the processor from needing to issue multiple instructions to read multiple columns of data, thereby improving instruction efficiency, improving data reading efficiency, and then improving computing performance.

[0022] In some embodiments, the internal memory further includes a mode register, in which j is pre-configured; the third instruction includes k, and the method further includes: determining j from the mode register according to the third instruction.

[0023] In the embodiment of the present application, the third instruction is a column selection instruction, and the data reading efficiency can be improved by cooperating with the mode register through the third instruction. When the mode register is pre-configured with j, and the third instruction includes k, the processor sends the third instruction to the internal memory, and the internal memory will recognize the third instruction, determine j from the mode register, and determine k from the third instruction, and then can automatically read out k columns of data starting from the jth column based on j and k, avoiding the processor from needing to issue multiple instructions to read multiple columns of data, thereby improving instruction efficiency, improving data reading efficiency, and then improving computing performance.

[0024] In some embodiments, the internal memory further includes a plurality of computing units, each Bank corresponds to a computing unit, and the method further includes: sending the target data to the computing units corresponding to the target Banks respectively.

[0025] In an embodiment of the present application, after the internal memory automatically reads out k columns of data starting from the jth column based on the starting column address j and the number of columns k, the read data can be sent to the PU corresponding to the Bank for calculation, avoiding the processor from issuing multiple instructions to read multiple columns of data, thereby improving data reading efficiency and improving the computing performance of the computing unit.

[0026] In some embodiments, the method further includes: sending the target data to the processor respectively.

[0027] In an embodiment of the present application, after the internal memory automatically reads out k columns of data starting from the jth column based on the starting column address j and the number of columns k, the read data can be sent to the processor separately, avoiding the processor from issuing multiple instructions to read multiple columns of data, thereby improving data reading efficiency and improving the computing performance of the processor.

[0028] In a third aspect, an embodiment of the present application provides an internal memory, which can be used to execute any one of the control methods in the first aspect above; or, execute any one of the control methods in the second aspect above.

[0029] In a fourth aspect, an embodiment of the present application provides a chip system, which includes a processor and the internal memory described in the third aspect.

[0030] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising the internal memory described in the third aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a schematic diagram of a chip system provided in an embodiment of the present application.

[0032] Figure 2A A schematic diagram of the structure of an internal memory using in-memory computing technology provided in an embodiment of the present application.

[0033] Figure 2B A schematic diagram of the structure of a PIM-Die provided in an embodiment of the present application.

[0034] Figure 2C A schematic diagram of the structure of a Bank provided in an embodiment of the present application.

[0035] Figure 2D A schematic diagram of an instruction flow for reading data from an internal memory provided in an embodiment of the present application.

[0036] Figure 2E A schematic diagram of an instruction flow for reading data from an internal memory and executing a computing task is provided in an embodiment of the present application.

[0037] Figure 3 A flow chart of a control method provided in an embodiment of the present application.

[0038] Figure 4 A schematic diagram of the structure of an internal memory provided in an embodiment of the present application.

[0039] Figure 5 A schematic diagram of reading data from an internal memory provided in an embodiment of the present application.

[0040] Figure 6A schematic diagram of the structure of another internal memory provided in an embodiment of the present application.

[0041] Figure 7 A schematic diagram of an instruction bus provided in an embodiment of the present application.

[0042] Figure 8 A schematic diagram of an instruction flow for reading data provided in an embodiment of the present application.

[0043] Fig. 9 A flowchart of another control method provided in an embodiment of the present application.

[0044] Fig.10 A schematic diagram of the structure of another internal memory provided in an embodiment of the present application.

[0045] Fig.11 A schematic diagram of another instruction bus provided in an embodiment of the present application.

[0046] Fig.12 A schematic diagram of another instruction bus provided in an embodiment of the present application.

[0047] Fig.13 A schematic diagram of another instruction flow for reading data provided in an embodiment of the present application.

[0048] Fig.14 A schematic diagram of a mode register provided in an embodiment of the present application.

[0049] Fig.15 A schematic diagram of another mode register provided in an embodiment of the present application.

[0050] Fig.16 A schematic diagram of another internal memory provided in an embodiment of the present application.

[0051] Fig.17A A schematic diagram of a multiplication-accumulation calculation unit and a calculation pipeline provided in an embodiment of the present application.

[0052] Fig. 17B Another multiplication-accumulation calculation unit and calculation pipeline schematic diagram provided for an embodiment of the present application.

[0053] Fig.18 A flowchart of another control method provided in an embodiment of the present application.

[0054] Fig.19 A flowchart of another control method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0056] The terms "first", "second", "third" and "fourth" etc. in the specification and claims of the present application and the drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0057] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0058] The chip system involved in the embodiment of the present application is introduced below:

[0059] The present application embodiment provides a chip system. Figure 1 , Figure 1 1 is a schematic diagram of a chip system provided in an embodiment of the present application, and the chip system may include but is not limited to a system on chip 101 (System on chip, Soc), an internal memory 102, and an external memory 103. The chip system can be integrated in any electronic device, such as a computer, a mobile phone, a tablet, a personal digital assistant, a smart wearable device, a smart car or a smart home appliance, and the chip system can work under the necessary software driver.

[0060] System on chip 101 (System on chip, Soc) refers to a technology that integrates a complete system on a single chip and groups all or part of the necessary electronic circuits. The system on chip 101 may include multiple processing units and a controller 1014. Among them, the multiple processing units may include one or more of a central processing unit 1011 (Central Processing Unit, CPU), a modem processing unit, a graphics processing unit 1012 (graphics processing unit, GPU), an image signal processing unit (image signal processor, ISP), a video codec unit, a digital signal processing unit (digital signal processor, DSP), a baseband processing unit and a neural network processing unit 1013 (neural-network processing unit, NPU). Optionally, different processing units can communicate with each other through a primary bus, and each processing unit can also communicate with the controller 1014 through a primary bus.

[0061] The central processing unit 1011 may also be referred to as a central processing unit (CPU), or a processor. The central processing unit 1011 may be a multi-core processor, that is, a plurality of processor cores are integrated in one chip. The central processing unit 1011 may also be a single-core processor, which is not specifically limited in this application. The processor core is the abbreviation of the processor core, also known as the kernel, and is the most important component of the CPU. It is manufactured from single crystal silicon with a certain production process. All calculations, receiving instructions or storing instructions, and processing data of the CPU are performed by the processor core. The processor core can load the instructions or data stored in the external memory 103 (such as a disk, etc.) into the internal memory 102 through the controller 1014, and transfer the instructions or data that need to be calculated from the internal memory 102 to the processor core through the instruction bus or the data bus through the controller 1014 for calculation. After the calculation is completed, the processor core can temporarily store the result in the internal memory 102, and store the instructions or data that need to be stored for a long time in the external memory 103 through the controller 1014. The processor core can run an operating system, a file system (such as a flash file system F2FS) or an application program, etc., to control multiple hardware or software elements connected to the CPU, and can process various data and perform operations. Optionally, the central processing unit 1011 may include a memory, which may be a cache memory (Cache). The Cache may include one or more of a first-level cache (L1 Cache), a second-level cache (L2 Cache), a third-level cache (L3Cache), etc. The Cache can save instructions or data that the central processing unit 1011 has just used or circulated. If the central processing unit 1011 needs to use the instruction or data again, it can be directly called from the Cache. Repeated access is avoided, the waiting time of the processor core is reduced, and the efficiency of the central processing unit 1011 is improved.

[0062] The controller 1014 can be used to manage and control the communication between the central processing unit 1011 and the internal memory 102. The controller 1014 can also be used to manage and control the communication between the central processing unit 1011 and the external memory 103 (such as a disk, etc.), and provide a standardized interface (such as the universal flash storage UFS standard) for the communication between the central processing unit 1011 and the external storage device. It can also be understood that the central processing unit 1011 can send memory access instructions (such as read / write instructions), control instructions, etc. to the internal memory 102 through the controller 1014, and the central processing unit 1011 can also send memory access instructions (such as read / write instructions) to the external memory 103 through the controller 1014.

[0063] The internal memory 102 (Memory), referred to as memory for short, can be located outside the system on chip 101, and is usually a power-off volatile memory. The content stored on it will be lost when the power is off. It can also be called the main memory. The internal memory 102 in the present application includes a readable and writable operating memory, which is used to temporarily store the operation data required by the central processing unit 1011, and to exchange data with the external memory 103 or other external memories, and can be used as a storage medium for temporary data of the operating system or other running programs.

[0064] The internal memory 102 may include one or more of dynamic random access memory (DRAM), static random access memory (SRAM), synchronous dynamic random access memory (SDRAM), etc. Among them, DRAM includes double data rate synchronous dynamic random access memory (Double Data Rate Synchronous Dynamic Random Access Memory, DDRSDRAM), referred to as DDR, second generation double data rate synchronous dynamic random access memory (DDR2), third generation double data rate synchronous dynamic random access memory (DDR3), fourth generation low power double data rate synchronous dynamic random access memory (Low Power Double Data Rate4, LPDDR4) and fifth generation low power double data rate synchronous dynamic random access memory (Low Power Double Data Rate5, LPDDR5), etc.

[0065] The external memory 103 is a non-volatile memory, and its stored contents will not be lost after power failure. The external memory 103 can be used for long-term storage of instructions and data involved in the operation of the central processing unit 1011, such as startup programs, operating systems, applications and data. Since the central processing unit 1011 cannot directly read instructions and data in the external memory 103 or directly write instructions or data to the external memory 103, when the central processing unit 1011 executes a read (or load) instruction, it actually temporarily loads the contents to be read (including instructions and / or data) stored in the external memory 103 into the internal memory 102 through the controller 1014, and then the central processing unit 1011 reads it from the internal memory 102; and when executing a write (i.e. store) instruction, it actually temporarily writes the data to be stored (including instructions and / or data) into the internal memory 102 by the central processing unit 1011, and then stores it from the internal memory 102 to the external memory 103 through the controller 1014.

[0066] The external memory 103 may include one or more of Flash flash memory (for example, NAND flash memory, NOR flash memory, etc.), universal flash storage (UFS), embedded multimedia card eMMC, universal flash storage multi-chip package uMCP memory, embedded multimedia card multi-chip package eMCP memory, solid-state drive (SSD), etc.

[0067] Understandably, Figure 1 The chip systems in the embodiments are just some exemplary implementations provided in the embodiments of the present application. The chip systems in the embodiments of the present application include but are not limited to the above implementations.

[0068] With the development of artificial intelligence technology, generative AI has emerged, such as ChatGPT. Deploying large models on the terminal side for AI reasoning and improving user experience and productivity have become one of the main competitive advantages of the terminal. In the large model reasoning process, a large amount of data needs to be moved from the internal memory 102 to the central processing unit 1011, or the graphics processing unit 1012, or the neural network processing unit 1013 for calculation, resulting in the current bottleneck of terminal-side AI reasoning performance mainly concentrated on memory bandwidth. At present, in order to improve the AI ​​reasoning performance on the terminal side, memory computing technology (Processing in Memory, PIM) has become the mainstream, that is, multiple independent computing units (Processing Units, PU) can be integrated in the internal memory 102, and some computing tasks can be offloaded to the memory for calculation, which can improve the computing performance of the terminal and accelerate the reasoning of large models. At the same time, the use of memory computing technology can reduce data movement, reduce terminal energy consumption, and improve terminal endurance. The following will combine Figure 2A-2E The structure of the internal memory 102 and the instruction flow for reading data from the internal memory 102 are described in detail.

[0069] See also Figure 2A , Figure 2A A schematic diagram of the structure of an internal memory using in-memory computing technology provided in an embodiment of the present application, the internal memory 102 may include one or more ranks (Rank), each Rank in the multiple ranks refers to a collection of a group of memory chips (Die), and the same group of memory chips share the same address and control signals. Each Rank is usually 64 bits wide, which means that it can process 64 bits of data at the same time. The internal memory 102 can have one or more ranks, such as a single Rank (1R) and a dual Rank (2R). Among them, the dual Rank may include two ranks, each of which can be accessed independently, thereby improving the bandwidth and performance of the internal memory 102. For example, Figure 2AAs shown, Rank1 can include multiple Dies, Rank0 can be used independently to access data; Rank1 can include multiple PIM-Dies, PIM-Die is a Die with a computing unit (PU) integrated inside, Rank1 can be used independently to access data and perform computing tasks. It should be noted that the same Rank can include multiple Dies with the same structure or multiple Dies with different structures, which is not specifically limited in this application. Figure 2B This section introduces the structure of PIM-Die in detail.

[0070] See also Figure 2B , Figure 2B A schematic diagram of the structure of a PIM-Die provided in an embodiment of the present application, each PIM-Die may include multiple storage areas (Bank), such as Figure 2B BankA-BankJ shown in the figure. Bank is the basic unit for storing and managing data. Each bank can be regarded as a storage area that can process data requests independently. Different banks allow parallel access and operation, which means that the memory controller can access multiple banks at the same time, thereby improving data reading efficiency. For example, a PIM-Die includes 4 banks, and the data is stored in these 4 banks. When accessing data, the memory controller can activate the data in these 4 banks at the same time. Each PIM-Die also integrates multiple computing units (PUs), such as Figure 2B PU1-PU5 shown in FIG. PU is used to perform computing tasks, so as to directly perform computing tasks in the internal memory 102, instead of transferring data to the external on-chip system 101 for processing, thereby improving computing efficiency. Each PU can correspond to one or more banks, such as Figure 2B The PU1 shown in the figure can correspond to BankA and BankF, that is, the data read from BankA and BankF can be directly transferred to PU1 for calculation. Each PIM-Die can also integrate multiple mode registers (Mode Register, MR), such as Figure 2BMode register 1 and mode register 2 shown in . MR can be used to configure the operation mode and function of the internal memory 102, for example, to set parameters such as data width, delay, refresh rate, etc. The existence of MR enables the memory controller to configure and optimize the internal memory 102 according to the needs of the system, thereby improving performance and compatibility. By programming these registers, the system can adjust the behavior of the internal memory 102 to adapt to different application scenarios and performance requirements. Each PIM-Die can also include a global buffer, which can be used as a cache to store frequently accessed data to reduce access delays and improve system response speed. The following will combine Figure 2C Introduce the structure of Bank in detail.

[0071] See also Figure 2C , Figure 2C A schematic diagram of the structure of a Bank provided in an embodiment of the present application, wherein the Bank may include a storage array and a row buffer. The storage array is responsible for actual data storage, and is composed of a plurality of storage cells, which are usually organized in the form of rows and columns, that is, the storage array may be composed of M rows and N columns of storage cells, allowing efficient data access. The row buffer is used to store row data of a row read from the storage array. Figure 2D The instruction flow for reading data from the internal memory 102 is described in detail.

[0072] See also Figure 2D , Figure 2D The present invention provides an instruction flow diagram for reading data from an internal memory. Since data is stored in a storage array of a bank by row and column, and in order to improve data reading efficiency, when reading data from the internal memory 102, it is first necessary to activate a row of data in a bank, which can also be called row activation. The purpose of row activation is to load the data of a specific row in a bank into a temporary storage area of ​​a row buffer, so that the data in the row can be quickly accessed. Figure 2DAs shown, when data needs to be read from the internal memory 102, the CPU first needs to send a row selection instruction (also called an ACTIVAE activation instruction) to the internal memory 102 to activate the row. The row selection instruction includes a Bank address and a row (Row) address. For example, the Bank address is BankA and the Row address is RowM. The instruction can be used to activate the Mth row in the storage array of BankA. It should be noted that the ACTIVAE activation instruction of the standard DRAM memory will only activate a row in a Bank at a time. However, in order to save instruction overhead, ACT4 and ACT16 activation instructions are proposed for the PIM-DRAM in-memory computing memory, which respectively indicate the simultaneous activation of a row of 4 or 16 Banks in the DRAM memory, thereby improving data reading efficiency. Once the row is activated, the row data will be loaded into the row buffer of the corresponding Bank, and then the CPU can send a column selection instruction to the internal memory 102. The column selection instruction includes a column (column, col.) address, which is used to specify the specific column data of the row data stored in the row buffer. That is, the CPU sends a column selection instruction once to read the data of a certain column. For example, if the col. address is i, the column selection instruction is used to read the data of the i-th column. If it is necessary to read data from different columns of the same row, the CPU needs to send multiple column selection instructions to the internal memory 102 to read multiple columns of data. In addition, the CPU can also send a pre-charge instruction to the internal memory 102 to pre-charge the specified Bank and close the activated row. After the pre-charge operation is completed, the CPU can continue to send row selection instructions to the internal memory 102 to activate other rows in the Bank. The following will combine Figure 2E The instruction flow for reading data from the internal memory 102 and executing computing tasks is described in detail.

[0073] See also Figure 2E , Figure 2E A schematic diagram of an instruction flow for reading data from an internal memory and performing a computing task provided for an embodiment of the present application. The instruction flow for reading data from the internal memory 102 and performing a computing task is similar to the instruction flow for reading data from the internal memory 102. First, the CPU needs to send a row selection instruction to the internal memory 102 to activate the row, and then the CPU sends a calculation instruction (COMPUTE) to the internal memory 102. The calculation instruction includes a column address, which is used to specify a specific column of row data stored in the row buffer, and transfer the column data to the PU corresponding to the Bank for calculation, that is, the CPU sends a calculation instruction once, reads a certain column of data into the PU for calculation, for example, col. The address is j, and the calculation instruction is used to read the data of the jth column into the corresponding PU for calculation. If it is necessary to read data from different columns of the same row for calculation, the CPU needs to send multiple calculation instructions to the internal memory 102 to read multiple columns of data for calculation.

[0074] It has been found through research that there are a large number of matrix vector multiplication (GEMV) operations in the large model reasoning process, which involves continuously reading data of different columns in the same row from the Bank to the PU, or CPU, or NPU, or GPU for calculation. In the large model reasoning process, a large number of GEMV operations are involved, and it is necessary to continuously read data of different columns in the same row from the Bank to the PU for calculation. However, at present, when continuously reading data of different columns in the same row from the Bank, after activating a row of data in the Bank, since the CPU issues an instruction (such as the above-mentioned column selection instruction, or calculation instruction) once to read a column of data for calculation, the CPU needs to send multiple instructions to the internal memory 102 in sequence to read multiple columns of data in sequence for calculation, so the instruction efficiency is low, resulting in low data reading efficiency. At the same time, the instruction address bus has little idle time, and it is difficult to perform multi-task concurrent operations. Therefore, in this application, a high-concurrency instruction scheme is proposed, which can significantly reduce the number of times column selection instructions or calculation instructions are issued in the large model reasoning process, improve instruction efficiency and calculation flow, improve data reading efficiency, and further improve the performance of large model reasoning. It will be described in detail later, so it will not be repeated here.

[0075] The following describes the process of the control method provided in the embodiment of the present application.

[0076] Figure 3 A flow chart of a control method provided in an embodiment of the present application. The control method is applied to an internal memory 102, the internal memory 102 is coupled to a processor, the internal memory 102 includes a plurality of storage areas Bank, each Bank includes M rows*N columns of storage cells, and M and N are integers greater than 0.

[0077] like Figure 3 As shown, the control method may include:

[0078] S201: The internal memory receives a first instruction sent by a processor.

[0079] Specifically, the first instruction is used to indicate the starting column address j and the column number k of the storage unit corresponding to the data to be read; k is an integer greater than 1 and less than or equal to N. Figure 4 , Figure 4 A schematic diagram of the structure of an internal memory provided in an embodiment of the present application is shown in FIG. Figure 4The internal memory 102 includes multiple banks, such as BankA, BankB, and BankC. Each of the multiple banks includes a storage array, which can be composed of M rows and N columns of storage cells. Each storage cell can be used to store data, such as a storage cell that can store 256 bits of data. In the large model reasoning process, there are a large number of matrix-vector multiplication operations. Generally, in order to improve the computing efficiency, the processor will put the weight data into the storage cells of the same row but different columns one by one according to the calculation order, that is, the processor will first put the first set of weight data into the first column storage cell, and then put the second set of weight data into the second column storage cell, and so on. The logical positions of these data in the storage space are adjacent, but the column positions are different. After storing a row, the data is stored in the storage cells of the next row, and so on. When it is necessary to fetch data from the internal memory 102 for calculation, the processor will send a first instruction to the internal memory 102. The first instruction may be a row activation instruction (ACTIVATE') proposed in the present application. The first instruction may be used to indicate the starting column address j and the number of columns k of the storage cells corresponding to the data to be read in the M rows and N columns of storage cells; j is an integer less than N, and k is an integer greater than 0 and less than or equal to N, so that the internal memory 102 can subsequently automatically read out k columns of data starting from the jth column based on the starting column address j and the number of columns k, thereby improving data reading efficiency.

[0080] S202: The internal memory determines the target Bank and the target row according to the first instruction.

[0081] Specifically, the target Bank is a Bank storing data to be read among the multiple Banks, and the target behavior is a Bank to be activated.

[0082] S203: The internal memory reads target data in response to the first instruction.

[0083] Specifically, the target data is the data stored in the j-th column to the (j+k-1)-th column of the target row in the target Bank.

[0084] Specifically, the target Bank is a Bank storing the to-be-read data among the multiple Banks; i is an integer less than M. When the internal memory 102 receives the first instruction, the internal memory 102 can read the data stored in the storage cell in the jth column of the i-th row to the storage cell in the (j+k-1)th column in the target Bank based on j and k; when there are multiple target Banks, the data stored in the storage cell in the jth column of the i-th row to the storage cell in the (j+k-1)th column of each target Bank can be read in parallel.

[0085] For example, Figure 5 As shown, Figure 5A schematic diagram of reading data from an internal memory provided in an embodiment of the present application, wherein the target Bank may include one or more, Figure 5 As an example, it is assumed that the storage arrays in the target Bank are composed of 8*8 storage cells, i=1, j=2, k=5. When data needs to be taken from the internal memory 102 for calculation, the processor will send a first instruction to the internal memory 102. The first instruction can be used to indicate the starting column address j=2 and the number of columns k=5 of the storage cell corresponding to the data to be read. When the internal memory 102 receives the first instruction, the internal memory 102 can read the data stored in the storage cell of the second column to the storage cell of the fifth column of the first row of the target Bank based on j=2 and k=5. Optionally, if the internal memory 102 includes multiple target Banks, the data stored in the storage cell of the second column to the storage cell of the fifth column of the first row of each target Bank can be read in parallel.

[0086] In summary, since the instructions currently sent by the processor to the internal memory 102 cannot indicate the starting column address j and the number of columns k of the storage unit corresponding to the data to be read in the M-row*N-column storage unit, when continuously reading data of different columns in the same row from the Bank, after activating a row of data in the Bank, since the processor sends an instruction once (the instruction only includes one column address) to read a column of data for calculation, the processor needs to send multiple instructions to the internal memory 102 in sequence to read multiple columns of data in sequence for calculation, so the instruction efficiency is low, resulting in low data reading efficiency. At the same time, the instruction address bus has little idle time, making it difficult to perform multi-task concurrent operations. In the present application, since the first instruction can be used to indicate the starting column address j and the number of columns k of the storage unit corresponding to the data to be read in the M-row*N-column storage unit, the internal memory 102 can automatically read out k columns of data starting from the jth column in the target Bank in parallel based on the starting column address j and the number of columns k, avoiding the need for the processor to send multiple instructions to read multiple columns of data, thereby improving data reading efficiency, improving computing performance, and thus improving user experience.

[0087] In some embodiments, see Figure 6 , Figure 6 A schematic diagram of the structure of another internal memory provided in an embodiment of the present application, Figure 6 The internal memory 102 also includes a mode register, which includes a starting column address field and a column number field, j is stored in the starting column address field and k is stored in the column number field; j is obtained from the starting column address field and k is obtained from the column number field according to the first instruction.

[0088] Specifically, the internal memory 102 includes one or more mode registers, and one or more mode registers are pre-configured with j and k, that is, j and k can be stored in one mode register at the same time, or j can be stored in one mode register and k can be stored in another mode register, or information of j and k can be stored in more mode registers, which is not specifically limited in this application. It should be noted that, Figure 6 Only the case where j and k are pre-configured in mode register 1 is shown. When the processor sends a first instruction to the internal memory 102, the internal memory 102 will recognize the first instruction and determine j and k from the starting column address field and the column number field of the mode register. Furthermore, the internal memory 102 can respond to the first instruction, and the internal memory 102 can automatically read out k columns of data starting from the jth column based on the starting column address j and the column number k, avoiding the processor from needing to issue multiple instructions to read multiple columns of data of the same row of storage units, thereby improving data reading efficiency, improving computing performance, and thus improving user experience.

[0089] In one embodiment, the first instruction includes the bank address of the target bank and the row address of the i-th row storage unit; the internal memory 102 recognizes the first instruction, and based on the bank address and the row address, reads the data stored in the i-th row storage unit of the target bank into the row buffer of the corresponding bank.

[0090] Specifically, the first instruction may include the bank address of one or more target banks and the row address of the i-th row storage unit, so as to determine the target bank from multiple banks according to the bank address, and perform row activation according to the row address. In this embodiment, the first instruction is a row activation instruction (also referred to as a high-concurrency computing activation instruction), and the data reading efficiency is improved by cooperating with the mode register through the first instruction. When the processor sends the first instruction to the internal memory 102, the internal memory 102 will recognize the first instruction, and based on the bank address and row address of the target bank, the data stored in the i-th row storage unit of the target bank will be read into the row buffer of the corresponding bank. After the internal memory 102 recognizes the first instruction, j and k can also be determined from the mode register. Furthermore, the internal memory 102 can respond to the first instruction, based on j and k, and the internal memory 102 can automatically read out k columns of data starting from the j-th column based on the starting column address j and the number of columns k, so as to avoid the processor having to issue multiple instructions to read multiple columns of data of the same row storage unit, thereby improving data reading efficiency, improving computing performance, and thus improving user experience.

[0091] In some embodiments, the mode register is pre-configured with j and k corresponding to one or more target banks, and j and k corresponding to other banks can be stored in other mode registers.

[0092] In some embodiments, the internal memory 102 and the processor may be connected via an instruction bus, which may include L pins (PINs), the first M bits of which may be used to indicate the instruction function, and the last LM bits of which may be used to carry the bank address and the row address. Figure 7 As shown, Figure 7 A schematic diagram of an instruction bus provided for an embodiment of the present application, assuming that the instruction bus may include 8 pins (PIN), namely P0, P1, P2, P3, P4, P5, P6, and P7, wherein the first four bits can be used to indicate the instruction function, and the last four bits can carry the bank address and the row address. The first instruction (ACTIVATE') is a row activation instruction. Unlike the current row activation instruction, the first instruction can also be used to indicate the determination of the starting column address j and the number of columns k from the mode register, so the first four bits of the first instruction are different from the first four bits of the current row activation instruction. After receiving the instruction, the internal memory 102 can identify the first instruction through the first 4 bits, and then perform row activation and determine the starting column address j and the number of columns k from the starting column address field and the column number field in the mode register.

[0093] For example, Figure 8 As shown, Figure 8 A schematic diagram of an instruction flow for reading data provided in an embodiment of the present application, when data needs to be read from the internal memory 102, the processor can send a first instruction to the internal memory 102 to specify one or more Bank addresses and row address i. After receiving the first instruction, the internal memory 102 identifies the first instruction, activates the i-th row of one or more target banks (such as Bank A) corresponding to the one or more Bank addresses, and determines the starting column address j and the number of columns k from the mode register. For the target Bank, the internal memory 102 can automatically read out k columns of data starting from the j-th column based on the starting column address j and the number of columns k, which can significantly reduce the number of times the processor issues column selection instructions, thereby improving data reading efficiency, improving computing performance, and further improving user experience. In addition, if it is necessary to read data stored in other row storage units, the processor needs to first send a PRECHARGE instruction to the internal memory 102 to close the activated row, and then send the first instruction to activate a new row.

[0094] Another example Fig. 9 As shown, Fig. 9 A flow chart of another control method provided in an embodiment of the present application, the control method may include:

[0095] S301: The internal memory receives a second instruction sent by the processor.

[0096] Specifically, when data needs to be read from the internal memory 102, the processor must first send a second instruction to the internal memory 102. The second instruction is a row activation instruction (Activate). The second instruction specifies one or more target banks and the row address of the i-th row of storage cells to determine the target bank from multiple banks based on the bank address and perform row activation based on the row address.

[0097] S302: The internal memory determines the target Bank and the target row according to the second instruction.

[0098] Specifically, the target Bank is a Bank storing the to-be-read data among the multiple Banks, and the target row is a row in the target Bank.

[0099] S303: The internal memory receives the third instruction sent by the processor.

[0100] Specifically, the third instruction is used to indicate the starting column address j and the number of columns k; k is an integer greater than 1 and less than or equal to N. The third instruction is a column selection instruction (also referred to as a high-concurrency column selection calculation instruction). After the internal memory 102 is activated in response to the second instruction row, the starting column address j and the number of columns k indicated by the third instruction are used to improve data reading efficiency.

[0101] S304: The internal memory reads the target data in response to the third instruction.

[0102] Specifically, the target data is the data stored in the j-th column to the (j+k-1)-th column of the target row in the target Bank.

[0103] In summary, when the processor sends the second instruction to the internal memory 102, the internal memory 102 will activate the target Bank based on the second instruction, and read the data stored in the i-th row storage unit of the target Bank into the row buffer of the corresponding Bank. The processor then sends the third instruction to the internal memory 102, and the internal memory 102 can respond to the third instruction. The internal memory 102 can automatically read out k columns of data starting from the j-th column based on the starting column address j and the number of columns k, avoiding the processor from needing to issue multiple instructions to read multiple columns of data of the same row storage unit, thereby improving data reading efficiency, improving computing performance, and further improving user experience.

[0104] In some embodiments, see Fig.10 , Fig.10A structural diagram of another internal memory provided for an embodiment of the present application, wherein each of the Banks also includes a row buffer; the internal memory 102 receives a second instruction sent by the processor, and the second instruction may include the Bank address of one or more target Banks, and the row address of the i-th row storage unit; based on the Bank address and row address of one or more target Banks, the data stored in the i-th row storage unit of the one or more target Banks are read into the row buffer of the corresponding Bank respectively; in response to a third instruction, based on j and k, the data stored in the j-th column storage unit to the (j+k-1)-th column storage unit of the i-th row are read from the row buffer of the one or more target Banks respectively.

[0105] In some embodiments, the third instruction includes j and k; j and k are determined according to the third instruction.

[0106] Specifically, the third instruction is a column selection instruction, which carries j and k. j and k can be determined based on the third instruction alone, and then based on j and k, k columns of data can be automatically read starting from the jth column, avoiding the processor from issuing multiple instructions to read multiple columns of data, thereby improving instruction efficiency, improving data reading efficiency, and then improving computing performance.

[0107] In some embodiments, the internal memory 102 further includes a mode register, in which k is pre-configured; the third instruction includes j, and the method further includes: determining k from the mode register according to the third instruction.

[0108] Specifically, the third instruction is a column selection instruction, and the third instruction can be used in conjunction with the mode register to improve data reading efficiency. When the mode register is pre-configured with k and the third instruction includes j, the processor sends the third instruction to the internal memory 102, and the internal memory 102 will recognize the third instruction, determine k from the mode register, and determine j from the third instruction, and then can automatically read out k columns of data starting from the jth column based on j and k, avoiding the processor from needing to issue multiple instructions to read multiple columns of data, thereby improving instruction efficiency, improving data reading efficiency, and then improving computing performance.

[0109] In some embodiments, the internal memory 102 further includes a mode register, in which j is pre-configured; the third instruction includes k, and the method further includes: determining j from the mode register according to the third instruction.

[0110] Specifically, the third instruction is a column selection instruction, and the data reading efficiency can be improved by cooperating with the mode register through the third instruction. When the mode register is pre-configured with j, and the third instruction includes k, the processor sends the third instruction to the internal memory 102, and the internal memory 102 will recognize the third instruction, determine j from the mode register, and determine k from the third instruction, and then can automatically read out k columns of data starting from the jth column based on j and k, avoiding the processor from needing to issue multiple instructions to read multiple columns of data, thereby improving instruction efficiency, improving data reading efficiency, and then improving computing performance.

[0111] In summary, when the third instruction is a column selection instruction, the data reading efficiency can be improved through the third instruction, or through the third instruction and the mode register in cooperation.

[0112] In some embodiments, the internal memory 102 and the processor may be connected via an instruction bus, which may include L pins (PINs), the first M bits of which may be used to indicate the instruction function, and the last LM bits of which may be used to carry the starting column address j and the column number k. For example, see Fig.11 As shown, Fig.11 A schematic diagram of another instruction bus provided in an embodiment of the present application, assuming that the instruction bus may include 8 pins (PINs), namely P0, P1, P2, P3, P4, P5, P6, and P7, wherein the first four bits may be used to indicate the instruction function, and the last four bits may carry the starting column address j and the number of columns k. The third instruction is a column selection instruction, which is different from the current column selection instruction in that the last four bits of the third instruction may be used to carry the starting column address j and the number of columns k. After the internal memory 102 receives the third instruction, j and k may be determined by the last 4 bits.

[0113] In some embodiments, the first M bits can be used to indicate the instruction function, and the last LM bits can be used to carry the starting column address j. Since the number of pins of the instruction bus is limited, the number of columns k that can be represented is also limited. Therefore, k can be configured to be stored in the mode register of the internal memory 102 first, so as to improve the data reading efficiency through the third instruction and the mode register. For example, see Fig.12 As shown, Fig.12 A schematic diagram of another instruction bus provided in an embodiment of the present application, assuming that the instruction bus may include 8 pins (PINs), namely P0, P1, P2, P3, P4, P5, P6, and P7, wherein the first four bits may be used to indicate the instruction function, and the last four bits may carry the starting column address j. The third instruction is a column selection instruction, which is different from the current column selection instruction in that the first four bits of the third instruction may be used to indicate the determination of k from the mode register, and the last four bits may be used to carry the starting column address j. After the internal memory 102 receives the third instruction, k may be determined from the mode register through the first four bits, and j may be determined through the last four bits.

[0114] In some embodiments, the first M bits can be used to indicate the instruction function, and the last LM bits can be used to carry the number of columns k. Since the number of pins of the instruction bus is limited, the starting column address j that can be represented is also limited, so j can be first configured to be stored in the mode register of the internal memory 102, so as to improve the data reading efficiency through the third instruction and the mode register. Assume that the instruction bus may include 8 pins (PIN), namely P0, P1, P2, P3, P4, P5, P6, and P7, wherein the first four bits can be used to indicate the instruction function, and the last four bits can carry the number of columns k. The third instruction is a column selection instruction. Unlike the current column selection instruction, the first four bits of the third instruction can be used to indicate the determination of j from the mode register, and the last four bits can be used to carry k. After the internal memory 102 receives the third instruction, j can be determined from the mode register through the first 4 bits, and k can be determined through the last 4 bits.

[0115] For example, Fig.13 As shown, Fig.13 Another instruction flow diagram for reading data provided in an embodiment of the present application, when it is necessary to read data from the internal memory 102, the processor can send a second instruction to the internal memory 102 to specify one or more Bank addresses and row address i. After receiving the second instruction, the internal memory 102 identifies the second instruction and activates the i-th row of one or more target Banks corresponding to the one or more Bank addresses. Then, the processor sends a third instruction to the internal memory 102, and the third instruction includes a starting column address j. After the internal memory 102 identifies the third instruction, it can determine j based on the third instruction and determine k from the mode register. The internal memory 102 can automatically read out k columns of data starting from the j-th column based on the starting column address j and the number of columns k, which can significantly reduce the number of times the processor issues column selection instructions, thereby improving data reading efficiency, improving computing performance, and further improving user experience. In addition, if it is necessary to read data stored in other row storage units, the processor needs to first send a PRECHARGE instruction to the internal memory 102 to close the activated row, and then send a second instruction to activate a new row.

[0116] In some embodiments, for the above two control methods, the mode register includes a target field, and the method also includes: before receiving the first instruction and / or the third instruction, after the processor receives the target task, it can determine the configuration information according to the target task, and the configuration information includes j and / or k; the internal memory receives the configuration information sent by the processor, determines j and / or k according to the configuration information, and stores j and / or k in the target field of the mode register.

[0117] Specifically, the mode register may include multiple fields, which may be used to indicate different operation modes or information. Before receiving the first instruction or the third instruction, the processor may first send configuration information to the internal memory 102 to configure the mode register in the internal memory 102, so that j and / or k are pre-configured in the target field of the mode register. The target field includes a starting column address field and a column number field, the starting column address field is used to store j, and the column number field is used to store k. When the target field of the mode register is pre-configured with j and k, the processor sends a first instruction to the internal memory 102, and the internal memory 102 recognizes the first instruction and determines j and k from the mode register; when the target field of the mode register is pre-configured with j and the third instruction includes k, the processor sends a third instruction to the internal memory 102, and the internal memory 102 recognizes the third instruction, determines j from the mode register, and determines k from the third instruction; when the target field of the mode register is pre-configured with k and the third instruction includes j, the processor sends the third instruction to the internal memory 102, and the internal memory 102 recognizes the third instruction, determines k from the mode register, and determines j from the third instruction.

[0118] In some embodiments, the mode register also includes a target Bank field; the target Bank and the target row are determined according to the first instruction or the third instruction, specifically: the target Bank is determined from the target Bank field according to the first instruction or the third instruction; the target row is determined according to the first instruction or the second instruction.

[0119] Specifically, the mode register may also include a target bank field, which may record relevant information of the bank to be activated, such as the bank address or the index of the bank to be activated, so that the address of the bank to be activated does not need to be carried in the first instruction or the second instruction, and the bank to be activated can be determined from the target bank field of the mode register by identifying the first instruction or the second instruction.

[0120] In some embodiments, the Target Bank field includes multiple Target Banks.

[0121] In an embodiment of the present application, the target bank field may include information of multiple banks to be activated, such as multiple bank addresses or indexes of multiple banks to be activated. When the processor sends the first instruction or the second instruction, it can recognize the first instruction or the second instruction, determine multiple banks to be activated from the mode register, and activate multiple banks at the same time.

[0122] For example, Fig.14 As shown, Fig.14A schematic diagram of a mode register provided in an embodiment of the present application, wherein the mode register may include multiple fields, namely MR#, MA[6:0], Access, OP[7], OP[6], OP[5], OP[4], OP[3], OP[2], OP[1], and OP[0]. Among them, MR# represents the index of the mode register; MA[6:0] represents the address of the mode register; if Access is R / W, it means that the register can be read or written; OP[6]-OP[0] can be the target field for storing j and k. OP[7] can be a configuration mode field. Optionally, when OP[7] is configured to 0, it means j=0, k=N, that is, entering the default mode; when OP[7] is configured to 1, j and k are determined from OP[6]-OP[0], that is, entering the custom mode. Optionally, in custom mode, OP[6]-OP[3] is the start column address field, which is used to store the index startcol.index of the start column address j. 16 different start column address indexes can be configured, such as 0, 3, 7, 11, 15, 19, 23, 27, 31, 35, 39, 43, 47, 51, 55, 59. Optionally, OP[2]-OP[0] is the column number field, which is used to store the column number k in custom mode. 8 different k can be configured, such as 4, 8, 12, 16, 32, 48, 64.

[0123] For example, Fig.15 As shown, Fig.15 A schematic diagram of another mode register provided in an embodiment of the present application. Since the types of k and j that can be configured by OP[7]-OP[0] are limited, k and j can be stored in two mode registers. The mode register MRy implements the configuration of the starting column address j, and the mode register MRz implements the configuration of the number of columns k (y, z are non-negative integers, representing the index of the register). Among them, the MRy register can configure 64 different j through OP[5]-OP[0] (for the starting column address field); the MRz register can configure 64 different k through OP[5]-OP[0] (for the column number field).

[0124] Optionally, the processor may send configuration information to the internal memory 102, wherein the configuration information includes information of banks activated simultaneously, so that the target bank field in the mode register may be pre-configured with information of one or more target banks.

[0125] For example, Fig.15As shown, the information of Banks to be activated simultaneously can be configured through OP[7]-OP[6] (target Bank field) of the MRz register. Assume that the internal memory 102 includes 16 Banks, and this information can be used to indicate the status of 16 Bnaks. For example, a 16-bit number is used to represent the status of the 16 Banks, each bit corresponds to a Bank, 0 indicates that the Bank does not need to be activated, and 1 indicates that the Bank needs to be activated. The information of the Banks to be activated (including the address or index of the Bank, etc.) is configured in advance in the mode register. When the processor sends a row activation instruction to the internal memory 102, one or more target Banks can be activated simultaneously according to the Banks information in the mode register, so that the internal memory 102 can read data from multiple Banks at the same time, thereby improving data reading efficiency.

[0126] In some embodiments, the first instruction or the second instruction is also used to indicate the activation of one or more target banks. After receiving the first instruction, the internal memory 102 can determine the one or more target banks to be activated from the target bank field in the mode register; or, the second instruction is also used to indicate the activation of one or more target banks. After receiving the second instruction, the internal memory 102 can determine the one or more target banks to be activated from the target bank field in the mode register.

[0127] In some embodiments, the method further includes: sending the data stored in the jth column storage unit to the (j+k-1)th column storage unit in the i-th row in one or more target banks to the processor respectively.

[0128] Specifically, after the internal memory 102 automatically reads out k columns of data starting from the jth column based on the starting column address j and the number of columns k, the read data can be sent to the processor separately, avoiding the processor from issuing multiple instructions to read multiple columns of data, thereby improving data reading efficiency, improving the computing performance of the processor, and thus improving the user experience.

[0129] In some embodiments, the internal memory also includes multiple computing units, each bank corresponds to a computing unit, and the method also includes: sending the data stored in the storage unit from the jth column of the i-th row to the (j+k-1)th column of the storage unit in the read one or more target banks to the computing units corresponding to the one or more target banks respectively.

[0130] Specifically, Fig.16 As shown, Fig.16A schematic diagram of another internal memory provided for an embodiment of the present application, in addition to multiple banks, such as Bank A, Bank B, and Bank C, the internal memory 102 also includes multiple computing units, such as PU1, PU2, and PU3. Each Bank can correspond to a computing unit, such as Bank A corresponds to PU1, Bank B corresponds to PU2, and Bank C corresponds to PU3. A computing unit can correspond to one or more banks, that is, the data read out from a bank can be put into the corresponding PU for calculation, and a PU can receive data sent by one or more banks and perform calculations. When the internal memory 102 automatically reads out k columns of data starting from the jth column based on the starting column address j and the number of columns k, the read data can be sent to the PUs corresponding to the Banks for calculation, avoiding the need for the processor to issue multiple instructions to read multiple columns of data, thereby improving data reading efficiency, improving the computing performance of the computing unit, and thus improving user experience.

[0131] For example, Fig.17A and Fig. 17B As shown, Fig.17A A schematic diagram of a multiplication-accumulation calculation unit and a calculation pipeline provided in an embodiment of the present application, assuming that the delay of taking out a column (col.) of data to complete the calculation operation is T1, such as Fig.17A As shown, the prior art solution issues eight calculation instructions to complete the calculation of col.j to col.j+7 columns of data with a delay of 8*T1, and the figure only shows that three instructions are issued. Fig. 17B A schematic diagram of a multiplication-accumulation calculation unit and a calculation pipeline provided in an embodiment of the present application is shown in FIG. Fig. 17B As shown, through high-concurrency column-select computing instructions, when the processor issues an instruction once, multiple columns of data can be automatically read out for calculation. Assuming that the delay for taking out a column (col.) of data to complete the calculation operation is T1, and the delay for automatically reading out a column of data and completing the calculation operation is T2, T2 is less than T1, and the solution of the present application completes the calculation delay of col.j~col.j+7 columns of data with a delay of T1+7*T2, (T1+7*T2)<8*T1, so the solution of the present application can effectively shorten the calculation delay.

[0132] It should be noted that in the large model training scenario, when performing calculations in the internal memory 102, the weight parameter W of the large model is generally stored in the Bank, and the input vector V will be transferred from the processor to the internal memory 102 and stored in the global buffer. Therefore, the current technical solution needs to issue a GBW instruction (Global Buffer Write, GBW) first, transfer all input vectors V to the global buffer inside the internal memory 102, and then issue an activation instruction and multiple column selection calculation instructions to complete the calculation. In the present application, after the processor issues the first instruction, such as COMP_SB_AC, the internal memory 102 will automatically and sequentially retrieve data from multiple columns for calculation, and the process of writing the input vector to the global buffer can be mostly buried in the high-concurrency calculation process. A GBW instruction is issued to transfer a small amount of input vector data, such as 64 bytes, to the internal memory 102 to meet the input vector data required for calculating the weight data of the first few columns; then an activation instruction and a high-concurrency column selection calculation instruction are issued to start sequentially taking out data from multiple columns from the specified starting column address for calculation; while calculating, a GBW instruction is issued to transfer the remaining input vector data to calculate with the weight parameters stored in the corresponding columns.

[0133] In addition, the internal memory 102 may include multiple Ranks, each Rank corresponds to a memory chipset, which are connected to the same command address and data bus, and need to address different memory chipsets through chip select signals (chip select), that is, select different Ranks. At present, when performing calculation operations, it is necessary to continuously issue multiple column selection calculation instructions to calculate multiple data, and the command address bus is always busy. In the present application, after a high-concurrency calculation instruction is issued to a memory corresponding to a certain Rank, the memory will automatically take out data from multiple columns for calculation. At this time, the command address bus is idle and can be switched to other Ranks for access. After a high-concurrency calculation instruction is issued to the PIM-DRAM corresponding to Rank0, the command address bus can be switched to Rank1 for read and write access, thereby realizing multi-task concurrent operations.

[0134] See also Fig.18 , Fig.18 A flow chart of another control method provided for an embodiment of the present application is provided, which is illustrated by taking as an example an example a first instruction for reading data from an internal memory 102 into a computing unit of the internal memory 102 for calculation, and the first instruction is a row activation instruction (also referred to as a high-concurrency computing activation instruction).

[0135] like Fig.18 As shown, the control method may include:

[0136] S401: The processor sends configuration information to the internal memory.

[0137] Specifically, the configuration information includes a starting column address j and a column number k, wherein the column number k can also be understood as a calculation length.

[0138] S402: The internal memory receives the configuration information, and configures the mode register based on the configuration information to store j and k in the mode register.

[0139] Specifically, the internal memory 102 includes multiple mode registers, j and k can be stored in the same mode register or in multiple mode registers. For details, please refer to the above description of Fig.14 , Fig.15 The relevant description will not be repeated here.

[0140] S403: The processor sends a first instruction to the internal memory.

[0141] Specifically, the first instruction can be represented by ACT_nB_COMP, where n represents an integer ranging from 1 to the total number of banks in the internal memory 102. For example, ACT_4B_COMP means activating the data in 4 banks at the same time for calculation; ACT_SB_COMP and ACT_AB_COMP respectively represent activating a single bank and activating the data in all banks for calculation. The first instruction may include the bank address of one or more target banks, and the row address of the i-th row of storage cells. For a detailed description of the first instruction, please refer to the above description. Figure 6-Figure 8 The relevant description will not be repeated here.

[0142] S404: The internal memory recognizes the first instruction and determines whether the data stored in the configuration mode field in the mode register is 1.

[0143] Specifically, when the data stored in the configuration mode field is not 1, S405 is executed; when the data stored in the configuration mode field is 1, S406 is executed. Assume that the name of the mode register is MRx, and OP[7] is the configuration mode field.

[0144] It should be noted that when OP[7]=1 of the MRx register, it indicates that the custom mode has been entered. In this mode, after the processor issues the ACT_SB_COMP command to activate the specified row, the internal memory 102 automatically starts to fetch data from the column corresponding to the start column address (startcol.index) for calculation, and automatically executes in sequence until the data calculation of the calculated length is completed. When OP[7]=0 of the MRx register, it indicates that the default mode has been entered. In this mode, after the processor issues the ACT_SB_COMP command to activate the specified row, the internal memory 102 automatically starts to fetch data from the column corresponding to col.0 for calculation until all col. data of a row are calculated (taking the PIM-LPDDR5 memory as an example, the data of 64 columns are calculated).

[0145] S405: The internal memory determines j=0, k=N.

[0146] S406: The internal memory determines j, k from the target field in the mode register.

[0147] S407: The internal memory performs row activation on one or more target banks.

[0148] For a detailed description of S407, please refer to the above Figure 6 The relevant description will not be repeated here.

[0149] S408: fetch col.j from the internal memory and send it to the calculation unit for calculation.

[0150] S409: The internal memory determines whether the calculation of the data of col.j is completed.

[0151] Specifically, if the calculation is completed, S410 is executed.

[0152] S410: The internal memory will j++, k--.

[0153] Specifically, the internal memory will j++, k-- to continue reading the next column of data.

[0154] S411: The internal memory determines whether k is equal to 0.

[0155] Specifically, if k=0, S412 is executed to end the current calculation; if k≠0, the next column of data is read to perform the calculation.

[0156] S412: The internal memory ends calculation.

[0157] It should be noted that the above description uses the ACT_SB_COMP instruction as an example. The execution process of instructions such as ACT_4B_COMP and ACT_AB_COMP is similar to that of ACT_SB_COMP. The difference is that ACT_4B_COMP simultaneously activates a row of data in 4 Banks and then performs calculations, while ACT_AB_COMP simultaneously activates a row of data in all Banks and then performs calculations, that is, the ACT_nB_COMP instruction indicates that after activating a row of data in n Banks at the same time, corresponding calculations are performed according to the configuration of the high-concurrency calculation mode register.

[0158] In summary, after the processor issues a high-concurrency computing activation instruction, the internal memory 102 will automatically execute the reading and calculation of multiple columns of data, reducing the number of times the column selection instruction is issued. A method for implementing customized high-concurrency computing through register configuration. By configuring the starting column address of the calculation, the calculation length, the number of banks activated at the same time, and other information in the register, the user-defined high-concurrency computing granularity is realized.

[0159] See also Fig.19 , Fig.19 A flow chart of another control method provided in an embodiment of the present application is taken as an example in which a third instruction is used to read data from the internal memory 102 into the computing unit of the internal memory 102 for calculation, and the third instruction is a column selection instruction (also referred to as a high-concurrency column selection calculation instruction).

[0160] like Fig.19 As shown, the control method may include:

[0161] S501: The processor sends a second instruction to the internal memory.

[0162] The second instruction is a row activation command. For a detailed description of the second instruction, please refer to the above description. Fig.10 The relevant description will not be repeated here.

[0163] S502: The internal memory performs row activation on one or more target banks.

[0164] S503: The processor sends a third instruction to the internal memory.

[0165] Specifically, the third instruction may also be called a high-concurrency column gating calculation instruction. Optionally, the third instruction may carry j and k; or, the third instruction may carry j and store k in advance through a mode register; or, the third instruction may carry k and store j in advance through a mode register.

[0166] In some embodiments, a high-concurrency column selection command configuration is implemented through a third instruction (which can be represented by COMP_nB_xC) and a mode register (x represents an integer, and the value range is 1 to the total number of columns included in a row in the internal memory 102). For example, instructions such as COMP_SB_SC, COMP_SB_4C, COMP_SB_8C, COMP_SB_16C, COMP_SB_32C, and COMP_SB_AC respectively indicate the selection of a certain Bank and the execution of 1 column, 4 columns, 8 columns, 16 columns, 32 columns, and all columns of data in sequence from the specified starting column address for calculation. The instructions COMP_4B_SC, COMP_4B_4C, COMP_4B_8C, COMP_4B_16C, COMP_4B_32C, and COMP_4B_AC respectively indicate that 4 banks are selected at the same time and the data of 1 column, 4 columns, 8 columns, 16 columns, 32 columns, and all columns are calculated in sequence starting from the specified starting column address. The instructions COMP_AB_SC, COMP_AB_4C, COMP_AB_8C, COMP_AB_16C, COMP_AB_32C, and COMP_AB_AC respectively indicate that all banks are selected at the same time and the data of 1 column, 4 columns, 8 columns, 16 columns, 32 columns, and all columns are calculated in sequence starting from the specified starting column address.

[0167] In some embodiments, before the processor sends the third instruction to the internal memory 102, the processor may also send configuration information to the internal memory 102, where the configuration information includes j or k. The internal memory stores j or k in the mode register based on the configuration information.

[0168] S504: The internal memory recognizes the third instruction, and determines j and k based on the third instruction.

[0169] Specifically, when the third instruction includes j and k, j and k are determined based on the third instruction. When the third instruction includes j and k is stored in the mode register, j is determined based on the third instruction and k is determined from the mode register. When the third instruction includes k and j is stored in the mode register, k is determined based on the third instruction and j is determined from the mode register.

[0170] S505: fetch col.j from the internal memory and send it to the calculation unit for calculation.

[0171] S506: The internal memory determines whether the calculation of the data of col.j is completed.

[0172] If the process is completed, S507 is executed, and if the process is not completed, S506 is continued to be executed.

[0173] S507: The internal memory will j++, k--.

[0174] Specifically, the internal memory will j++, k-- to continue reading the next column of data.

[0175] S508: The internal memory determines whether k is equal to 0.

[0176] If yes, execute S509; if no, execute S505.

[0177] S509: The internal memory ends the calculation.

[0178] In summary, after the processor issues a high-concurrency calculation column selection instruction, the internal memory 102 will automatically perform the reading and calculation of multiple columns of data, reducing the number of times the column selection instruction is issued. The calculation delay can be shortened by internal pipelining.

[0179] An embodiment of the present application provides an internal memory, which can be used to execute any one of the above control methods.

[0180] An embodiment of the present application provides a chip system, which includes a processor and the above-mentioned internal memory.

[0181] An embodiment of the present application provides an electronic device, comprising the above-mentioned internal memory.

[0182] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0183] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0184] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the above-mentioned units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0185] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0186] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0187] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform all or part of the steps of the above-mentioned methods of each embodiment of the present application. Among them, the aforementioned storage medium may include: U disk, mobile hard disk, magnetic disk, optical disk, read-only memory (Read-Only Memory, abbreviated: ROM) or random access memory (Random Access Memory, abbreviated: RAM) and other media that can store program codes.

[0188] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A control method, characterized in that: Applied to an internal memory, the internal memory is coupled to a processor, the internal memory includes a plurality of storage areas Bank, each of the Banks includes M rows and N columns of storage units, M and N are integers greater than 0; the method includes: receiving a first instruction sent by the processor, wherein the first instruction is used to indicate a starting column address j and a column number k; k is an integer greater than 1 and less than or equal to N; Determine the target bank and the target bank according to the first instruction; In response to the first instruction, the target data is read, and the target data is the data stored in the jth column to the (j+k-1)th column of the target row in the target Bank.

2. The method according to claim 1, characterized in that: The internal memory further comprises a mode register, the mode register comprising a start column address field and a column number field, the j is stored in the start column address field and the k is stored in the column number field; According to the first instruction, the j is obtained from the starting column address field, and the k is obtained from the column number field.

3. The method according to claim 2, characterized in that: The mode register also includes a configuration mode field, If the configuration mode field is a first preset value, determining that j=0 and k=N-1; If the configuration mode field is a second preset value, the j is obtained from the start column address field, and the k is obtained from the column number field.

4. The method according to claim 2 or 3, characterized in that: The mode register also includes a target Bank field; The determining of the target bank and the target row according to the first instruction is specifically: Determine the target Bank from the target Bank field according to the first instruction; The target row is determined according to the first instruction.

5. According to the method of claim 4, the target bank field includes multiple target banks.

6. The method according to any one of claims 2 to 5, further comprising: Before receiving the first instruction, receiving configuration information sent by the processor, the configuration information including the j and the k; The j is stored in the start column address field, and the k is stored in the column number field.

7. A control method, characterized in that: Applied to an internal memory, the internal memory is coupled to a processor, the internal memory includes a plurality of storage areas Bank, each of the Banks includes M rows and N columns of storage units, M and N are integers greater than 0; the method includes: receiving a second instruction sent by the processor; Determine the target bank and the target line according to the second instruction; receiving a third instruction sent by the processor, wherein the third instruction is used to indicate a starting column address j and a column number k; k is an integer greater than 1 and less than or equal to N; In response to the third instruction, the target data is read, and the target data is the data stored in the jth column to the (j+k-1)th column of the target row in the target Bank.

8. The method according to claim 7, characterized in that: The third instruction includes the j and the k; The j and the k are determined according to the third instruction.

9. The method according to claim 7, characterized in that: The internal memory further includes a mode register, in which the k is pre-configured; the third instruction includes the j, and the method further includes: The k is determined from the mode register according to the third instruction.

10. The method according to claim 7, characterized in that: The internal memory further includes a mode register, in which j is pre-configured; the third instruction includes k, and the method further includes: The j is determined from the mode register according to the third instruction.

11. The method according to any one of claims 7 to 10, characterized in that: The internal memory further includes a plurality of computing units, each of the Banks corresponds to a computing unit, and the method further includes: The target data are respectively sent to the computing units corresponding to the target Banks.

12. The method according to any one of claims 7 to 10, characterized in that: The method further comprises: The target data are sent to the processors respectively.

13. An internal memory, characterized in that: The internal memory is used to execute the control method described in any one of claims 1 to 6, or to execute the control method described in any one of claims 7 to 12.

14. A chip system, characterized in that: The chip system includes a processor and the internal memory as claimed in claim 13.

15. An electronic device, characterized in that: Includes the internal memory as claimed in claim 13 above.

Images