In-memory processing device, method, chip and electronic device
By having the controller access the storage unit based on a standard protocol in the in-memory processing device, and the processing unit actively performing calculations, the problem of modifying the memory protocol is solved, and PIM is made easy to apply and highly versatile.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING HOUMO TECH CO LTD
- Filing Date
- 2024-12-19
- Publication Date
- 2026-07-10
AI Technical Summary
Existing technologies require modifications to the memory standard protocol and controller for in-memory processing, which increases the difficulty of deploying PIM in practice.
An in-memory processing apparatus and method are provided, in which an instruction sequence and operands are written into a memory unit by a controller, the processing unit actively reads and performs calculations, and the controller accesses the memory based on a standard protocol, thus avoiding modification of the memory protocol.
This reduces the difficulty of implementing PIM and improves its versatility and compatibility with different controllers.
Smart Images
Figure CN122363591A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of integrated circuit technology, and in particular to an in-memory processing apparatus, method, chip, and electronic device. Background Technology
[0002] With the continuous development of artificial intelligence and big data, the demand for computing power in various application scenarios is constantly increasing. However, since the mainstream computing architecture adopts the von Neumann architecture with memory-compute separation, the growth rate of memory bandwidth is far behind the growth rate of processor computing power. Therefore, there is a memory wall problem, where insufficient bandwidth limits the actual computing power of the computing system. To solve the memory wall problem, Processing-In-Memory (PIM) technology has emerged. In this technology, computing units are added to memory to perform calculations, thereby reducing data transfer between the processor and memory. However, after adding computing units to memory, the memory protocol needs to be modified. For example, the JEDEC (Joint Electron Device Engineering Council) standard DDR (Double Data Rate Synchronous Dynamic Random-Access Memory) protocol needs to be modified. For computational operations within PIM, the controller operating PIM needs to be modified, and computation-related instructions need to be added. This increases the difficulty of deploying PIM in practical applications. Summary of the Invention
[0003] The embodiments of this disclosure provide an in-memory processing apparatus, method, chip, and electronic device that can avoid modifying the controller and the memory standard protocol, enabling the controller to access the storage units in PIM based on the standard memory protocol, effectively reducing the difficulty of deploying PIM in applications.
[0004] According to one aspect of the present disclosure, an in-memory processing apparatus is provided, comprising:
[0005] Controller;
[0006] The in-memory processing module includes a storage unit and a processing unit;
[0007] The controller is configured to write the instruction sequence to be executed and the operands to be calculated into the storage unit;
[0008] The processing unit is configured as follows:
[0009] Read the instruction sequence and the operands from the storage unit;
[0010] The operands are processed based on the instruction sequence, and the processing result is written to the storage unit.
[0011] In some embodiments, the controller is further configured to:
[0012] After writing the instruction sequence to be executed and the operands to be calculated into the storage unit, the first storage address information of the instruction sequence and the second storage address information of the operands are written into the first designated storage area in the storage unit.
[0013] The processing unit is specifically configured as follows:
[0014] Read the first storage address information of the instruction sequence and the second storage address information of the operand from the first designated storage area;
[0015] Read the instruction sequence from the first storage area corresponding to the first storage address information;
[0016] Read the operand from the second storage area corresponding to the second storage address information;
[0017] The instruction sequence is executed to process the operands and the processing result is written to a third storage area specified in the instruction sequence.
[0018] In some embodiments, the storage unit includes a first number of storage array libraries, each storage array library including at least one storage array; the in-memory processing module includes a second number of processing units; the first number of storage array libraries are connected to the second number of processing units in a preset arrangement.
[0019] The controller is specifically configured to write the instruction sequence and operands corresponding to each of the second number of processing units into the storage array library corresponding to each processing unit.
[0020] Each of the second number of processing units is configured to: read the instruction sequence and operands corresponding to the processing unit from the memory array library corresponding to the processing unit; process the operands based on the instruction sequence; and write the processing result into the memory array library corresponding to the processing unit.
[0021] In some embodiments, the processing unit is further configured to:
[0022] After executing the instruction sequence, update the value of the second designated storage area of the storage unit;
[0023] The controller is further configured to: read the value of the second designated storage area and determine that the instruction sequence has been executed based on the value of the second designated storage area.
[0024] In some embodiments, the processing unit is specifically configured as follows:
[0025] After executing the instruction sequence, the third storage address information of the processing result is written into the second designated storage area;
[0026] The controller is specifically configured to: read the third storage address information of the processing result from the second designated storage area, and read the processing result from the corresponding third storage area according to the third storage address information.
[0027] In some embodiments, it also includes:
[0028] A pre-defined standard interface is connected to the controller and the memory processing module, respectively.
[0029] The controller is specifically configured to write the instruction sequence and the operands into the storage unit through the interface of the preset standard.
[0030] According to another aspect of the present disclosure, an in-memory processing method is provided, comprising:
[0031] The controller writes the sequence of instructions to be executed and the operands to be calculated into the storage unit in the in-memory processing module;
[0032] The processing unit in the in-memory processing module reads the instruction sequence and the operands from the storage unit;
[0033] The processing unit processes the operands based on the instruction sequence and writes the processing result into the storage unit.
[0034] In some embodiments, after the controller writes the sequence of instructions to be executed and the operands to be computed into the storage unit in the in-memory processing module, the method further includes:
[0035] The controller writes the first storage address information of the instruction sequence and the second storage address information of the operand into the first designated storage area in the storage unit;
[0036] The processing unit reads the first storage address information of the instruction sequence and the second storage address information of the operand from the first designated storage area;
[0037] The processing unit reads the instruction sequence from the first storage area corresponding to the first storage address information;
[0038] The processing unit reads the operand from the second storage area corresponding to the second storage address information;
[0039] The processing unit executes the instruction sequence, processes the operands, and writes the processing result into a third storage area specified in the instruction sequence.
[0040] In some embodiments, after the processing unit executes the instruction sequence, the processing unit writes the third storage address information of the processing result into the second designated storage area;
[0041] The controller reads the third storage address information of the processing result from the second designated storage area, and reads the processing result from the corresponding third storage area according to the third storage address information.
[0042] According to another aspect of the present disclosure, a chip is provided, including: the in-memory processing apparatus provided in any of the above embodiments.
[0043] According to another aspect of the present disclosure, an electronic device is provided, comprising: a chip provided in any of the above embodiments.
[0044] The in-memory processing apparatus, method, chip, and electronic device provided in the above embodiments of this disclosure, since the controller only needs to write the instruction sequence corresponding to the processing unit as the data to be written into the storage unit in the PIM, and write the operands to be calculated into the storage unit in the PIM, and the processing unit in the PIM actively reads the instruction sequence and calculates the operands based on the instruction sequence, on the controller side, the controller can avoid being aware of the specific instructions in the instruction sequence. The controller only needs to read and write to the storage unit of the PIM based on the standard memory protocol, thereby avoiding the need to add support for calculation instructions to the controller, avoiding modification of the standard memory protocol, effectively reducing the difficulty of deploying PIM in applications, and improving the versatility of PIM.
[0045] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0046] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.
[0047] Figure 1 This is an exemplary application scenario of the in-memory processing device provided in the embodiments of this disclosure;
[0048] Figure 2 This is a schematic diagram of the structure of an in-memory processing apparatus provided in an exemplary embodiment of the present disclosure;
[0049] Figure 3 This is a schematic diagram of the structure of an in-memory processing apparatus provided in another exemplary embodiment of this disclosure;
[0050] Figure 4 This is a schematic diagram of the structure of an in-memory processing apparatus provided in yet another exemplary embodiment of the present disclosure;
[0051] Figure 5 This is a structural block diagram of an in-memory processing apparatus provided in an exemplary embodiment of the present disclosure;
[0052] Figure 6 This is a flowchart illustrating an exemplary embodiment of the present disclosure of an in-memory processing method.
[0053] Figure 7 This is a schematic flowchart of an in-memory processing method provided in another exemplary embodiment of this disclosure;
[0054] Figure 8 This is a flowchart illustrating an in-memory processing method provided in yet another exemplary embodiment of this disclosure. Detailed Implementation
[0055] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.
[0056] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values of the components and steps set forth in these embodiments do not limit the scope of this disclosure.
[0057] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.
[0058] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.
[0059] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.
[0060] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.
[0061] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.
[0062] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.
[0063] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.
[0064] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.
[0065] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.
[0066] This disclosure outlines
[0067] Figure 1 This is an exemplary application scenario of the in-memory processing device provided in this disclosure. For example... Figure 1As shown, in applications such as artificial intelligence and big data, at least one processor (represented as xPU 10 in the figure) from CPU (Central Processing Unit), GPU (Graphics Processing Unit), and NPU (Neural Processing Unit) can generate computational tasks for the in-memory processing device 20. These computational tasks may include a sequence of instructions to be executed and operands. xPU 10 sends the computational tasks to the in-memory processing device 20. The in-memory processing device 20 includes a controller 21 and an in-memory processing (PIM) module 22. The controller 21 writes the instruction sequence and operands into the storage units of the PIM module 22. The processing units in the PIM module 22 read the instruction sequence and operands from the storage units, process the operands based on the instruction sequence, and write the processing results back to the storage units, specifically to a designated storage area within the storage units. The controller 21 can read the processing results from the designated storage area and return them to xPU 10, or return the storage address information of the processing results to xPU 10. Based on the standard memory protocol for reading and writing PIM, the controller 21 implements in-memory processing, avoiding the need to add PIM calculation-related instructions to the controller 21. This effectively reduces the difficulty of deploying PIM and improves its versatility. In practical applications, the controller 21 can be set in the xPU 10, and is not limited to the setup shown in the figure.
[0068] Exemplary device
[0069] Figure 2 This is a schematic diagram of the structure of an in-memory processing device provided in an exemplary embodiment of this disclosure. The various components of this in-memory processing device can be integrated into a single chip or disposed in discrete components, with data communication links established between these discrete components. For example... Figure 2 As shown, the memory processing device 20 may include a controller 21 and a memory processing module 22.
[0070] The in-memory processing module (which may be called the PIM module or simply PIM) 22 includes a storage unit 221 and a processing unit 222.
[0071] The controller 21 is configured to write the sequence of instructions to be executed and the operands to be calculated into the storage unit 221.
[0072] The processing unit 222 is configured to: read instruction sequence and operands from storage unit 221; process the operands based on the instruction sequence; and write the processing result to storage unit 221.
[0073] The sequence of instructions to be executed and the operands to be computed can be obtained from the computing tasks issued by the external processor.
[0074] In some optional embodiments, the instruction sequence may include one or more instructions, which instruct the computation array in processing unit 222 to perform corresponding operations on the operands. For example, the instruction sequence may include MAC (multiply-add) instructions, instructions for calculating activation functions (AF), element-wise multiplication (EWMUL) instructions, and so on. This is merely an exemplary instruction sequence illustration; in actual applications, the types and representations of instructions included in the instruction sequence can be set according to actual needs and are not limited to the instruction sequence in the example above.
[0075] In some optional embodiments, the computing task may also include storage address information of instruction sequence and storage address information of operands. The controller 21 writes the instruction sequence and operands into the designated storage area in the storage unit 221 according to the storage address information of instruction sequence and storage address information of operands.
[0076] In some optional embodiments, the controller 21 can determine the storage address information of the instruction sequence and the storage address information of the operands, and then write the instruction sequence and operands into the designated storage area in the storage unit 221 according to the storage address information of the instruction sequence and the storage address information of the operands. The specific method for determining the storage location of the instruction sequence and operands is not limited.
[0077] In some optional embodiments, storage unit 221 may include one or more storage array banks, and the number of processing units 222 may be one or more, without limitation. For example, each bank may correspond to one processing unit 222, or multiple banks may share one processing unit 222. For example, taking storage unit 221 as having 32 banks (represented as bank0 to bank31), a processing unit 222 may be set between bank0 and bank16, between bank1 and bank17, ..., and between bank15 and bank31.
[0078] In some optional embodiments, the controller 21 may be located in an external processor or set up independently of an external processor. The external processor may include one or more processors such as a CPU, GPU, and NPU. The in-memory processing module 22 is a module consisting of a computing array of processing units added to the memory's storage array. The computing array may include, for example, an array for performing matrix operations, logical operations, etc. Specifically, it may include a multiply-accumulate (MAC) array, or logical operation structures such as AND, OR, and NOT operations. The specific structure of the processing units can be set according to actual needs, and this disclosure does not limit the specific structure.
[0079] In some optional embodiments, after the controller 21 writes the instruction sequence to be executed and the operands to be calculated into the storage unit 221, it can write the storage address information of the instruction sequence and the storage address information of the operands into a preset storage area in the storage unit 221, and the processing unit 222 can obtain the storage address information of the instruction sequence and the operands from the preset storage area.
[0080] The in-memory processing device provided in this embodiment only requires the controller to write the instruction sequence corresponding to the processing unit as data to be written into the storage unit within the PIM, and to write the operands to be calculated into the storage unit within the PIM. The processing unit within the PIM actively reads the instruction sequence and performs calculations on the operands based on the instruction sequence. On the controller side, the controller can avoid being aware of the specific instructions in the instruction sequence. The controller only needs to read and write to the storage unit of the PIM based on the standard memory protocol. This avoids adding support for calculation instructions to the controller and avoids modifying the standard memory protocol, effectively reducing the difficulty of deploying PIM in applications and improving the versatility of PIM.
[0081] In some alternative embodiments, in the above... Figure 2 Based on the embodiment shown, the controller 21 is further configured to: after writing the instruction sequence to be executed and the operands to be calculated into the storage unit 221, write the first storage address information of the instruction sequence and the second storage address information of the operands into the first designated storage area in the storage unit 221.
[0082] The processing unit 222 is specifically configured to: read the first storage address information of the instruction sequence and the second storage address information of the operand from the first designated storage area; read the instruction sequence from the first storage area corresponding to the first storage address information; read the operand from the second storage area corresponding to the second storage address information; execute the instruction sequence, process the operand, and write the processing result into the third storage area specified in the instruction sequence.
[0083] The first storage address information of the instruction sequence is related information that can determine the storage address of the instruction sequence. For example, the first storage address information may include the row address and length of the stored instruction sequence. The second storage address information of the operand is related information that can determine the storage address of the operand. For example, the second storage address information may include the row address and length of the stored operand. The specific content of the storage address information can be set according to actual needs and is not limited to the representation of row address and length. The first designated storage area can be a pre-configured designated storage area for storing the first storage address information and the second storage address information. The first designated storage area is the storage area specified by the controller 21 and the PIM module protocol. For example, address information corresponding to the first designated storage area can be pre-configured in controller 21. After the controller 21 writes the instruction sequence and operands into storage unit 221, it writes the first storage address information for storing the instruction sequence and the second storage address information for storing the operands into the first designated storage area according to the address information corresponding to the first designated storage area, in a pre-configured format. This triggers the processing unit 222 in PIM module 22 to read the first and second storage address information. Responding to the write operation to the first designated storage area, the processing unit 222 reads the instruction sequence from the first storage area corresponding to the first storage address information and the operands from the second storage area corresponding to the second storage address information. It then executes each instruction according to the instruction sequence, processes the operands, and writes the processing result into the third storage area specified in the instruction sequence. The third storage area is the storage area in storage unit 222 used to store the processing result.
[0084] In the embodiments of this disclosure, after the controller 21 writes the instruction sequence and the operands to be calculated into the storage unit 221, it writes the first storage address information of the instruction sequence and the second storage address information of the operands into a first designated storage area in the storage unit 221. This allows the processing unit 222 to read the first and second storage address information of the instruction sequence from the first designated storage area. Based on the first and second storage address information, it can read the instruction sequence and operands and complete the operand calculation. This allows the controller 21 to access the PIM module 22 based on a standard memory access protocol to achieve in-memory computation, avoiding the need to add parsing functions for in-memory computation-related calculation instructions and controlling the PIM based on calculation instructions to the controller 21. The controller 21 does not need to be aware of the calculation instructions; it only treats the instruction sequence as data to be written into the storage unit 221 and writes the instruction sequence into the storage unit through standard memory read / write access. This improves the compatibility of the PIM module 22 with different controllers and greatly reduces the difficulty of deploying PIM in applications.
[0085] Figure 3This is a schematic diagram of the structure of an in-memory processing apparatus provided in another exemplary embodiment of the present disclosure.
[0086] In some optional embodiments, based on the above embodiments, the storage unit 221 includes a first number of storage array banks (or memory array banks) 2211, each storage array bank 2211 including at least one memory array (or simply array); the in-memory processing module 22 includes a second number of processing units 222; the first number of storage array banks 2211 are connected to the second number of processing units 222 according to a preset arrangement.
[0087] The first and second quantities can be arbitrary and are not limited in this embodiment. For example, the first quantity can be 1, 2, 3, 4, ... The second quantity can be 1, 2, 3, 4, ... The number of storage arrays included in each Bank 2211 can be set according to the bit width requirements of the Bank 2211. For example, an 8-bit Bank can be constructed using 8 storage arrays, and 8 bits of data can be read at a time. The preset arrangement can be set according to actual needs. For example, the preset arrangement can include one Bank corresponding to one processing unit 222, multiple Banks corresponding to one processing unit, etc. The specific arrangement is not limited.
[0088] The controller 21 is specifically configured to write the instruction sequence and operands corresponding to each of the second number of processing units 222 into the memory array library 2211 corresponding to each processing unit 222.
[0089] Each of the second number of processing units 222 is configured to: read the instruction sequence and operands corresponding to the processing unit 222 from the memory array library 2211 corresponding to the processing unit 222; process the operands based on the instruction sequence; and write the processing result into the memory array library 2211 corresponding to the processing unit 222.
[0090] In this second set of processing units 222, the instruction sequences of any two processing units 222 can be the same or different. When the instruction sequences of all processing units 222 are the same, the controller 21 writes the same instruction sequence into each processing unit 222. The operands to be calculated by each processing unit 222 can be different to achieve parallel computing and improve processing efficiency. For example, multiple sets of operands can be calculated by multiple processing units 222, or data with a higher bit width (bit width greater than the operand bit width supported by the processing unit) can be split into multiple sets of operands that conform to the bit width of the processing unit 222, and the higher bit width data can be calculated by multiple processing units 222. The instruction sequence may include a storage instruction for the processing result, which includes the storage address information corresponding to the processing result. By executing the storage instruction, the processing result is written to a specified storage area in the storage array library 2211 corresponding to the processing unit 222.
[0091] It should be noted that, Figure 3 This is merely an exemplary arrangement of processing unit 222 and bank 2211. In practical applications, the number of processing units, the number of banks, and the number of banks corresponding to each processing unit can be set according to actual needs, and are not limited to the arrangement shown in the figure. For example, each processing unit can access 1 bank, 2 banks, 3 banks, ... That is, one or more banks share one processing unit.
[0092] In some optional embodiments, based on any of the above embodiments, the processing unit 222 is further configured to:
[0093] After executing the instruction sequence, the value of the second designated storage area of storage unit 221 is updated.
[0094] Controller 21 is also configured to: read the value of the second specified storage area and determine the completion of the instruction sequence based on the value of the second specified storage area.
[0095] The second designated storage area of storage unit 221 can be a storage area agreed upon by the protocol. After the processing unit 222 finishes executing the instruction sequence, it updates the value of the second designated storage area so that the controller 21 can sense the completion of the calculation of PIM module 22, so that the controller 21 can obtain the processing result in a timely manner.
[0096] In some optional embodiments, the third storage area for the processing result can be a storage area agreed upon by the PIM module 22 and the controller 11. After the controller 21 detects an update to the value of the second designated storage area, it reads the processing result from the third storage area and returns the processing result to the task issuer. For example... Figure 1 xPU in the middle.
[0097] In some optional embodiments, after the processing unit 222 has executed the instruction sequence, it can update the value of the second designated storage area of the storage unit 221 to a preset value. In response to reading the preset value from the second designated storage area, the controller 21 determines that the processing unit 222 has executed the instruction sequence.
[0098] In the embodiments of this disclosure, after the processing unit finishes executing the instruction sequence, it can update the value of the second designated storage space so that the controller can effectively sense that the processing unit has finished executing the instruction sequence and can read the processing result in a timely manner.
[0099] In some optional embodiments, based on any of the above embodiments, the processing unit 222 is further configured to: after executing the instruction sequence, write the third storage address information of the storage processing result into the second designated storage area in the storage unit 221.
[0100] The controller 21 is specifically configured to: read the third storage address information of the processing result from the second designated storage area, and read the processing result from the corresponding third storage area according to the third storage address information.
[0101] After executing the instruction sequence, the processing unit 222 can write the address information of the third storage area storing the processing result (i.e., the third storage address information) into the second designated storage area. The second designated storage area can be found in the above embodiment. The controller 21 can then read the third storage address information of the processing result from the second designated storage area, and further read the processing result from the corresponding third storage area based on the third storage address information. For example, a read instruction can be generated based on the third storage address information and sent to the PIM module. In response to the read instruction, the PIM module sends the processing result stored in the third storage area corresponding to the third storage address information to the controller 21. The controller 21 can then return the processing result to the task sender.
[0102] In embodiments of this disclosure, after the processing unit executes the instruction sequence, it writes the address information storing the processing result into a second designated storage area, so that the controller can read the processing result in a timely manner.
[0103] Figure 4 This is a schematic diagram of the structure of an in-memory processing apparatus provided in yet another exemplary embodiment of the present disclosure.
[0104] In some alternative embodiments, based on any of the above embodiments, such as Figure 4 As shown, the memory processing device in this embodiment may further include a preset standard interface 23.
[0105] The preset standard interface 23 is connected to the controller 21 and the memory processing module 22 respectively.
[0106] The controller 21 is specifically configured to write instruction sequences and operands into the storage unit 221 through a preset standard interface 23.
[0107] The preset standard can be the protocol standard corresponding to the type of memory used to construct the PIM module. For example, if the memory is DDR, the preset standard is the DDR protocol standard; if the memory is LPDDR, the preset standard is the LPDDR protocol standard. Memory types can include LPDDR4, LPDDR5, DDR4, DDR5, GDDR (Graphics Double Data Rate Synchronous Dynamic Random Access Memory) 5, GDDR6, etc. Regardless of the type of memory into which the processing unit is inserted to implement PIM, it can be implemented using the device of this embodiment. The PIM module operates based on the standard protocol, avoiding the need to add parsing support for calculation-related instructions on the controller side. The interface of the preset standard is the communication interface between the controller 21 and the PIM module, and can be any interface of the relevant protocol in the related technology. This embodiment does not limit it.
[0108] In some alternative embodiments, Figure 5 This is a structural block diagram of an in-memory processing apparatus provided in an exemplary embodiment of this disclosure. For example... Figure 5As shown, taking the insertion of processing unit 222 into LPDDR (Low Power Double Data Rate Synchronous Dynamic Random-Access Memory) to form PIM module 22 as an example, xPU10 represents an external processor, and controller 21 is the controller in xPU10. Controller 21 communicates with PIM module 22 through a standard LPDDRIP interface, and can send instructions and addresses (e.g., read instructions and addresses, write instructions and addresses) to PIM module 22, send write data (i.e., data to be written into PIM module 22) to PIM module 22, and read data from PIM module 22, etc. The storage unit 221 in PIM module 22 includes 32 Banks 2211, namely Bank2211-0, Bank2211-2, ..., Bank2211-15, Bank2211-16, Bank2211-17, ..., Bank2211-31. In this configuration, Bank2211-0 and Bank2211-16 share processing unit 222-0, Bank2211-1 and Bank2211-17 share processing unit 222-1, ..., Bank2211-15 and Bank2211-31 share processing unit 222-15. The rectangular bars of different gray levels within each Bank2211 can represent storage areas storing different data, such as storage areas for instruction sequences, operands, designated storage areas for the address information of instruction sequences and operands, storage areas for processing results, and so on. In practical applications, the xPU10 can control at least one Bank2211 and at least one processing unit 222 in the PIM module 22 to perform in-memory computation in parallel through the controller 21, thus completing the xPU10's computational tasks. The specific computational functions of the processing unit 222 can be set according to actual needs, and this embodiment does not limit them.
[0109] In some optional embodiments, the processing unit 222 can be implemented in hardware, hardware + software, or other ways. The processing unit 222 may include a computing array for performing calculations, an instruction fetch subunit for reading instruction sequences, a buffer for buffering input data of the computing array, a buffer for buffering output data, a control subunit for bank control, and so on. The specific structure of the processing unit 222 can be set according to actual needs and is not limited to the exemplary structure described above, as long as it can achieve the corresponding functions of the embodiments of this disclosure. This disclosure does not impose limitations on the embodiments.
[0110] Related technologies require the addition of support for PIM calculation-related calculation instructions in the controller. The controller then controls the PIM calculation based on the calculation instructions. Therefore, the standard protocol of the memory needs to be modified, which makes it difficult to put PIM into application.
[0111] The in-memory processing apparatus of this disclosure embodiment can operate PIM using the standard JEDEC memory interface protocol, without needing to add calculation-related instructions to the external controller 21 of the PIM module, avoiding modifications to the memory standard protocol, greatly reducing the difficulty of deploying PIM in applications, and improving the versatility of PIM and its compatibility with controllers of various standard protocols.
[0112] The various embodiments and optional examples disclosed herein can be implemented individually or in any combination without conflict, and can be set according to actual needs.
[0113] Exemplary methods
[0114] Figure 6 This is a schematic flowchart of an exemplary embodiment of the present disclosure providing an in-memory processing method. This method can be implemented using the in-memory processing apparatus of any of the above embodiments. For example... Figure 6 The method shown may include the following steps:
[0115] Step 510: The controller writes the instruction sequence to be executed and the operands to be calculated into the storage unit in the in-memory processing module.
[0116] Step 520: The processing unit in the in-memory processing module reads the instruction sequence and operands from the storage unit.
[0117] Step 530: The processing unit processes the operands based on the instruction sequence and writes the processing result to the storage unit.
[0118] Figure 7 This is a schematic flowchart of an in-memory processing method provided in another exemplary embodiment of this disclosure.
[0119] In some alternative embodiments, based on any of the above embodiments, such as Figure 7 As shown, after the controller in step 510 writes the instruction sequence to be executed and the operands to be calculated into the storage unit in the in-memory processing module, the method of this embodiment may further include:
[0120] Step 610: The controller writes the first storage address information of the instruction sequence and the second storage address information of the operand into the first designated storage area in the storage unit.
[0121] The processing unit in the in-memory processing module in step 520 reads the instruction sequence and operands from the memory unit, which may include:
[0122] Step 5210: The processing unit reads the first storage address information of the instruction sequence and the second storage address information of the operand from the first designated storage area.
[0123] Step 5220: The processing unit reads the instruction sequence from the first storage area corresponding to the first storage address information.
[0124] Step 5230: The processing unit reads the operands from the second storage area corresponding to the second storage address information.
[0125] The processing unit in step 530 processes the operands based on the instruction sequence and writes the processing result to the storage unit, including:
[0126] Step 5310: The processing unit executes the instruction sequence, processes the operands, and writes the processing result into the third storage area in the storage unit specified in the instruction sequence.
[0127] In some optional embodiments, the storage unit includes a first number of storage array libraries, each storage array library including at least one storage array; the in-memory processing module includes a second number of processing units; the first number of storage array libraries are connected to the second number of processing units in a preset arrangement.
[0128] Step 510, where the controller writes the sequence of instructions to be executed and the operands to be computed into the storage unit of the in-memory processing module, may include:
[0129] The controller writes the instruction sequence and operands corresponding to the second number of processing units into the memory array library corresponding to each processing unit.
[0130] The process unit in the in-memory processing module in step 520 reads the instruction sequence and operands from the storage unit, which may include: each of the second number of processing units reading the instruction sequence and operands corresponding to the processing unit from the storage array library corresponding to the processing unit; processing the operands based on the instruction sequence; and writing the processing result into the storage array library corresponding to the processing unit.
[0131] Figure 8 This is a flowchart illustrating an in-memory processing method provided in yet another exemplary embodiment of this disclosure.
[0132] In some alternative embodiments, based on any of the above embodiments, such as Figure 8 As shown, the method in this embodiment of the disclosure may further include:
[0133] Step 710: After the processing unit finishes executing the instruction sequence, the processing unit writes the third storage address information of the processing result into the second designated storage area.
[0134] Step 720: The controller reads the third storage address information of the processing result from the second designated storage area, and reads the processing result from the corresponding third storage area according to the third storage address information.
[0135] In some optional embodiments, based on any of the above embodiments, the method of this disclosure embodiment may further include: after the processing unit finishes executing the instruction sequence, the processing unit updates the value of the second designated storage area of the storage unit; the controller reads the value of the second designated storage area and determines that the instruction sequence has been completed based on the value of the second designated storage area.
[0136] In some alternative embodiments, the controller writes instruction sequences and operands into the storage unit through a preset standard interface.
[0137] The above embodiments can be implemented individually or in any combination without conflict, and the embodiments disclosed herein are not limited.
[0138] The specific operations and beneficial effects of each step in this method embodiment can be found in the aforementioned corresponding device embodiments, and will not be repeated here.
[0139] This disclosure also provides a chip, including the in-memory processing device provided in any of the above embodiments.
[0140] In some alternative embodiments, the chip may further include at least one processor. The processor includes at least one of a CPU, GPU, NPU, etc.
[0141] This disclosure also provides an electronic device including the chip provided in any of the above embodiments.
[0142] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.
[0143] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.
[0144] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.
[0145] The apparatus of this disclosure may be implemented in many ways. For example, it may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the method in the apparatus is for illustrative purposes only, and the steps of the method of this disclosure are not limited to the order specifically described above, unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the method according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the method according to this disclosure.
[0146] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.
[0147] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.
[0148] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.
Claims
1. An in-memory processing device, comprising: Controller; The in-memory processing module includes a storage unit and a processing unit; The controller is configured to write the instruction sequence to be executed and the operands to be calculated into the storage unit; The processing unit is configured as follows: Read the instruction sequence and the operands from the storage unit; The operands are processed based on the instruction sequence, and the processing result is written to the storage unit.
2. The apparatus according to claim 1, wherein, The controller is also configured to: After writing the instruction sequence to be executed and the operands to be calculated into the storage unit, the first storage address information of the instruction sequence and the second storage address information of the operands are written into the first designated storage area in the storage unit. The processing unit is specifically configured as follows: Read the first storage address information of the instruction sequence and the second storage address information of the operand from the first designated storage area; Read the instruction sequence from the first storage area corresponding to the first storage address information; Read the operand from the second storage area corresponding to the second storage address information; The instruction sequence is executed to process the operands and the processing result is written to a third storage area specified in the instruction sequence.
3. The apparatus according to claim 1, wherein, The storage unit includes a first number of storage array libraries, each storage array library including at least one storage array; the in-memory processing module includes a second number of processing units; the first number of storage array libraries are connected to the second number of processing units according to a preset arrangement. The controller is specifically configured to write the instruction sequence and operands corresponding to each of the second number of processing units into the storage array library corresponding to each processing unit. Each of the second number of processing units is configured to: read the instruction sequence and operands corresponding to the processing unit from the memory array library corresponding to the processing unit; process the operands based on the instruction sequence; and write the processing result into the memory array library corresponding to the processing unit.
4. The apparatus according to claim 1, wherein, The processing unit is further configured to: After executing the instruction sequence, update the value of the second designated storage area of the storage unit; The controller is further configured to: read the value of the second designated storage area and determine that the instruction sequence has been executed based on the value of the second designated storage area.
5. The apparatus according to claim 4, wherein, The processing unit is specifically configured as follows: After executing the instruction sequence, the third storage address information of the processing result is written into the second designated storage area; The controller is specifically configured to: read the third storage address information of the processing result from the second designated storage area, and read the processing result from the corresponding third storage area according to the third storage address information.
6. The apparatus according to any one of claims 1-4, further comprising: A pre-defined standard interface is connected to the controller and the memory processing module, respectively. The controller is specifically configured to write the instruction sequence and the operands into the storage unit through the interface of the preset standard.
7. A memory processing method, comprising: The controller writes the sequence of instructions to be executed and the operands to be calculated into the storage unit in the in-memory processing module; The processing unit in the in-memory processing module reads the instruction sequence and the operands from the storage unit; The processing unit processes the operands based on the instruction sequence and writes the processing result into the storage unit.
8. The method according to claim 7, wherein, After the controller writes the sequence of instructions to be executed and the operands to be calculated into the storage unit of the in-memory processing module, the method further includes: The controller writes the first storage address information of the instruction sequence and the second storage address information of the operand into the first designated storage area in the storage unit; The processing unit reads the first storage address information of the instruction sequence and the second storage address information of the operand from the first designated storage area; The processing unit reads the instruction sequence from the first storage area corresponding to the first storage address information; The processing unit reads the operand from the second storage area corresponding to the second storage address information; The processing unit executes the instruction sequence, processes the operands, and writes the processing result into a third storage area specified in the instruction sequence.
9. The method according to claim 7 or 8, wherein, After the processing unit finishes executing the instruction sequence, the processing unit writes the third storage address information of the processing result into the second designated storage area; The controller reads the third storage address information of the processing result from the second designated storage area, and reads the processing result from the corresponding third storage area according to the third storage address information.
10. A chip, comprising: The memory processing apparatus according to any one of claims 1-6.
11. An electronic device comprising: The chip according to claim 10.