Memory device and operating method thereof
By embedding logic operation circuits in the memory device, the problem of communication and transmission delay between stacked static random access memory and external operation logic circuits is solved, achieving faster data processing speed and lower access latency, which is suitable for data processing of large language models.
Patent Information
- Application Number
- CN202410921061.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-10
- Publication Date
- 2026-01-13
AI Technical Summary
In large language models, the communication and transmission delay between stacked static random access memory and external arithmetic logic circuits is relatively long, which affects the overall performance.
The logic operation circuit is placed in the memory device, including the input data memory, the weight data memory and the output data memory. Data operation is performed through the logic operation circuit, reducing access latency.
By embedding logic operation circuits, the communication and transmission time between the memory device and the logic operation circuits is reduced, the data processing speed is improved, and the needs of large language models for processing large amounts of data are met.
Smart Images
Figure CN121326221A_ABST
Abstract
Description
Technical Field
[0001] This application relates to an artificial intelligence technology, and more particularly to a memory device and a method of operating the same. Background Technology
[0002] When using a Large Language Model (LLM), a stacked static random access memory (SRAM) device is typically used in conjunction with external arithmetic logic circuits to process data. However, in LLMs with large data volumes, the access latency caused by communication and transmission between the stacked SRAM and the arithmetic logic circuits increases significantly, thus affecting overall performance. Summary of the Invention
[0003] The technical problem to be solved by this application is to provide a memory device and its operation method. By setting the logic operation circuit in the memory device, the communication and transmission time between the logic operation circuit and the memory device can be avoided, thereby improving the speed of data operation and reducing access latency.
[0004] According to embodiments of this application, the memory device includes: an input data memory, a weighted data memory, an output data memory, and a logic operation circuit. The logic operation circuit is coupled to the input data memory, the weighted data memory, and the output data memory. The input data memory stores multiple input data entries. The weighted data memory stores multiple weight values corresponding to the multiple input data entries. The logic operation circuit calculates output data based on at least one input data entry and its corresponding at least one weight value, and stores the output data in the output data memory. The output data memory transmits the output data.
[0005] According to an embodiment of this application, the operation method of the memory device of this application includes: storing multiple input data through an input data memory; storing multiple weight values corresponding to the multiple input data through a weight data memory; calculating output data through a logic operation circuit based on at least one input data and its corresponding at least one weight value, and storing the output data in an output data memory; and transmitting the output data through the output data memory.
[0006] Based on the above, the memory and its operation method provided in this application can avoid communication and transmission time between the logic operation circuit and the memory device by using the logic operation circuit set in the memory device, thereby greatly reducing access latency time and meeting the processing needs of the large amount of data in Large Language Models (LLM).
[0007] The other effects and embodiments of this application are described in detail below with reference to the accompanying drawings. Attached Figure Description
[0008] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0009] Figure 1 This is a schematic diagram of a memory device according to an embodiment of this application;
[0010] Figure 2 This is a schematic diagram of the operation of a logic operation circuit according to an embodiment of this application;
[0011] Figure 3 This is a schematic diagram of a memory device according to an embodiment of this application;
[0012] Figure 4 This is a flowchart of an operation method of a memory device according to an embodiment of this application;
[0013] Figure 5 This is a flowchart of an operation method of a memory device according to an embodiment of this application. Detailed Implementation
[0014] Reference will now be made in detail to exemplary embodiments of this application, examples of which are illustrated in the accompanying drawings. Wherever possible, the same component symbols are used in the drawings and description to denote the same or similar parts.
[0015] Figure 1 This is a schematic diagram of a memory device according to an embodiment of this application; Figure 2 This is a schematic diagram illustrating the operation of a logic operation circuit according to an embodiment of this application. Please refer to... Figure 1 and Figure 2 In this embodiment, the memory device 10 may be, for example, a stacked static random access memory (SRAM). The memory device 100 includes an input data memory 110, a weight data memory 120, an output data memory 130, and a logic operation circuit 140. The logic operation circuit 140 is coupled to the input data memory 110, the weight data memory 120, and the output data memory 130. The input data memory 110 stores multiple input data entries. The weight data memory 120 stores multiple weight values corresponding to the multiple input data entries.
[0016] The logic operation circuit 140 calculates the output data Z based on the input data D1 to DN and their corresponding weight values W1 to WN, and stores the output data Z in the output data memory 130. Specifically, the logic operation circuit 140 includes a multiplier 141 and an accumulator 142. The multiplier 141 is coupled to the accumulator 142. Figure 2 As shown, multiplier 141 can calculate multiple products of input data D1 to DN and their corresponding weight values W1 to WN. For example, multiplier 141 can calculate the product Di*Wi of input data Di and weight value Wi, where i is any integer from 1 to N. Next, accumulator 142 sums the products calculated by multiplier 141 to calculate output data Z. That is, Z = D1*W1 + D1*W1 + ... + DN*WN.
[0017] The logic circuit 140 can store the output data Z in the output data memory 130. When the output data Z is needed later, the output data memory 130 can output the output data Z to the corresponding component. For example, the output data memory 130 can output the output data Z to the central processing unit (CPU) (not shown). That is, the central processing unit can access the output data memory 130 to obtain the required data (i.e., the output data Z).
[0018] In this way, the memory device 10 of this application can perform data operations through the logic operation circuit 140 within the device, thereby reducing access latency.
[0019] Figure 3 This is a schematic diagram of a memory device according to an embodiment of this application; Figure 4 This is a flowchart illustrating an operation method of a memory device according to an embodiment of this application. Please refer to... Figure 3 and Figure 4 The memory device 30 includes an input data memory 310, a weighted data memory 320, an output data memory 330, a logic operation circuit 340, a data read logic circuit 350, a control logic circuit 360, an input buffer 370, a weighted buffer 380, an output buffer 390, an address bus BA, and a data bus BD. The logic operation circuit 340 includes a multiplier 341 and an accumulator 342. The data read logic circuit 350 includes an address generator 351 and a data fetcher 352.
[0020] In step S401, the input data memory 310 stores multiple input data entries, and the weight data memory 320 stores multiple weight values corresponding to the multiple input data entries. In step S402, the address generator 351 generates at least one first address A-D1 to A-DN for at least one input data D1 to DN and at least one second address A-W1 to A-WN corresponding to at least one weight value W1 to WN for at least one input data D1 to DN, based on the address control signal CON-A.
[0021] Specifically, when multiple input data entries are written to the input data memory 310, the address generator 351 can generate multiple addresses accordingly to ensure that each input data entry written to the input data memory 310 has a unique address. Similarly, when multiple weight values corresponding to multiple input data entries are written to the weight data memory 320, the address generator 351 can generate multiple addresses accordingly to ensure that each weight value written to the weight data memory 320 has a unique address. In this way, the address generator 351 can generate at least one first address A-D1 to A-DN and at least one second address A-W1 to A-WN based on the address control signal CON-A generated by the control logic circuit 360, which indicates at least one input data D1 to DN and its corresponding at least one weight value W1 to WN.
[0022] In step S403, the address generator 351 transmits at least one first address A-D1 to A-DN and at least one second address A-W1 to A-WN to the input data memory 310 and the weight data memory 320 respectively via the address bus BA.
[0023] In step S404, the data extractor 352 extracts at least one input data D1 to DN and at least one weight value W1 to WN from the input data memory 310 and the weight data memory 320, respectively, based on the data control signal CON-D. Specifically, the data control signal CON-D generated by the control logic circuit 360 can be used to indicate the timing relationship of the extracted data. The data extractor 352 can perform the extraction of input data D1 to DN and weight values W1 to WN based on the data control signal CON-D to ensure the correctness of data reading.
[0024] Based on the above, the control logic circuit 360 can control the operation of the address generator 351 and the data extractor 352 through the address control signal CON-A and the data control signal CON-D respectively, so as to ensure the correctness and timing of the data read operation performed by the data read logic circuit 350.
[0025] In step S405, the control logic circuit 360 can determine whether the data extractor 352 has successfully extracted the data. If the data extractor 352 has successfully extracted the data, proceed to step S406. Otherwise, if the data extractor 352 has failed to extract the data, return to step S404, so that the data extractor 352 can re-extract the input data D1~DN and the weight values W1~WN.
[0026] In one embodiment, the control logic circuit 360 can determine whether the data extractor 352 has successfully extracted data based on the first addresses A-D1 to A-DN and the second addresses A-W1 to A-WN. Specifically, the control logic circuit 360 can verify whether the first addresses A-D1 to A-DN and the second addresses A-W1 to A-WN all conform to the expected range. If they do, then the first addresses A-D1 to A-DN and the second addresses A-W1 to A-WN are correct addresses, and the data extractor 352 can successfully extract the corresponding input data D1 to DN and weight values W1 to WN. If they do not conform, then one of the first addresses A-D1 to A-DN and the second addresses A-W1 to A-WN is an incorrect address, meaning that the address generated by the address generator 351 is incorrect, and the data extractor 352 cannot successfully extract the corresponding input data D1 to DN and / or weight values W1 to WN. In other words, the data extractor 352 fails to extract data.
[0027] In one embodiment, the control logic circuit 360 can determine whether the data extractor 352 has successfully extracted the data based on the input data D1-DN and the weight values W1-WN. Specifically, the control logic circuit 360 can determine whether the input data D1-DN and the weight values W1-WN are valid data. For example, the control logic circuit 360 can check the validity flag bits of the input data D1-DN and the weight values W1-WN. If the validity flag bit is a logic value of 1, it means the data is valid, and the data extractor 352 has successfully extracted the data. If the validity flag bit is a logic value of 0, it means the data is invalid, and the data extractor 352 has failed to extract the data. In another embodiment, if the validity flag bit is a logic value of 0, it means the data is valid, and the data extractor 352 has successfully extracted the data. If the validity flag bit is a logic value of 1, it means the data is invalid, and the data extractor 352 has failed to extract the data.
[0028] In one embodiment, the control logic circuit 360 can determine whether the data extractor 352 has successfully extracted the data based on the performance of the input buffer 370 and the weight buffer 380. Specifically, the input buffer 370 and the weight buffer 380 need to have sufficient storage capacity and / or sufficient transmission rate to store and / or transmit the input data D1 to DN and weight values W1 to WN extracted by the data extractor 352. If the storage capacity and / or transmission rate of the input buffer 370 and the weight buffer 380 are insufficient, it will lead to problems such as data delay or loss, and the control logic circuit 360 can determine that the data extractor 352 has failed to extract the data. If the input buffer 370 and the weight buffer 380 have sufficient storage capacity and transmission rate, the control logic circuit 360 can determine that the data extractor 352 has successfully extracted the data.
[0029] In one embodiment, the control logic circuit 360 can determine whether the data extractor 352 has successfully extracted the data based on whether noise interference or transmission errors have occurred. If noise interference or transmission errors occur during the extraction of input data D1 to DN and weight values W1 to WN by the data extractor 352, the control logic circuit 360 can determine that the data extractor 352 has failed to extract the data. If no noise interference or transmission errors occur during the extraction of input data D1 to DN and weight values W1 to WN by the data extractor 352, the control logic circuit 360 can determine that the data extractor 352 has successfully extracted the data.
[0030] In one embodiment, the control logic circuit 360 can determine whether the data extractor 352 has successfully extracted data based on the correctness of the address control signal CON-A and the data control signal CON-D. Specifically, the control logic circuit 360 can control the operation of the address generator 351 and the data extractor 352 respectively through the address control signal CON-A and the data control signal CON-D to ensure the correctness and timing of the data. The control logic circuit 360 can check the correctness of the address control signal CON-A and the data control signal CON-D it generates through simulation testing, verification testing, and other testing methods, that is, check whether the address control signal CON-A and the data control signal CON-D meet the requirements. If the address control signal CON-A and / or the data control signal CON-D are incorrect, the control logic circuit 360 can determine that the data extractor 352 has failed to extract data. If the address control signal CON-A and the data control signal CON-D are correct, the control logic circuit 360 can determine that the data extractor 352 has successfully extracted data.
[0031] In step S406, the data extractor 352 transmits at least one input data D1 to DN and at least one weight value W1 to WN to the input buffer 370 and the weight buffer 380 respectively via the data bus BD. The input buffer 370 is used to temporarily store the input data D1 to DN retrieved from the input data memory 310. The weight buffer 380 is used to temporarily store the weight values W1 to WN retrieved from the weight data memory 320.
[0032] In step S407, the logic operation circuit 340 obtains at least one input data D1 to DN and at least one weight value W1 to WN from the input buffer 370 and the weight buffer 380, respectively, to calculate the output data Z based on the at least one input data D1 to DN and the at least one weight value W1 to WN. Specifically, the register (not shown) of the input buffer 370 can load the input data D1 to DN into the register (not shown) of the multiplier 341. Similarly, the register (not shown) of the weight buffer 380 can load the weight values W1 to WN into the register of the multiplier 341. Next, the multiplier 341 can calculate multiple products of the input data D1 to DN and their corresponding weight values W1 to WN. The register of the multiplier 341 loads the multiple products into the register (not shown) of the accumulator 342, and the accumulator 342 can sum the multiple products to calculate the output data Z.
[0033] In step S408, the logic operation circuit 340 outputs the output data Z to the output buffer 390. In step S409, the output data memory 330 transfers the output data Z from the output buffer 390. Specifically, a register (not shown) of the output buffer 390 can load the output data Z into a register (not shown) of the output data memory 330. The output data memory 330 can then transfer the output data Z to the corresponding component (e.g., the central processing unit).
[0034] In this way, the memory device 30 of this application can perform data operations through the logic operation circuit 340 within the device, thereby reducing access latency. Furthermore, the memory device 30 can also control the operation of the data read logic circuit 350 through the address control signal CON-A and data control signal CON-D generated by the control logic circuit 360, ensuring data correctness and timing. In addition, the control logic circuit 360 can further determine whether the data read logic circuit 350 has successfully retrieved the data, to reconfirm data correctness. Accordingly, the memory device 30 of this application can significantly reduce the communication and transmission time between the logic operation circuit 340 and the input data memory 310, weighted data memory 320, and output data memory 330, and ensure data correctness and timing, thus meeting the processing requirements of large data volumes for Large Language Models (LLMs).
[0035] Figure 5 This is a flowchart illustrating an operation method of a memory device according to an embodiment of this application. The operation method can be... Figure 1 This is implemented using memory device 10. Please refer to... Figure 1 and Figure 5 In step S501, multiple input data entries are stored in the input data memory 110. In step S502, multiple weight values corresponding to the multiple input data entries are stored in the weight data memory 120. In step S503, the logic operation circuit 140 calculates the output data Z based on at least one input data entry D1 to DN and its corresponding weight value W1 to WN, and stores the output data Z in the output data memory 130. In step S504, the output data Z is transmitted through the output data memory 130.
[0036] Based on the above, the memory device and its operation method provided in this application can significantly reduce the communication and transmission time between the logic operation circuit and the input data memory, weight data memory and output data memory, and ensure the correctness and timing relationship of the data extracted by the data reading logic circuit, so as to meet the processing needs of the large amount of data for large language models (LLM).
[0037] The embodiments and / or implementation methods described above are merely preferred embodiments and / or implementation methods for implementing the technology of this application, and are not intended to limit the implementation methods of the technology of this application in any way. Any person skilled in the art may make some modifications or alterations to other equivalent embodiments without departing from the scope of the technical means disclosed in this application, but these should still be regarded as the technology or embodiments that are substantially the same as those of this application.
Claims
1. A memory device, characterized in that, The memory device includes: Input data storage device, used to store multiple input data entries; A weighted data storage device is used to store multiple weight values corresponding to the multiple input data; Output data memory; and The logic operation circuit is coupled to the input data memory, the weight data memory, and the output data memory, wherein... The logic operation circuit calculates output data based on at least one input data and its corresponding at least one weight value, and stores the output data in the output data memory. The output data storage device transmits the output data.
2. The memory device according to claim 1, characterized in that, The logic operation circuit includes: Multipliers; and An accumulator, coupled to the multiplier, wherein The multipliers respectively calculate at least one product of the at least one input data and its corresponding at least one weight value. The accumulator sums the at least one product to calculate the output data.
3. The memory device according to claim 1, characterized in that, Also includes: The data reading logic circuit includes: An address generator is configured to generate, based on an address control signal, at least a first address of the at least one input data and at least a second address corresponding to the at least one weight value of the at least one input data: and A data extractor is used to extract the at least one input data and the at least one weight value from the input data memory and the weight data memory, respectively, based on a data control signal.
4. The memory device according to claim 3, characterized in that, Also includes: Address bus; as well as Data bus, in which The address generator transmits the at least one first address and the at least one second address to the input data memory and the weight data memory, respectively, via the address bus. The data extractor transmits the at least one input data and the at least one weight value through the data bus.
5. The memory device according to claim 4, characterized in that, Also includes: The control logic circuit is used to determine whether the data extractor has successfully extracted the data. In response to the control logic circuit determining that the data extractor has failed to extract data, the data extractor re-extracts the at least one input data and the at least one weight value from the input data memory and the weight data memory, respectively.
6. The memory device according to claim 5, characterized in that, In response to the control logic circuit determining that the data extractor has successfully extracted data, the data extractor transmits the at least one input data and the at least one weight value to the input buffer and the weight buffer respectively via the data bus.
7. The memory device according to claim 5, characterized in that, The logic operation circuit obtains the at least one input data and the at least one weight value from the input buffer and the weight buffer, respectively, to calculate the output data based on the at least one input data and the at least one weight value.
8. The memory device according to claim 5, characterized in that, The control logic circuit determines whether the data extractor has successfully extracted data based on the at least one first address and the at least one second address.
9. The memory device according to claim 5, characterized in that, The control logic circuit determines whether the data extractor has successfully extracted data based on the at least one input data and the at least one weight value.
10. The memory device according to claim 5, characterized in that, The control logic circuit determines whether the data extractor has successfully extracted data based on the performance of the input buffer and the weight buffer.
11. The memory device according to claim 5, characterized in that, The control logic circuit determines whether the data extractor has successfully extracted data based on whether noise interference or transmission errors have occurred.
12. The memory device according to claim 5, characterized in that, The control logic circuit determines whether the data extractor has successfully extracted data based on the correctness of the address control signal and the data control signal.
13. The memory device according to claim 1, characterized in that, The logic operation circuit outputs the output data to the output buffer.
14. The memory device according to claim 1, characterized in that, The memory device is a stacked static random access memory.
15. A method of operating a memory device, characterized in that, The method of operating the memory device includes: Multiple input data entries are stored in the input data storage device; Multiple weight values corresponding to the multiple input data are stored in a weight data memory; The logic circuit calculates output data based on at least one input data and its corresponding at least one weight value, and stores the output data in an output data memory; and The output data is transmitted through the output data storage device.