High bandwidth memory system

Abstract

A high bandwidth memory system according to an embodiment of the present invention may comprise: a bank group including memory banks including matrix data and operation units respectively corresponding to the memory banks and each including registers; an operation controller electrically connected to the bank group and transmitting, to the bank group, an activation signal for delaying the execution timing of the bank group and a synchronization command for aligning execution timings within the bank group; and a memory controller electrically connected to the bank group and the operation controller and transmitting an operation command and addresses to the operation controller.

Description

High-bandwidth memory system

[0001] The present invention relates to a high-bandwidth memory system, and more particularly, to a processing-in-memory (PIM) device designed to efficiently perform computations for artificial intelligence (AI) and machine learning (ML). In particular, the present invention relates to a technique that enables a processing-in-memory device to execute commands for all memory banks while complying with power constraints.

[0002] Meanwhile, the present invention was supported by the following national research and development project.

[0003] Assignment ID: 1711195774

[0004] Assignment Number: 00228255

[0005] Ministry of Science and ICT

[0006] Project Management Agency Name: Information and Communications Technology Planning and Evaluation Institute

[0007] Research Project Name: Development of Giant Artificial Neural Network Artificial Intelligence Semiconductor SW Technology

[0008] Research Project Name: Development of SW Technology for a PIM-NPU Support System for Large-Scale Artificial Neural Network Processing

[0009] Project execution organization name: Sapion Korea Co., Ltd.

[0010] Research period: April 1, 2024 - December 31, 2024

[0011] High-bandwidth memory systems (HIGH-BANDWIDTH MEMORY SYSTEMS), widely used in high-performance electronic systems, are increasing in capacity and speed. HIGH-BANDWIDTH MEMORY SYSTEMS can be used to store various types of information, including data. For example, HIGH-BANDWIDTH MEMORY SYSTEMS can be used to store data used in various types of computational processing, such as neural network operations, or to store computational processing results. Furthermore, methods have been proposed in which at least some computational operations are performed within HIGH-BANDWIDTH MEMORY SYSTEMS to efficiently perform massive computations.

[0012] In relation to high-bandwidth memory systems, recent artificial intelligence and machine learning algorithms require extensive computation and memory access. Hardware solutions such as the Graphics Processing Unit (GPU) and Neural Processing Unit (NPU) have been proposed to efficiently perform these operations. However, their performance still suffers from significant variations depending on memory bandwidth. Therefore, Processing-In-Memory (PIM) technology is being proposed to reduce latency and energy consumption associated with data movement by placing memory and computational devices adjacent to each other.

[0013] One embodiment of the present invention provides a high-bandwidth memory system with a stable structure and improved characteristics. In particular, the high-bandwidth memory system according to one embodiment of the present invention can operate all banks in synchronization with a single command while complying with power constraints.

[0014] A high-bandwidth memory system according to one embodiment of the present invention may include a bank group including memory banks including matrix data and operation units corresponding to the memory banks, each of which includes registers; an operation controller electrically connected to the bank group and transmitting an activation signal for delaying an operation time of the bank group and a synchronization command for synchronizing an operation time of the bank group to the bank group; and a memory controller electrically connected to the bank group and the operation controller and transmitting an operation command and addresses to the operation controller.

[0015] Additionally, the operation controller can buffer the operation commands and addresses received from the memory controller, generate the activation signal including the buffered operation commands and buffered addresses, and transmit the activation signal to the bank group.

[0016] Additionally, the registers may include vector data corresponding to the matrix data.

[0017] Additionally, the operation controller can control the matrix data of the memory banks to be transmitted to the operation units, and the operation units can operate the matrix data and the vector data.

[0018] In addition, the operation controller can be controlled to sequentially transmit the matrix data of the memory banks to the operation units with a predetermined time delay by the activation signal.

[0019] Additionally, the above operation units can sequentially start operation of the matrix data and the vector data by delaying for a certain period of time by the activation signal.

[0020] Additionally, the operation units can have the operation end points of the matrix data and the vector data match each other by the synchronization command.

[0021] In addition, the operation controller can control the operation results of the matrix data and the vector data to be transmitted from the operation units to the memory banks, and the memory banks can store the operation results.

[0022] Additionally, the above operation units can sequentially transmit the operation results to the memory banks with a certain time delay by the activation signal.

[0023] Additionally, the memory banks can sequentially start storing the operation results with a certain amount of delay by the activation signal.

[0024] Additionally, the memory banks can have the same storage end time of the operation result by the synchronization command.

[0025] In addition, the operation controller may further include a mode conversion structure, and the operation controller may control to convert from an operation mode to a general mode through the mode conversion structure and then transmit the operation result to the memory banks.

[0026] In addition, after the storage of the operation results of the above memory banks is terminated, the registers can be initialized by reconverting from the general mode to the operation mode through the mode conversion structure.

[0027] According to the present technology, a high-bandwidth memory system having a stable structure and improved reliability can be provided.

[0028] FIGS. 1A to 1C are drawings for explaining a high-bandwidth memory system according to one embodiment of the present invention.

[0029] FIG. 2a and FIG. 2b are drawings for explaining a high bandwidth memory system according to one embodiment of the present invention.

[0030] FIG. 3 is a diagram for explaining the operation sequence of a high-bandwidth memory system according to one embodiment of the present invention.

[0031] FIGS. 4 to 6 are drawings for explaining a high-bandwidth memory system according to one embodiment of the present invention.

[0032] The purpose, technical configuration, and resulting operational effects of the present invention will be more clearly understood through the following detailed description based on the drawings attached to the specification of the present invention. Reference will now be made to the accompanying drawings, which will further describe embodiments of the present invention.

[0033] The embodiments disclosed herein should not be construed or used to limit the scope of the present invention. Those skilled in the art will readily appreciate that the descriptions herein, including the embodiments, have a wide range of applications. Therefore, any embodiments described in the detailed description of the present invention are intended to serve as illustrative examples to better illustrate the present invention and are not intended to limit the scope of the present invention to the embodiments.

[0034] The functional blocks depicted in the drawings and described below are merely examples of possible implementations. Other implementations may utilize other functional blocks without departing from the spirit and scope of the detailed description. Furthermore, while one or more functional blocks of the present invention are depicted as individual blocks, one or more of the functional blocks of the present invention may be a combination of various hardware and software configurations that perform the same function.

[0035] Additionally, the expression “including certain components” is an “open” expression, simply indicating the presence of those components, and should not be understood as excluding additional components.

[0036] Furthermore, when it is said that a component is “connected” or “connected” to another component, it should be understood that it may be directly connected or connected to that other component, but there may also be other components in between.

[0037] Hereinafter, various embodiments of the present invention will be described with reference to the accompanying drawings. However, this is not intended to limit the present invention to specific embodiments, and it should be understood that various modifications, equivalents, and / or alternatives of the embodiments of the present invention are included.

[0038]

[0039] FIGS. 1A to 1C are drawings for explaining a high-bandwidth memory system according to one embodiment of the present invention.

[0040] Referring to FIGS. 1A to 1C, a high bandwidth memory system (100) may include a memory controller (110) and a memory device (120).

[0041] The memory controller (110) can access the memory device (120) according to a request from the host (HOST), and the memory controller (110) can communicate with the host (HOST) using various protocols. For example, the memory controller (110) can be electrically connected to the bank group (130) and the operation controller (140) of the memory device (120), and can access the bank group (130) and the operation controller (140). The memory controller (110) may correspond to the host (HOST), or the memory controller (110) may correspond to a configuration located inside the host (HOST). The host (HOST) and the memory device (120) may constitute a data processing system, and accordingly, the high-bandwidth memory system (100) may be defined as a configuration corresponding to the data processing system or included in the data processing system.

[0042] The memory controller (110) can control the memory device (120) to read data stored in the memory device (120) or to program data in the memory device (120) in response to a read / write request from the host (HOST). The memory controller (110) can control program, read, and erase operations for the memory device (120) by providing commands (CMD), addresses (ADD), and control signals to the memory device (120). For example, the memory controller (110) can transmit commands (CMD) and addresses (ADD) to the operation controller (140). In addition, data for programming and received data can be transmitted and received between the memory controller (110) and the memory device (120).

[0043] The memory device (120) may include a bank group (130) and an operation controller (140). In this drawing, only one bank group (130) and one operation controller (140) are illustrated, but the present invention is not limited thereto. For example, the memory device (120) may include multiple bank groups (130) and multiple operation controllers (140).

[0044] A bank group (130) may include n memory banks (130A1, …, 130An). For example, the bank group (130) may include a first memory bank (130A1) to an n-th memory bank (130An). Each of the memory banks (130A1, …, 130An) may include a plurality of memory cells. Each of the memory banks (130A1, …, 130An) may include matrix data.

[0045] A bank group (130) may include n operation units (130B1, …, 130Bn) corresponding to memory banks (130A1, …, 130An), respectively. For example, the bank group (130) may include a first operation unit (130B1) corresponding to a first memory bank (130A1), and an n-th operation unit (130Bn) corresponding to an n-th memory bank (130An). The operation units (130B1, …, 130Bn) may be configured to perform operations on the bank group (130). Here, the operation units (130B1, …, 130Bn) may include registers (130C1, …, 130Cn), respectively. Registers (130C1, …, 130Cn) serve as temporary storage spaces and can store data or commands (CMD), etc. For example, registers (130C1, …, 130Cn) may each include vector data or operation commands corresponding to matrix data of memory banks (130A1, …, 130An). Here, the operation commands may include information on what operation the operation units (130B1, …, 130Bn) will perform. For example, the operation commands may be used to determine what operation the operation units (130B1, …, 130Bn) will perform according to the type and address (ADD) of the command (CMD) received from the memory controller (110).

[0046] The operation controller (140) may include a first-in, first-out structure (140A) and a mode conversion structure (140B). The first-in, first-out structure (140A) may generate an activation signal (ESG) that delays the operation timing of the bank group (130). The mode conversion structure (140B) may convert an operation mode for operation with the bank group (130) and a general mode for reading the operation result from the host (HOST). In addition, the operation controller (140) may transmit a synchronization command (SCMD) that synchronizes the operation timing of the bank group (130). When storing vector data in registers (130C1, ..., 130Cn) of operation units (130B1, ..., 130Bn) through the operation controller (140), a command (CMD) for an unused column in a row used may be ignored. The first memory bank (130A1) whose operation is completed first must wait until the operations of the remaining memory banks (130A2, …, 130An) are completed. However, the first memory bank (130A1) can receive a sufficient number of commands (CMD) that can be ignored to match the completion time of the operations of the remaining memory banks (130A2, …, 130An). In this case, the command (CMD) used can be defined as a synchronization command (SCMD).

[0047] The operation controller (140) can transmit a command (CMD) received from the memory controller (110) or an activation signal (ESG) stored in the first-in, first-out structure (140A) to the bank group (130). Here, when the operation controller (140) is in operation mode, it can control the transmission of matrix data of the memory banks (130A1, ..., 130An) to the operation units (130B1, ..., 130Bn). Alternatively, when the operation controller (140) is in general mode, it can perform reading / writing of data (operation results, etc.) from the host (HOST).

[0048] The operation controller (140) may further include a decoder. The decoder may perform a decoding operation for a command (CMD) and an address (ADD) from the memory controller (110), and may perform an internal control operation on the memory device (120) so that a memory operation is performed according to the decoding result. In addition, the operation controller (140) may perform an internal control operation on the operation units (130B1, …, 130Bn) so that the operation processing is performed according to the decoding result when the operation processing within the operation units (130B1, …, 130Bn) is controlled by the memory controller (110).

[0049] The memory device (120) may further include peripheral circuitry (not shown). The peripheral circuitry may provide a plurality of signals for accessing data, such as writing or reading data to or from memory cells included in a plurality of bank groups (130) or each of the plurality of bank groups (130). The peripheral circuitry may provide a plurality of signals to the plurality of bank groups (130) by including a plurality of logics. For example, the peripheral circuitry may include arithmetic logic and control logic.

[0050]

[0051] Hereinafter, a method for constructing a high-bandwidth memory system (100) according to one embodiment of the present invention will be specifically described.

[0052] First, the memory controller (110) can receive a command (CMD) and addresses (ADD) from the host (HOST) and transmit them to the memory device (120). For example, the memory controller (110) can transmit the command (CMD) and addresses (ADD) to the operation controller (140).

[0053] Next, the operation controller (140) can generate an activation signal (ESG). For example, the FIFO structure (140A) can generate an activation signal (ESG) that delays the operation timing of the bank group (130). The FIFO structure (140A) can buffer the command (CMD) and addresses (ADD) received from the memory controller (110). For example, the FIFO structure (140A) can buffer the command (CMD) and addresses (ADD) received from the memory controller (110) to generate a buffered command (BCMD) and a buffered address (BADD). Next, the FIFO structure (140A) can generate an activation signal (ESG) including the buffered command (BCMD) and the buffered address (BADD).

[0054] The first-in, first-out structure (140A) can transmit an activation signal (ESG) to the bank group (130). In this case, the command (CMD) transmitted by the memory controller (110) can be delayed by a certain period of time by the activation signal (ESG) and transmitted sequentially to the memory banks (130A1, …, 130An). Subsequently, the memory banks (130A1, …, 130An) can transmit matrix data to the operation units (130B1, …, 130Bn).

[0055] Next, the operation units (130B1, …, 130Bn) can operate on matrix data and vector data stored in the registers (130C1, …, 130Cn). In other words, the activation signal (ESG) can sequentially operate the memory banks (130A1, …, 130An) and / or the operation units (130B1, …, 130Bn) within the bank group (130) by delaying the operation time of the memory banks (130A1, …, 130An) and / or the operation units (130B1, …, 130Bn) within the bank group (130). That is, the first-in, first-out structure (140A) can delay the command (CMD) received from the memory controller (110) by a predetermined time and transmit it to the memory banks (130A1, …, 130An) and / or the operation units (130B1, …, 130Bn) within the bank group (130). Therefore, according to the present invention, the high bandwidth memory system (100) does not have to operate the memory banks (130A1, …, 130An) at once, but operates each memory bank (130A1, …, 130An) individually, so that it can operate while complying with power constraints.

[0056] The operation controller (140) can transmit a synchronization command (SCMD) that matches the operation timing of the bank group (130). A command (CMD) for an unused column in a row used when storing vector data in the registers (130C1, …, 130Cn) of the operation units (130B1, …, 130Bn) can be ignored. The command (CMD) used here can be a synchronization command (SCMD). The synchronization command (SCMD) can be added before the command (CMD) for storing vector data in the registers (130C1, …, 130Cn) of the operation units (130B1, …, 130Bn), and in this case, the timing of storing new input vector data in the registers (130C1, …, 130Cn) can be delayed.

[0057] On the other hand, the command (CMD) for the column used in the row used when storing vector data in the registers (130C1, …, 130Cn) of the operation units (130B1, …, 130Bn) cannot be ignored. In other words, before storing vector data in the registers (130C1, …, 130Cn), it is necessary to synchronize the operation timing of all the operation units (130B1, …, 130Bn).

[0058] The synchronization command (SCMD) can align the operation end times for the memory banks (130A1, …, 130An). In addition, the operation end times of the operation units (130B1, …, 130Bn) can be aligned by the synchronization command (SCMD). For example, by adding the synchronization command (SCMD) at a time after the operation for the first memory bank (130A1) is completed, the operation end times for the n-th memory bank (130An) can be aligned. In this case, the operation end times for all the memory banks (130A1, …, 130An) can be aligned, and new input data can be received and processed thereafter. Therefore, according to the present invention, a collision occurring when a plurality of commands (CMD) are transmitted to the operation units (130B1, …, 130Bn) can be prevented through the synchronization command (SMCD).

[0059] The memory controller (110) of the host (HOST) can transmit a write command (CMD) to the operation controller (140) to store the operation results performed by the operation units (130B1, …, 130Bn) in the memory banks (130A1, …, 130An). In this case, the operation controller (140) can sequentially transmit the command (CMD) to the memory banks (130A1, …, 130An) and the operation units (130B1, …, 130Bn) with a predetermined delay by the activation signal (ESG). At this time, the memory banks (130A1, …, 130An) can sequentially receive and store the operation results from the operation units (130B1, …, 130Bn) according to the command (CMD) received with a predetermined delay by the activation signal (ESG). Here, the memory banks (130A1, …, 130An) can be synchronized by a synchronization command (SCMD) until the time at which all memory banks (130A1, …, 130An) store their corresponding operation results coincides.

[0060] Next, after the storage of the operation results of the memory banks (130A1, …, 130An) is completed, the registers (130C1, …, 130Cn) can be initialized. For example, the registers (130C1, …, 130Cn) that store vector data can be initialized by converting from the operation mode to the general mode through the mode conversion structure (140B) and then converting from the general mode to the operation mode again.

[0061]

[0062] Figures 2a and 2b are diagrams illustrating a high-bandwidth memory system according to one embodiment of the present invention. Hereinafter, any content that overlaps with the previously described content will be omitted for brevity.

[0063] Referring to FIGS. 2A and 2B, a high-bandwidth memory system (100) may include a memory controller (110) and a memory device (120). The memory device (120) may include a bank group (130) and an operation controller (140). The bank group (130) may include a memory bank (130A), an operation unit (130B), and a register (130C). The operation controller (140) may include a first-in, first-out structure (140A) and a mode conversion structure (140B).

[0064] The memory controller (110) can transmit commands (Cmd) and addresses to the operation controller (140). Here, the addresses can include a row address (RA) or a column address (CA).

[0065] For reference, the operation controller (140) may further include a bank decoder. The bank decoder may perform a decoding operation on commands (Cmd) and addresses received from the memory controller (110). The bank address (BA) generated through this may be transmitted to the first-in, first-out structure (140A) together with the command (Cmd), row address (RA), and column address (CA).

[0066] Next, the operation controller (140) can buffer the commands (Cmd) and addresses received from the memory controller (110) through the first-in, first-out structure (140A). For example, the first-in, first-out structure (140A) can generate buffered commands and addresses (Buffered C / A). Here, when the command (Cmd) is enabled (ACT), the 13th bit of the row address (RA) required to generate the buffer enable signal (ENABLE_BUF) can be stored in the row address register (Row Address Register), and even if another command (Cmd) is received thereafter, the buffer enable signal (ENABLE_BUF) can be generated. The buffered commands and addresses (Buffered C / A) can be transmitted to the bank group (130) with a predetermined delay together with the buffer enable signal (ENABLE_BUF). Here, buffered commands and addresses (Buffered C / A) can be defined as enable signals. However, this is not limited to this, and the enable signal can include a buffer enable signal (ENABLE_BUF).

[0067] An activation signal can be generated in a tree form. Hereinafter, referring to FIG. 2A, the bank group (130) is illustrated as including one memory bank (130A), but it will be described assuming that there are multiple memory banks. For example, assuming that there are a total of eight memory banks, seven activation signals can be generated, similar to FIG. 2B. Here, the activation signals can be generated in a tree form. The activation signals generated in the tree form can be transmitted to each of the seven memory banks so that each memory bank is operated with a predetermined time delay. Here, the memory bank that starts operation first does not need to be delayed, so an activation signal is not required. In other words, the operation controller (140) can generate an activation signal that delays the operation time of the bank group (130) by a predetermined time through the first-in, first-out structure (140A). In this case, since the high-bandwidth memory system (100) does not have to operate multiple memory banks at the same time, it can operate while complying with power constraints.

[0068] Next, the bank group (130) can transmit matrix data included in the memory bank (130A) to the operation unit (130B). Here, when there are multiple memory banks (130A), the memory banks can sequentially transmit matrix data to the operation units with a certain time delay by an activation signal.

[0069] Next, if there are operation units corresponding to the number of memory banks in the bank group (130), the operation units can start operations by utilizing matrix data received from the memory banks one after another with a certain amount of delay by an activation signal.

[0070] The operation controller (140) can transmit a synchronization command that synchronizes the operation timing of the bank group (130). According to the present invention, the high-bandwidth memory system (100) can access the operation unit (130B) without accessing the memory bank (130A) in some cases. In other words, it can access the operation units corresponding to the memory banks without accessing the plurality of memory banks. The synchronization command can be added as many times as necessary after the operation of the memory bank that starts the operation first is completed, thereby synchronizing the time point with the operation of the memory bank that starts the operation last is completed. Therefore, according to the present invention, a conflict that occurs when a plurality of commands are transmitted to the operation units can be prevented through the synchronization command.

[0071] Next, the operation controller (140) can transmit an activation signal to the memory bank (130A) and the operation unit (130B) to store the operation result calculated by the operation unit (130B) in the memory bank (130A). Here, the memory banks can sequentially receive and store the operation results from the operation units according to the received command with a certain time delay by the activation signal. Here, the time at which the operation results of the memory banks are stored can be matched by a synchronization command.

[0072] Next, after the storage of the operation result in the memory bank (130A) is completed, the register (130C) can be initialized. For example, the register (130C) for storing vector data can be initialized by converting the mode of the operation controller (140) from the operation mode to the general mode through the mode conversion structure (140B) and then converting the mode from the general mode to the operation mode again.

[0073] For reference, the mode conversion structure (140B) may include a controller that converts the mode of even-numbered banks (Mode Controller for even bank), a controller that converts the mode of odd-numbered banks (Mode Controller for odd bank), and a mode output structure (Mode Generator) that outputs an operation mode or a normal mode.

[0074]

[0075] Figure 3 is a diagram illustrating the operational sequence of a high-bandwidth memory system according to one embodiment of the present invention. Any details that overlap with those previously described will be omitted for brevity.

[0076] Referring to FIG. 3, preprocessing (S310) can be performed. Preprocessing can be performed similarly to conventional high-bandwidth memory systems. For example, in the preprocessing step, the mode conversion structure of the operation controller can be used to change from normal mode to operation mode. Furthermore, in the preprocessing step, commands transmitted from the host can be stored in the registers of the operation unit.

[0077] Next, vector data can be stored in the registers of the operation units (S320). For example, vector data corresponding to matrix data of memory banks can be stored in the operation units. Here, before storing the vector data in the registers, a synchronization command can be added to synchronize the operation timings of all operation units. Therefore, the timing of storing the vector data in the registers of the operation units can be consistent.

[0078] Next, the vector data of the operation unit and the matrix data of the memory bank can be calculated (S330). First, the operation controller can transmit the command received from the host to the memory banks and the operation units. The operation controller can control to transmit the matrix data of the memory banks to the operation units. Here, the operation controller can control to sequentially transmit the matrix data of the memory banks to the operation units with a predetermined delay by an activation signal generated in a first-in, first-out structure. Next, the operation units can receive the matrix data from the memory banks and calculate the matrix data and the vector data. At this time, the operation units can sequentially start the calculation with a predetermined delay by the activation signal.

[0079] Next, it can be checked (S340) whether operations have been performed on all input tiles. The storage space of the registers of the operation units is limited, and not all input vectors can be processed at once. In this case, all input vectors can be divided into input tiles and processed. It is checked whether operations have been performed on all input tiles, and if all input tiles have been processed, all input vectors have been processed for one output tile, so subsequent processes can be performed. It is checked whether operations have been performed on all input tiles, and if not all input tiles have been processed, other input tiles can be continuously processed.

[0080] Next, the operation result can be stored in the memory bank (S350). First, the operation controller can transmit the command received from the host to the memory banks and operation units. The operation controller can control the operation result to be transmitted from the operation units to the memory banks. Here, the operation controller can control the operation result to be transmitted to the memory banks in sequence with a predetermined time delay by an activation signal generated in a first-in, first-out structure. Then, the memory banks can receive the operation result. At this time, the memory banks can store the operation result in sequence with a predetermined time delay by the activation signal.

[0081] Next, post-processing (S360) can be performed. The post-processing can include a step of checking whether there is an output tile to be processed. If there is an output tile to be processed, the mode conversion structure can be used to convert from the operation mode to the general mode, and then from the general mode to the operation mode to initialize the registers of the operation units, and then another output tile can be processed. If there is no output tile to be processed, the operation mode can be converted to the general mode so that the host can read the operation result.

[0082]

[0083] Figures 4 to 6 are diagrams illustrating a high-bandwidth memory system according to one embodiment of the present invention. Hereinafter, any content that overlaps with the previously described content will be omitted for brevity.

[0084] For reference, FIG. 4 is a diagram illustrating an example of the operation of a high-bandwidth memory system of the prior art. FIGS. 5 and 6 are diagrams illustrating an example of the operation of a high-bandwidth memory system according to an embodiment of the present invention.

[0085] Referring to Figure 4, conventional high-bandwidth memory systems require all memory banks to be activated simultaneously to execute commands simultaneously. This can result in high peak power consumption without separate power optimization, making it impossible for the high-bandwidth memory system to operate within its power constraints.

[0086] Referring to FIGS. 5 and 6, when a write command is performed for a specific row in which the 13th bit of the row address is 1, there is no need to access all memory banks, and only the registers of the operation units need to be accessed, so it can be performed simultaneously in all operation units.

[0087] On the other hand, when performing a write command for a general row where the 13th bit is not 1, it is necessary to access all memory banks using the local data bus, which limits the ability to physically activate all memory banks simultaneously. Therefore, in order to perform operations on all memory banks, an activation signal including a buffered command and address can be sequentially transmitted to all memory banks with a predetermined delay through the first-in, first-out structure of the operation controller according to an embodiment of the present invention.

[0088] When memory banks are operated with a certain time delay due to an activation signal, the end times of the memory bank where the operation ends the latest and the memory bank where the operation ends the fastest need to be synchronized. This is to receive new input data into the registers of the computational units. When storing vector data in the registers of the computational units, if a read or write command is sent to an unused column in a row where the 13th bit of the row address is 1, no operation may occur. Commands sent to such specific columns can be defined as synchronization commands.

[0089] Referring to FIG. 5, when a synchronization command (SYNC) is added between a command for operation (All Bank Operation) and a command for receiving new input data (DATA SETUP), a conflict between the command for operation and the command for receiving new input data can be prevented, and memory banks and operation units can be synchronized without modifying the memory controller. That is, according to one embodiment of the present invention, a required number of synchronization commands can be added in advance from the host between the command for operation and the command for receiving new input data, taking into account timing variables related to power constraints.

[0090] Referring to FIG. 6, after the operation results are transmitted from the registers of the operation units to the memory banks and stored in the memory banks, a mode conversion step (MODE) can be performed through the mode conversion structure of the operation controller. Here, the high-bandwidth memory system according to one embodiment of the present invention can be converted from the operation mode to the general mode, and then converted from the general mode to the operation mode again. Through this, the registers of the operation units can be initialized before processing the next output tile.

[0091] Note that all commands may require a step of closing the activated row (Precharge, P) and activating the newly accessed row (Activation, ACT) if they access a new row rather than an activated row in a specific memory bank.

[0092]

[0093] The various embodiments and terms used in this document are not intended to limit the technical features described in this document to specific embodiments, but should be understood to encompass various modifications, equivalents, or alternatives of the embodiments. In connection with the description of the drawings, similar reference numerals may be used to refer to similar or related components. The singular form of a noun corresponding to an item may include one or more items, unless the context clearly indicates otherwise.

[0094] In this document, the phrases "A or B," "at least one of A and B," "at least one of A or B," "A, B, or C," "at least one of A, B, and C," and "at least one of A, B, or C" can each include all possible combinations of the items listed together in that phrase. Terms such as "1," "2," or "first" or "second" may be used merely to distinguish the corresponding component from other corresponding components and do not limit the corresponding components in any other respect (e.g., importance or order). When a (e.g., a first) component is referred to as "coupled" or "connected" to another (e.g., a second) component, with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0095] The term "module" as used herein may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0096] Various embodiments of the present document may be implemented as software (e.g., a program) including one or more instructions stored in a storage medium (e.g., a memory) readable by a device (e.g., an electronic device). The storage medium may include random access memory (RAM), a memory buffer, a hard drive, a database, erasable programmable read-only memory (EPROM), electrically erasable read-only memory (EEPROM), read-only memory (ROM), and / or the like.

[0097] Additionally, the processor of the embodiments of the present document can call at least one command among one or more commands stored from a storage medium and execute it. This enables the device to operate to perform at least one function according to the at least one command called. These one or more commands may include code generated by a compiler or code executable by an interpreter. The processor may be a general-purpose processor, a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), and / or the like.

[0098] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is stored semi-permanently or temporarily on the storage medium.

[0099] The methods according to various embodiments disclosed in this document may be provided as a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated on a machine-readable storage medium, such as a manufacturer's server, an application store's server, or a server's memory.

[0100] According to various embodiments, each component (e.g., a module or a program) of the described components may include one or more entities. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. A bank group including memory banks containing matrix data and operation units corresponding to the memory banks and each including registers; An operation controller electrically connected to the bank group and transmitting an activation signal for delaying the operation time of the bank group and a synchronization command for matching the operation time of the bank group to the bank group; and A memory controller electrically connected to the bank group and the operation controller, and transmitting operation commands and addresses to the operation controller. A high-bandwidth memory system including:

2. In paragraph 1, The above operation controller buffers the operation commands and addresses received from the memory controller, generates the activation signal including the buffered operation commands and buffered addresses, and transmits the activation signal to the bank group. High bandwidth memory system.

3. In paragraph 1, The above registers contain vector data corresponding to the matrix data. High bandwidth memory system.

4. In paragraph 3, The above operation controller controls the matrix data of the memory banks to be transmitted to the operation units, The above operation units operate on the matrix data and the vector data. High bandwidth memory system.

5. In paragraph 4, The above operation controller controls the transmission of the matrix data of the memory banks to the operation units in sequence with a certain time delay by the activation signal. High bandwidth memory system.

6. In paragraph 4, The above operation units sequentially start the operation of the matrix data and the vector data with a certain time delay by the activation signal. High bandwidth memory system.

7. In paragraph 4, The above operation units are synchronized by the synchronization command so that the operation end time of the matrix data and the vector data matches. High bandwidth memory system.

8. In paragraph 3, The above operation controller controls the operation results of the matrix data and the vector data to be transmitted from the operation units to the memory banks, The above memory banks store the results of the above operations. High bandwidth memory system.

9. In paragraph 8, The above operation units are delayed for a certain period of time by the activation signal and sequentially transmit the operation results to the memory banks. High bandwidth memory system.

10. In paragraph 8, The above memory banks start storing the operation results in sequence with a certain delay by the activation signal. High bandwidth memory system.

11. In paragraph 8, The above memory banks have the same end point of storing the operation result by the above synchronization command. High bandwidth memory system.

12. In paragraph 11, The above operation controller further includes a mode conversion structure, The above operation controller controls to convert from operation mode to general mode through the mode conversion structure and then transmit the operation results to the memory banks. High bandwidth memory system.

13. In paragraph 12, After the storage of the operation results of the above memory banks is completed, the registers are initialized by reconverting from the general mode to the operation mode through the mode conversion structure. High bandwidth memory system.

Images