High-bandwidth memory and systems having such high-bandwidth memory

Abstract

A high-bandwidth memory and a system having the high-bandwidth memory are disclosed. The high-bandwidth memory includes a buffer die and a plurality of memory dies, each memory die including at least one first processing element memory bank group and at least one second processing element memory bank group. The at least one first processing element memory bank group includes one or more first memory banks connected to one or more first memory bank input / output line groups and a first processing element controller connected to the one or more first memory bank input / output line groups and a first global input / output line group, and is configured to perform a first processing operation on first data output from one of the one or more first memory bank input / output line groups and second data transmitted through the first global input / output line group based on a first instruction, the first instruction being generated based on a first processing command.

Description

[0001] This application claims priority to Korean Patent Application No. 10-2020-0007177, filed on January 20, 2020, with the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0002] The methods and apparatus consistent with the example embodiments relate to a high-bandwidth memory and a system having the high-bandwidth memory. Background Technology

[0003] High-bandwidth memory (HBM) may include logic dies and multiple memory dies stacked on top of the logic dies. A system with high-bandwidth memory may include high-bandwidth memory and a control unit (e.g., a central processing unit (CPU) or a graphics processing unit (GPU)). Generally, high-bandwidth memory can store data under the control of a control unit, and the control unit can perform processing operations on the data. Summary of the Invention

[0004] One or more example embodiments provide a high-bandwidth memory and a system having the high-bandwidth memory, which is capable of performing different processing operations internally simultaneously using data stored therein.

[0005] The example embodiments are not limited to the purposes mentioned above, and other unmentioned purposes will be clearly understood by those skilled in the art based on the following description.

[0006] A high-bandwidth memory according to an example embodiment includes: a buffer die; and a plurality of memory dies stacked on the buffer die. Each of the plurality of memory dies includes at least one first processing element memory bank group and at least one second processing element memory bank group. The at least one first processing element memory bank group includes: one or more first memory banks connected to one or more first memory bank input / output line groups; and a first processing element controller connected to the one or more first memory bank input / output line groups and a first global input / output line group, wherein the first processing element controller is configured to perform a first processing operation on first data output from one of the first memory bank input / output line groups and second data transmitted through the first global input / output line group based on a first instruction, the first instruction being generated based on a first processing command. The at least one second processing element memory bank group includes: one or more second memory banks connected to one or more second memory bank input / output line groups; and a second processing element controller connected to the one or more second memory bank input / output line groups and a second global input / output line group, wherein the second processing element controller is configured to perform a second processing operation based on a second instruction to obtain second data from one of the one or more second memory banks and transfer the second data to the second global input / output line group, the second instruction being different from a first instruction generated based on a first processing command. The first global input / output line group and the second global input / output line group are jointly connected to a data bus.

[0007] A high-bandwidth memory according to an example embodiment includes: a buffer die; and a plurality of memory dies stacked on the buffer die. Each of the plurality of memory dies includes at least one first processing element memory bank group and at least one second processing element memory bank group. The at least one first processing element memory bank group includes: one or more first memory banks connected to one or more first memory bank input / output line groups connected to a first global input / output line group; and a first processing element controller connected to the first global input / output line group and a data bus, wherein the first processing element controller is configured to receive first data output through one of the first memory bank input / output line groups based on a first instruction, receive second data through the data bus, and perform a first processing operation on the first data and the second data, wherein the first instruction is received based on a first processing command. The at least one second processing element memory bank group includes: one or more second memory banks connected to one or more second memory bank input / output line groups, the one or more second memory bank input / output line groups being connected to a second global input / output line group; and a second processing element controller connected to the second global input / output line group and a data bus, wherein the second processing element controller is configured to perform a second processing operation based on a second instruction different from a first instruction, obtaining second data from one of the one or more second memory bank input / output line groups and transmitting the second data to the data bus via the second global input / output line group, the second instruction being received based on a first processing command.

[0008] A system according to an example embodiment includes: a high-bandwidth memory including a buffer die and a plurality of memory dies stacked on the buffer die; and a controller configured to transmit commands and addresses to the buffer dies and receive output data from the buffer dies. Each of the plurality of memory dies includes at least one first processing element memory bank group and at least one second processing element memory bank group. The at least one first processing element memory bank group includes: one or more first memory banks connected to one or more first memory bank input / output line groups; and a first processing element controller connected to the one or more first memory bank input / output line groups and a first global input / output line group, wherein the first processing element controller is configured to perform a first processing operation on first data output from one of the first memory bank input / output line groups and second data transmitted through the first global input / output line group based on a first instruction, the first instruction being generated based on a first processing command. The at least one second processing element memory bank group includes: one or more second memory banks connected to one or more second memory bank input / output line groups; and a second processing element controller connected to the one or more second memory bank input / output line groups and a second global input / output line group, wherein the second processing element controller is configured to perform a second processing operation based on a second instruction to retrieve second data from one of the one or more second memory banks and transfer the second data to the second global input / output line group, the second instruction being different from a first instruction generated based on a first processing command. The first global input / output line group and the second global input / output line group are jointly connected to a data bus. Attached Figure Description

[0009] The above and other objects and features will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings, in which:

[0010] Figure 1 This is a view showing the structure of a system with HBM (more specifically, a system manufactured with 2.5D packaging) according to an example embodiment.

[0011] Figure 2 This is a view showing the structure of a system with HBM (more specifically, a system manufactured using 3D packaging) according to an example embodiment.

[0012] Figure 3 This is a view showing the configuration of multiple memory dies according to an example embodiment.

[0013] Figure 4 This is a view showing the configuration of the memory bank group of the pseudo memory channel according to an example embodiment.

[0014] Figure 5 This is a view illustrating the transfer of commands, addresses, and data between the control unit and the buffer die according to an example embodiment.

[0015] Figure 6 This is a block diagram illustrating the configuration of the input / output unit, command and address generator, and data bus unit according to an example embodiment.

[0016] Figure 7 This is a detailed block diagram of the processing element storage group according to an example embodiment.

[0017] Figure 8 This is a block diagram illustrating the configuration of the processing element units according to an example embodiment.

[0018] Figure 9 This is a view illustrating the structure of an instruction storage device according to an example embodiment.

[0019] Figure 10A and Figure 10B The format of instructions stored in the instruction storage area of ​​the instruction storage device according to an example embodiment is shown.

[0020] Figure 11 Configuration information stored in the configuration information storage area of ​​the instruction storage device according to an example embodiment is shown.

[0021] Figure 12 This is a block diagram illustrating the configuration of a processing element processor according to an example embodiment.

[0022] Figure 13 This is a view showing the structure of the register according to an example embodiment.

[0023] Figure 14A and Figure 14B A truth table of processing commands for HBM according to an example embodiment is shown.

[0024] Figure 15 This is a table showing the memory bank and / or processing element controller specified by the memory bank address applied together with the processing command, according to an example embodiment.

[0025] Figure 16 This is a timing diagram illustrating the processing setup operation of HBM according to an example embodiment.

[0026] Figure 17 , Figure 18 , Figure 19A , Figure 19B and Figure 20 This is a view illustrating the processing operations of HBM according to an example embodiment.

[0027] Figure 21This is a view showing the configuration of the memory bank group of the pseudo memory channel according to an example embodiment.

[0028] Figure 22 This is a block diagram illustrating the configuration of the processing element storage group according to an example embodiment. Detailed Implementation

[0029] The above and other aspects and features will become more apparent from the detailed description of exemplary embodiments with reference to the accompanying drawings. It will be understood that when an element or layer is referred to as "above," "over," "on," "connected to," or "bonded to" another element or layer, the element or layer may be directly above, over, or on the other element or layer, or may be directly connected to or directly bonded to the other element or layer, or there may be intermediate elements or intermediate layers. Conversely, when an element is referred to as "directly above," "over," "on," "directly connected to," or "directly bonded to" another element or layer, there are no intermediate elements or intermediate layers. The same reference numerals always denote the same element.

[0030] Figure 1 This is a view showing the structure of a system with HBM (more specifically, a system manufactured using 2.5D packaging) according to an example embodiment.

[0031] Reference Figure 1 System 1000 may include HBM 100, control unit 200, interposer 300, and printed circuit board (PCB) 400. HBM 100 may include memory dies MD1 to MD4 and buffer die BD.

[0032] HBM 100 may include memory dies MD1 to MD4 and buffer dies BD. The memory dies MD1 to MD4 and the buffer dies BD may be stacked, and the stacked memory dies MD1 to MD4 may be placed on the buffer die BD. First bumps MB may be formed between the dies MD1 to MD4 and the BD, and through-silicon vias (TSVs) through the memory dies MD1 to MD4 may be formed between the first bumps MB. A first direct access (DA) bump dab, a first power bump pb1, and a first command, address, and data bump cadb1 may be disposed on the lower surface of the buffer die BD.

[0033] The second command, address, and data bump cadb2, the second power bump pb2, and the first control signal and data bump cdb can be arranged on the lower surface of the control unit 200. The control unit 200 can be a graphics processing unit (GPU) die, a central processing unit (CPU) die, or a system-on-a-chip (SoC) die.

[0034] The first bump MB, the first DA bump dab, the first power bump pb1, the second power bump pb2, the first command, address and data bump cadb1, the second command, address and data bump cadb2, and the first control signal and data bump cdb can be micro bumps.

[0035] The second DA bump DAFB, the third power bump PBFB, and the second control signal and data bump CDFB can be disposed on the lower surface of the interposer layer 300. The interposer layer 300 may include a DA line dal connecting the first DA bump dab to the second DA bump DAFB, a command, address, and data line cal connecting the first command, address, and data bump cdb1 to the second command, address, and data bump cdb2, and a control signal and data line cdl connecting the first control signal and data bump cdb to the second control signal and data bump CDFB. The interposer layer 300 may also include a power line connecting the first power bump pb1 to the third power bump PBFB and a power line connecting the second power bump pb2 to the third power bump PBFB. The second DA bump DAFB, the third power bump PBFB, and the second control signal and data bump CDFB can be flip-chip bumps.

[0036] The DA bump (DAB), power bump (PB), and control signal and data bump (CDB) can be located on the lower surface of the PCB 400. In the PCB 400, the second DA bump (DAFB) can be connected to the DA bump (DAB), the third power bump (PBFB) can be connected to the power bump (PB), and the second control signal and data bump (CDFB) can be connected to the control signal and data bump (CDB).

[0037] Figure 2 This is a view showing the structure of a system with HBM (more specifically, a system manufactured using 3D packaging) according to an example embodiment.

[0038] Reference Figure 2 ,and Figure 1 Unlike other systems, system 1100 may not include intermediary layer 300. Additionally, the buffer die BD of HBM 100 can be directly disposed on the upper surface of control unit 200. That is, the first DA bump dab, the first power bump pb1, and the first command, address, and data bump cadb1 can be disposed on the upper surface of control unit 200.

[0039] The first DA bump dab and the second DA bump DAFB can be connected in the control unit 200. The first power bump pb1 and the third power bump PBFB can be connected in the control unit 200.

[0040] The control unit 200 can receive control signals and data applied through the second control signal and data bump CDFB, and send channel commands, addresses and channel data to the first command, address and data bump cadb1.

[0041] Figure 1 and Figure 2 The control unit 200 shown can process data in response to control signals applied by a second control signal and a data bump CDFB, generate processed data as channel data, and send the channel data with a channel command and address to the buffer die BD via a first command, address, and data bump cadb1.

[0042] Figure 3 This is a view illustrating the configuration of multiple memory dies according to an example embodiment. Memory dies MD1 through MD4 may each include two memory channels MCH1 and MCH2, MCH3 and MCH4, MCH5 and MCH6, or MCH7 and MCH8. Each of the memory channels MCH1 through MCH8 may include two pseudo-memory channels PCH1 and PCH2, an input / output unit INOUTU, a command and address generator CADDG, and a data bus unit DBUSU.

[0043] Reference Figure 3 Each of the pseudo-memory channels PCH1 and PCH2 may include four memory banks BG0 to BG3, which can transmit data via global input / output lines BGIO0, BGIO1, BGIO2, and BGIO3, respectively. The global input / output lines BGIO0 to BGIO3 of pseudo-memory channel PCH1 can be connected to the data bus DBUS1 for data transmission, and the global input / output lines BGIO0 to BGIO3 of pseudo-memory channel PCH2 can be connected to the data bus DBUS2 for data transmission.

[0044] Figure 4 This is a view illustrating the configuration of the memory bank groups of the pseudo memory channel according to an example embodiment. Memory bank groups BG0 to BG3 can each be configured with two processing element memory bank groups PEBG0 and PEBG1, PEBG2 and PEBG3, PEBG4 and PEBG5, or PEBG6 and PEBG7.

[0045] Reference Figure 4Processing element storage group PEBG0 may include two storage units BA0 and BA1 and a processing element controller PE0. Processing element storage group PEBG1 may include two storage units BA2 and BA3 and a processing element controller PE1. Processing element storage group PEBG2 may include two storage units BA4 and BA5 and a processing element controller PE2. Processing element storage group PEBG3 may include two storage units BA6 and BA7 and a processing element controller PE3. Similarly, processing element storage group PEBG4 may include two storage units BA8 and BA9 and a processing element controller PE4. Processing element storage group PEBG5 may include two storage units BA10 and BA11 and a processing element controller PE5. Processing element storage group PEBG6 may include two storage units BA12 and BA13 and a processing element controller PE6. Processing element storage group PEBG7 may include two storage units BA14 and BA15 and a processing element controller PE7. Each of the processing element controllers PE0 to PE7 may be arranged between two storage units.

[0046] Figure 4 An example embodiment is shown where each processing element memory bank group includes two memory banks. However, the example embodiment is not limited thereto. For example, according to the example embodiment, each processing element memory bank group may include one memory bank.

[0047] Figure 5 This is a view illustrating the transfer of commands, addresses, and data between the control unit and the buffer die according to an example embodiment. The control unit 200 may include a memory controller 210. The buffer die BD may include an input driver and an output driver.

[0048] Reference Figure 5 The memory controller 210 can send commands and addresses 1CA for memory channel MCH1, data 1PCH1DQ for pseudo memory channel PCH1 of memory channel MCH1, and data 1PCH2DQ for pseudo memory channel PCH2 of memory channel MCH1 to the buffer die BD. The buffer die BD can buffer commands and addresses 1CA, data 1PCH1DQ, and data 1PCH2DQ, and send commands and addresses 1CA, data 1PCH1DQ, and data 1PCH2DQ to memory channel MCH1.

[0049] Similarly, the memory controller 210 can send commands and addresses 2CA to 8CA for the corresponding memory channels MCH2 to MCH8, data 2PCH1DQ to 8PCH1DQ for the pseudo memory channel PCH1 for the corresponding memory channels MCH2 to MCH8, and data 2PCH2DQ to 8PCH2DQ for the pseudo memory channel PCH2 for the corresponding memory channels MCH2 to MCH8 to the buffer die BD. The buffer die BD can buffer commands and addresses 2CA to 8CA, data 2PCH1DQ to 8PCH1DQ, and data 2PCH2DQ to 8PCH2DQ, generate commands and addresses 2ca to 8ca, data 2pch1dq to 8pch1dq, and data 2pch2dq to 8pch2dq, and send the commands and addresses 2ca to 8ca, data 2pch1dq to 8pch1dq, and data 2pch2dq to 8pch2dq to the memory channels MCH2 to MCH8.

[0050] Figure 6 This is a block diagram illustrating the configuration of the input / output unit, command and address generator, and data bus unit (more specifically, including the input / output unit INOUTU, the command and address generator CADDG, and the data bus unit DBUSU in the pseudo memory channel PCH1) according to an example embodiment.

[0051] Reference Figure 6 The input / output unit INOUTU may include a command and address driver CADRV and a data driver DQDRV. The command and address generator CADDG may include a command and address decoder CADEC. The data bus unit DBUSU may include a data bus driver DBUSDRV. The command and address driver CADRV and the command and address decoder CADEC may be a block shared by the pseudo memory channel PC2.

[0052] The following text will provide an example of... Figure 6 The description of the function of each block shown in the figure.

[0053] The command and address driver CADRV can drive command and address Ica to generate command and address signals ca.

[0054] The data driver DQDRV can drive data 1pch1dq to generate data dq, and drive data dq to generate data 1pch1dq.

[0055] The command and address decoder CADEC can decode the command and address signal ca to generate the command signal com, the memory address ba, the row address rowa, and the column address cola.

[0056] The data bus driver DBUSDRV can drive the data transferred between the global input / output lines BGIO0 to BGIO3 and the data bus DBUS1.

[0057] Figure 7 It is a processing element storage group according to an example embodiment (more specifically, Figure 4 The diagram shows a detailed block diagram of the processing element storage group (PEBG0).

[0058] Reference Figure 7 Each of the memory banks BA0 and BA1 may include a memory cell array MCA, a memory bank controller BANKC, a row decoder ROWD, a column decoder COLD, an input / output sense amplifier IOSA, a write driver WDRV, and a global input / output gating unit BGIOG. The processing element controller PE0 may include a processing element unit PEU and processing element input / output gating units PEIOG0 and PEIOG1.

[0059] The following text will provide an example of... Figure 7 A description of the function of each box shown in the figure.

[0060] During normal operation, the bank controller BANKC can receive command signal com, bank address ba, row address rowa, and column address cola, and generate row address signal RADD, column address signal CADD, write control signal wc, and read control signal rc in response to command signal com and bank address ba. During processing, in response to bank control signal bacon0 or bacon1, the bank controller BANKC can be disabled to remain inactive, or enabled to operate and generate row address signal RADD, column address signal CADD, write control signal wc, and read control signal rc.

[0061] The row decoder ROWD can decode the row address signal RADD to generate multiple word line selection signals wl, and activate at least one of the multiple word line selection signals wl.

[0062] The column decoder COLD can decode the column address signal CADD to generate multiple column select signals csl, and activate at least one of the multiple column select signals csl.

[0063] A memory cell array (MCA) can include multiple memory cells and can store data in a selected memory cell among the multiple memory cells or output data stored in a selected memory cell in response to multiple word line select signals wl and multiple column select signals csl.

[0064] The input / output sense amplifier IOSA can amplify and output data from the memory cell array MCA in response to the read control signal rc.

[0065] The write driver WDRV can drive data in response to the write control signal wc and output the data to the memory cell array MCA.

[0066] The global input / output gating unit BGIOG can transfer data from the input / output sense amplifier IOSA to the memory input / output line group baio0, and transfer data from the memory input / output line group baio0 to the write driver WDRV.

[0067] Each of the processing element input / output gating units PEIOG0 and PEIOG1 can transfer data between the memory input / output line group baio0 or baio1 and the global input / output line group BGIO0 during normal operation, and control the data transfer between the memory input / output line group baio0 or baio1 and the global input / output line group BGIO0 in response to the processing element input / output gating signal pegc0 or pegc1 during processing operation.

[0068] The processing element unit (PEU) can receive a command signal com, a memory address ba, a row address rowa, and a column address cola, and execute a processing operation based on a first instruction when the command signal com is a processing command. For example, the PEU of the processing element memory group PEBG0 can generate a processing element input / output gating signal pegc0 to disconnect the processing element input / output gating unit PEIOG0, thereby performing a first processing operation on first data transmitted through memory input / output line group baio0 or baio1 and second data transmitted through global input / output line group BGIO0. Simultaneously, the PEU of at least one of the processing element memory groups PEBG1 to PEBG7 (e.g., PEBG7) can generate a processing element input / output gating signal pegc0 based on a second instruction different from the first instruction to turn on the processing element input / output gating unit PEIOG0, thereby transmitting second data output from the memory (e.g., BA14) to global input / output line group BGIO0 via global input / output line group BGIO3 and data bus DBUS1.

[0069] Figure 8 This is a block diagram illustrating the configuration of a processing element unit according to an example embodiment. The processing element unit (PEU) may include a processing element controller 20, an instruction storage device 22, and a processing element processor 24.

[0070] The following text will provide an example of... Figure 8A description of the function of each box shown in the figure.

[0071] The processing element controller 20 can receive command signals com, memory address ba, row address rowa, and column address cola. During processing setup operations, it generates instruction input control signals insic and instruction output control signals insoc to receive and decode instruction inst to generate processing element control signals pecon. For example, the processing element control signal pecon may include input selection signals inc, processing element execution control signals pec, register control signals rerc, output selection signals outc, and processing element input / output gate signals pegc0 and pegc1.

[0072] The instruction storage device 22 can store data transmitted via the global input / output line group BGIO0 as an instruction in response to the instruction input control signal insic, and output the stored instruction inst to the processing element controller 20 in response to the instruction output control signal insoc.

[0073] The processing element processor 24 can perform processing operations on at least two of the data transmitted via memory input / output lines baio0 and baio1, data transmitted via global input / output line BGIO0, and register output data REGO in response to the processing element control signal pecon.

[0074] Figure 9 This is a view illustrating the structure of an instruction storage device according to an example embodiment. The instruction storage device 22 may include k storage areas INSTQ0 to INSTQk for storing a plurality of instructions and a configuration information storage area CONFG.

[0075] Instruction storage device 22 can be selected by a specific row address (e.g., rowa, where all bits are "0"), and k storage areas INSTQ0 to INSTQk and configuration information storage area CONFG can be selected by column address cola.

[0076] Figure 10A and Figure 10B The format of instructions stored in the instruction storage area of ​​the instruction storage device according to an example embodiment is shown.

[0077] Reference Figure 10AThe instruction format may include index information (INDEX), operator information (OPE), destination location information (DST), first operand location information (SRC1), second operand location information (SRC2), third operand location information (SRC3), and mask information (MASK). Operator information (OPE) may specify data for the multiplication-accumulation operator (MAC), multiplication operator (MUL), addition / subtraction operators (ADD / SUB), shift operator (MOV), load operator (LOAD), store operator (STORE), jump operator (JUMP), or no-operation (NOP) operator (NOP). Destination location information (DST) may specify data for register REG, global input / output line group BGIO, or write driver WDRV. First operand location information (SRC1) may specify data for input / output sense amplifier IOSA, register REG, or global input / output line group BGIO. Second operand location information (SRC2) may specify data for register REG or global input / output line group BGIO. Third operand location information (SRC3) may specify data for register REG. Instructions other than those including the multiplication-accumulation operator MAC may omit the third operand position information SRC3. Instructions other than those including the multiplication-accumulation operator MAC, the multiplication operator MUL, or the addition / subtraction operator ADD / SUB may omit the second operand position information SRC2.

[0078] Reference Figure 10B The mask information (MASK) can specify the operation for all processing element controllers PE0 to PE7 when it is "0000", and specify the operation for each processing element controller PE0 to PE7 when it is "1000" to "1111". That is, the mask information (MASK) can specify the operation for processing element controller PE0 when it is "1000", and specify the operation for processing element controller PE7 when it is "1111".

[0079] Reference Figure 8 , Figure 10A and Figure 10B During the processing operation, the processing element controller 20 can generate an input selection signal inc, a register control signal rec, or an output selection signal outc based on the first operand position information SRC1, the second operand position information SRC2, and the third operand position information SRC3; generate a processing element execution control signal pec based on the operator information OPE; and generate a register control signal rec and an output selection signal outc based on the destination position information DST.

[0080] Figure 11The illustration shows configuration information stored in the configuration information storage area of ​​the instruction storage device according to an example embodiment. The configuration information may include processing element enable information PEEN, reset information RESET, and program counter information PCNT.

[0081] Reference Figure 11 The processing element enable information PEEN can be information that enables the processing operation of the processing element controllers PE0 to PE7; the reset information RESET can be information that resets the processing element controllers PE1 to PE7; and the program counter information PCNT can be information that represents the index of the instruction currently being executed.

[0082] Figure 12 This is a block diagram illustrating the configuration of a processing element processor according to an example embodiment. The processing element processor 24 may include an input selector INMUX, a calculator ALU, a register REG, and an output selector OUTMUX.

[0083] The following text will provide an example of... Figure 12 A description of the function of each box shown in the figure.

[0084] The input selector INMUX can select at least two of the following in response to the input selection signal inc: data transmitted through the memory input / output lines baio0 and baio1, data transmitted through the global input / output line BGIO0, and register output data REGO, and generate at least two of the following: first operand S1, second operand S2, and third operand S3.

[0085] The calculator ALU can perform processing operations on at least two of the first operand S1, the second operand S2, and the third operand S3 in response to the processing element execution control signal pec.

[0086] The REG register can store the processing operation result data result in response to the register control signal regc, and generate the stored processing operation result data result as the register output data REGO.

[0087] The output selector OUTMUX can output the register output data REGO to at least one of the memory input / output lines baio0 and baio1, the global input / output line group BGIO0, and the input selector INMUX in response to the output select signal outc.

[0088] Figure 13 This is a view illustrating the structure of a register according to an example embodiment. The register REG may include x storage areas REG0 to REGx.

[0089] Reference Figure 12 and Figure 13 In response to the register control signal regc, the register REG can store the result data of the processing operation, result, in a selected memory region among the x memory regions REG0 to REGx, or generate register output data REGO from the selected memory region.

[0090] Figure 14A and Figure 14B A truth table of processing commands for HBM according to an example embodiment is shown, wherein the command and address ca can be the command and row address or the command and column address.

[0091] Reference Figure 14A When the command and address ca are the command and row address, and the clock enable signal applied through the clock enable signal terminal CKE is at a "high (H)" level for two clock cycles of the clock signal applied through the clock signal terminal CK (at the first rising edge, first falling edge, second rising edge, and second falling edge of the clock signal), the valid command PEACT for the processing element can be generated by decoding the different signals of the two R0 and R1 applied to the row address terminals R0 to Ri at the first rising edge of the clock signal. The bank address and row address can be applied through other terminals at the first rising edge of the clock signal, and through terminals R0 to Ri at the first falling edge, second rising edge, and second falling edge of the clock signal.

[0092] Reference Figure 14B When the command and address ca are the command and column address, and the clock enable signal can be at a "high (H)" level for one clock cycle (at the rising and falling edges of the clock signal), the processing element write command PEWR, the processing element read command PERD, and the processing element read / write command PERW can be generated by decoding four different signals C0 to C3 applied to the j column address terminals C0 to Cj. The bank address and column address can be applied through other terminals at the rising edge of the clock signal and through terminals C0 to Cj at the falling edge of the clock signal.

[0093] Figure 15 This is a table showing the memory bank and / or processing element controller specified by the memory bank address applied together with the processing command, according to an example embodiment.

[0094] Reference Figure 15When the command and address ca are processing commands, the 4-bit memory address applied along with the processing command can be used to select memory banks BA0 to BA15 and processing element controllers PE0 to PE7. For example, processing element controllers PE0 to PE7 can be specified to operate at memory addresses “0000”, “0010”, “0100”, “0110”, “1000”, “1010”, “1100”, and “1110”, respectively. Even-numbered memory banks BA0, BA2, BA4, BA6, BA8, BA10, BA12, and BA14, along with processing element controllers PE0 to PE7, can be specified to operate at memory address “0001”. Odd-numbered memory banks BA1, BA3, BA5, BA7, BA9, BA11, BA13, and BA15, along with processing element controllers PE0 to PE7, can be specified to operate at memory address “0011”. Memory banks BA0 to BA15 and processing element controllers PE0 to PE7 can be specified to operate at memory address “1111”.

[0095] Reference Figure 7 , Figure 8 , Figure 10A , Figure 14A , Figure 14B and Figure 15 When the memory address ba is "0001", and the command signal com is the processing element read command PERD or the processing element read / write command PERW, when the first operand location information SRC1 specifies the input / output sense amplifier IOSA based on the operator information OPE of the instruction stored in the instruction storage device 22, the processing element controller 20 can generate memory control signals bacon0 and bacon1, and processing element input / output gating signals pegc0 and pegc1. The memory controller BANKC of memory BA0 can be enabled in response to the memory control signal bacon0, and the processing element input / output gating unit PEIOG0 of processing element controller PE0 can be deactivated in response to the processing element input / output gating signal pegc0. As a result, data output through the memory input / output line group baio0 can be input to the processing element processor 24 instead of being transmitted to the global input / output line group BGIO0. The memory controller BANKC of memory BA1 can be disabled in response to the memory control signal bacon1.

[0096] Reference Figure 7 , Figure 8 , Figure 10A , Figure 14A , Figure 14B and Figure 15When the memory address ba is "0001", and the command signal com is the processing element write command PEWR or the processing element read / write command PERW, when the second operand location information SRC2 specifies the global input / output line group BGIO based on the operator information OPE of the instruction stored in the instruction storage device 22, the processing element controller 20 can generate memory control signals bacon0 and bacon1, and processing element input / output gating signals pegc0 and pegc1. The memory controller BANKC of memory bank BA0 can be enabled in response to the memory control signal bacon0, and the processing element input / output gating unit PEIOG0 of processing element controller PE0 can be deactivated in response to the processing element input / output gating signal pegc0. As a result, data transmitted to the global input / output line group BGIO0 can be input to the processing element processor 24 instead of being transmitted to the memory input / output line group BAIO0. The memory controller BANKC of memory bank BA1 can be disabled in response to the memory control signal bacon1.

[0097] For example, when the command signal com is the processing element read command PERD, a processing element read operation can be performed. In this operation, data is input from the memory input / output line group baio0 or baio1 of at least one processing element memory bank group (e.g., PEBG0) to the processing element processor 24 of that group. When the command signal com is the processing element write command PEWR, a processing element write operation can be performed. In this operation, data is transferred from the memory input / output line group baio0 or baio1 of at least one processing element memory bank group (e.g., PEBG7) to the global input / output line group BGIO3 and input to the processing element processor 24 of that group. When the command signal com is the processing element read / write command PERW, both the processing element read and write operations can be performed simultaneously.

[0098] Figure 16 This is a timing diagram illustrating the processing setup operation of HBM according to an example embodiment.

[0099] Reference Figure 16 When the command signal com in the command and line address ca1 is a valid command PEACT in the processing element, and the line address rowa is the specified... Figure 9 The instruction storage device 22 shown is a specific row address (i.e., a row address where all bits are "0"), and the memory address ba is Figure 15When the specified processing element controller PE0 in the table is “0000”, the processing element controller PE0 can perform the processing setting operation.

[0100] Next, when the command signal com in the command and column address ca2 is the processing element that writes the command PEWR, and the column address cola is specified... Figure 9 When the column address of the memory region INSTQ0 shown is "0" (all its bits are "0"), Figure 8 The processing element controller 20 shown can generate the instruction input control signal insic, and Figure 8 The instruction storage device 22 shown can store the data dq1 applied through the global input / output line group BGIO0 as an instruction in response to the instruction input control signal insic. Figure 9 The storage area INSTQ0 is shown in the figure.

[0101] Next, when the command signal com in the command and column address ca3 is the processing element that writes the command PEWR, and the column address cola is specified... Figure 9 When the column address "31" of the configuration information storage area CONFG shown in the figure is "1" (all bits except the least significant bit), Figure 8 The instruction storage device 22 shown can store the data dq2 applied through the global input / output line group BGIO0 as configuration information in response to the instruction input control signal insic. Figure 9 The configuration information is stored in the CONFG area shown in the diagram.

[0102] Although the processing element controller PE0 has been described as performing the above processing setup operation, other processing element controllers PE1 to PE7 can store the same or different instructions by repeating the above processing setup operation when the memory address ba is changed.

[0103] Figures 17 to 20 This is a view illustrating the processing operations of HBM according to an example embodiment.

[0104] Figure 17 The diagram shows how to obtain a 7×1 matrix C by performing a multiplication operation between a 7×16 matrix A and a 16×1 matrix B.

[0105] Reference Figure 17 and Figure 18For the multiplication operation of a 7×16 matrix A and a 16×1 matrix B, in response to the memory address ba ("0000"), the row address rowa ("1"), and the column address cola ("0"), the eight data points D11 to D18 of the first row of the 7×16 matrix A can be stored in the memory cell array MCA of memory bank BA0. In response to the memory address ba ("0001"), the same row address rowa, and the same column address cola, the remaining eight data points D19 to D116 of the first row can be stored in the memory cell array MCA of memory bank BA1. In this way, the sixteen data points D21 to D216 of the second row of the 7×16 matrix A can be stored separately in memory banks BA2 and BA3, and the sixteen data points of each row in the third to seventh rows of the 7×16 matrix A can be stored separately in the corresponding memory banks BA4 and BA5, ..., BA12 and BA13. Similarly, in response to the memory address ba of “1110”, the same row address rowa and the same column address cola, the eight data points V11 to V81 of the 16×1 matrix B can be stored in the memory cell array MCA of memory bank BA14, and in response to the memory address ba of “1111”, the same row address rowa and the same column address cola, the remaining eight data points V91 to V161 of the 16×1 matrix B can be stored in the memory cell array MCA of memory bank BA15.

[0106] Figure 19A and Figure 19B This shows how to execute Figure 16 The processing settings shown are for the storage Figure 18 Data execution in memory banks BA0 to BA15 shown in the figure Figure 17 The table shows the multiplication operation and the instructions stored in the instruction storage device 22 of the processing element controller PE7 and the instruction storage devices 22 of the processing element controllers PE0 to PE6.

[0107] Reference Figure 19A and Figure 19BThe move operator MOV indicates a move operation that moves data from the first operand position IOSA or REG0 to the destination position BGIO. The multiply-accumulate operator MAC indicates a multiply-accumulate operation that adds the result of multiplying the data from the first operand position IOSA and the data from the second operand position BGIO to the data from the third operand position REG0 and outputs the result to the destination position REG0. The jump operator JUMP indicates a jump operation that jumps to the destination position INDEX0 (the value of index information 0 in the instruction) to repeat the operation CNT(1) a number of times as described in the first operand position information. Additionally, the NOP operator NOP indicates no operation. (See also...) Figure 12 The mask information MASK can specify the operation of the processing element controllers PE0 to PE7.

[0108] Figure 20 This indicates that execution is performed in response to the command and address ca. Figure 19A and Figure 19B The timing diagram of the instructions shown is shown in the figure.

[0109] Reference Figure 8 , Figure 12 , Figure 15 and Figures 17 to 20 When the command signal com in command and address ca4 is the processing element valid command PEACT, the row address rowa is "1" and the memory bank address ba is "0001", the memory cell corresponding to the row address rowa of the memory cell array MCA with even-numbered memory banks BA0, BA2, ..., BA14 can be selected. Additionally, the processing element controllers PE0 to PE7 can be selected.

[0110] Next, when the command signal com in command and address ca5 is the processing element read / write command PERW, the column address cola is "0", and the memory bank address ba is "0001", the memory cells corresponding to the row address rowa ("1") and column address cola ("0") of the memory cell array MCA of even-numbered memory banks BA0, BA2, ..., BA14 can be selected. The processing element controller PE7 can execute the move operation MOV corresponding to index 0 in the instruction storage device 22 in response to the processing element read / write command PERW to transfer the data V11 stored in memory bank BA14 to the global input / output line group BGIO3. Each of the processing element controllers PE0 to PE6 can perform a multiplication-accumulation operation (MAC) based on the instruction corresponding to index 0 in instruction memory 22. The result obtained by multiplying each of the data D11 to D18 to D71 to D78 stored in the even-numbered memory banks BA0, BA2, ..., BA12 with the data V11 stored in the even-numbered memory bank BA14 is added to the data V11 stored in the storage area REG0 of register REG, thereby generating the processing operation result data result and storing the processing operation result data result in the storage area REG0 of register REG. In this way, until the command and address ca12 are applied, when the processing element read / write command PERW is applied as a command signal, the memory address ba of "0001" is applied, and the column addresses cola of "1" to "7" are applied, the processing operation result data result can be generated by accumulating the data obtained by multiplying each of the data D11 to D18 to D71 to D78 stored in even-numbered memory banks BA0, BA2, ..., BA12 with each of the data V21 to V81 stored in even-numbered memory bank BA14, and the data stored in the storage area REG0 of register REG. The processing operation result data result is then stored in the storage area REG0 of register REG. That is, the processing operation result data of ((D11 × V11) + 11 × (D18 × V81)) can be stored in the storage area REG0 of register REG of the processing element controller PE0. For Figure 17 The eight data points in each row of matrix A from the second to the seventh row and Figure 17 The processing results of the eight data in the first to eighth rows of matrix B can be stored in the storage area REG0 of the corresponding register REG in the processing element controllers PE1 to PE6.

[0111] Next, when the command signal com in the command and address ca13 is the processing element read command PERD, the column address cola is "X (not important)" and the memory address ba is "0011" (or "0001" or "1111"), the processing element controllers PE0 to PE7 can operate. Each of the processing element controllers PE0 to PE7 can jump to the destination location INDEX0 (the value 0 of the instruction's index information) based on the jump operator JUMP to repeat the operation a number of times CNT(1) as described in the first operand location information.

[0112] Next, until commands and addresses ca14 through ca22 are applied, except that the storage address ba is changed to "0011", the same command signal com, the same row address rowa, and the same column address cola as when commands and addresses ca4 through ca12 are applied can be applied.

[0113] As a result, in response to the row address "rowa" and column addresses "0" to "7" of the memory cell array MCA of the odd-numbered memory banks BA1, BA3, ..., BA15, the processing operation result data result is generated by accumulating the result data obtained by multiplying each of the data D19 to D116 to D79 to D716 stored in the odd-numbered memory banks BA1, BA3, ..., BA13 with each of the data V91 to V161 stored in the odd-numbered memory bank BA15, and the data stored in the storage area REG0 of the register REG. The processing operation result data result is then stored in the storage area REG0 of the register REG.

[0114] Therefore, the processing result data 011 of ((D11×V11)+……+(D116×V161)) can be stored in the storage area REG0 of the REG register of the processing element controller PE0. The processing result data 021, 031, ..., 071 can be stored in the storage area REG0 of the REG registers of the processing element controllers PE1 to PE6, respectively. That is, the processing operation can be completed.

[0115] Next, when the command signal com in command and address ca23 is the processing element read command PERD, the memory address ba is "0000", and the column address cola is "X", the processing element controller PE0 can be selected in response to the memory address ba. The processing element controller PE0 can perform a move operation MOV to output the processing operation result data 011 stored in the memory area REG0 of register REG via the global input / output line group BGIO0 and the data bus DBUS1. Processing element controllers PE1 through PE6 can sequentially output the processing operation result data 021 through 071 in response to the sequential selection of memory address ba.

[0116] Figure 21 This is a view illustrating the configuration of the memory bank groups of the pseudo memory channel according to an example embodiment. Memory bank groups BG0 to BG3 can be processing element memory bank groups PEBG0 to PEBG4, respectively. Each of the memory bank groups BG0 to BG3 may include four memory banks.

[0117] Reference Figure 21 Processing element storage group PEBG0 may include four storage units BA0 to BA3 and a processing element controller PE0; processing element storage group PEBG1 may include four storage units BA4 to BA7 and a processing element controller PE1; processing element storage group PEBG2 may include four storage units BA8 to BA11 and a processing element controller PE2; and processing element storage group PEBG3 may include four storage units BA12 to BA15 and a processing element controller PE3.

[0118] Figure 22 This illustrates a processing element storage group according to an example embodiment (more specifically, Figure 21 The diagram shows the configuration of the processing element memory group PEBG0 among the processing element memory groups PEBG0 to PEBG4.

[0119] Reference Figure 22 Each of the memory banks BA0 to BA3 in the processing element memory bank group PEBG0 can be configured with Figure 7The memory bank BA0 shown is the same. Memory bank input / output lines baio0 to baio3 can be connected to the global input / output line group BGIO0. The processing element input / output gating unit PEIOG can control the connection between the global input / output line group BGIO0 and the data bus DBUS1 in response to the processing element input / output gating signal pegc0. The processing element controller PE0, except that it is connected to the global input / output line group BGIO0 and the data bus DBUS1, can perform the same operations as described above. That is, the processing element controller PE0 can perform processing operations on data transmitted through the global input / output line group BGIO0 and data transmitted through the data bus DBUS1.

[0120] Based on the above Figures 1 to 20 The description will be easy to understand. Figure 21 and Figure 22 The configuration is shown in detail below.

[0121] As is apparent from the above description, the high-bandwidth memory according to the example embodiment can use the data stored therein to perform different processing operations simultaneously internally. Furthermore, a system with high-bandwidth memory can perform processing operations without data transfer between the high-bandwidth memory and the control unit. Therefore, it is possible to improve processing speed and reduce power consumption.

[0122] According to an exemplary embodiment, the control unit, buffer die, memory die, controller, processor, multiplexer, calculator, register, selector, driver, decoder, processing element, gating unit, amplifier, and other components include... Figures 1 to 8 , Figure 12 , Figure 21 and Figure 22At least one of the other elements represented by the boxes shown can be implemented as various numbers of hardware, software, and / or firmware structures to perform the corresponding functions described above. For example, at least one of the control unit, buffer die, memory die, controller, processor, multiplexer, calculator, register, selector, driver, decoder, processing element, gating unit, amplifier, and other elements can use DC circuit structures, such as memories, processors, logic circuits, lookup tables, etc., that can perform corresponding functions under the control of one or more microprocessors or other control devices. Furthermore, at least one of the control unit, buffer die, memory die, controller, processor, multiplexer, calculator, register, selector, driver, decoder, processing element, gating unit, amplifier, and other elements can be specifically implemented by a module, program, or code containing one or more executable instructions for performing a specific logical function, and can be executed by one or more microprocessors or other control devices. Furthermore, at least one of the control unit, buffer die, memory die, controller, processor, multiplexer, calculator, register, selector, driver, decoder, processing element, gating unit, amplifier, and other elements may also include or be implemented by a processor (such as a central processing unit (CPU), microprocessor, etc., that performs the corresponding function). Two or more of the control unit, buffer die, memory die, controller, processor, multiplexer, calculator, register, selector, driver, decoder, processing element, gating unit, amplifier, and other elements may be combined to form a single component, element, module, or unit that performs all the operations or functions of the two or more combinations of the control unit, buffer die, memory die, controller, processor, multiplexer, calculator, register, selector, driver, decoder, processing element, gating unit, amplifier, and other elements. Furthermore, at least a portion of the functionality of at least one of the following components—control unit, buffer die, memory die, controller, processor, multiplexer, calculator, register, selector, driver, decoder, processing element, gating unit, amplifier, and others—can be performed by another of these components. Therefore, although a bus is not shown in each of the above block diagrams, communication between components can be performed via a bus. The functional aspects of the above example embodiments can be implemented as algorithms executed on one or more processors. Furthermore, the control unit, buffer die, memory die, controller, processor, multiplexer, calculator, register, selector, driver, decoder, processing element, gating unit, amplifier, or other components, represented by blocks or processing steps, can employ any number of related techniques for electronic configuration, signal processing and / or control, data processing, etc.

[0123] While the disclosure has been specifically shown and described with reference to exemplary embodiments, it will be apparent to those skilled in the art that various changes in form and detail may be made without departing from the spirit and essential characteristics of the disclosure. Therefore, the exemplary embodiments described above are to be construed in all respects as illustrative rather than restrictive.

Claims

1. A high bandwidth memory, the high bandwidth memory comprising: a buffer die; and a plurality of memory dies stacked on the buffer die, wherein each of the plurality of memory dies comprises at least one first processing element memory bank group and at least one second processing element memory bank group, wherein the at least one first processing element memory bank group comprises one or more first memory banks connected to one or more first memory bank input / output line groups and a first processing element controller connected to the one or more first memory bank input / output line groups and a first global input / output line group, wherein the first processing element controller is configured to perform a first processing operation on first data output from one of the one or more first memory bank input / output line groups based on a first instruction and second data transmitted through the first global input / output line group, the first instruction being generated based on a first processing command, wherein the at least one second processing element memory bank group comprises one or more second memory banks connected to one or more second memory bank input / output line groups and a second processing element controller connected to the one or more second memory bank input / output line groups and a second global input / output line group, wherein the second processing element controller is configured to perform a second processing operation of obtaining second data from one of the one or more second memory banks and transmitting the second data to the second global input / output line group based on a second instruction, the second instruction being different from the first instruction generated based on the first processing command, wherein the first global input / output line group and the second global input / output line group are collectively connected to a data bus. each of the plurality of memory dies comprises at least two memory channels, 2. The high-bandwidth memory of claim 1, wherein, wherein each of the at least two memory channels comprises at least two pseudo memory channels, wherein each of the at least two pseudo memory channels comprises one first processing element memory bank group from among the at least one first processing element memory bank group and one second processing element memory bank group from among the at least one second processing element memory bank group, wherein each of the at least two pseudo memory channels comprises a command and address generator configured to receive a command and an address from the buffer die and to generate a command signal, a bank address, a row address, and a column address. each of the one or more first memory banks and the one or more second memory banks comprises:

3. The high-bandwidth memory of claim 2, wherein, a row decoder configured to generate a plurality of word line selection signals based on a row address; a column decoder configured to generate a plurality of column selection signals based on a column address; a memory cell array comprising a plurality of memory cells, the plurality of memory cells being partially selected based on the plurality of word line selection signals and the plurality of column selection signals; an input / output sense amplifier configured to amplify and output data output from the memory cell array; a write driver configured to drive input data and output the driven input data to the memory cell array; and a read driver configured to drive the data output from the memory cell array and output the driven data. a global input / output gating unit configured to control connections between the input / output sense amplifiers and write drivers and corresponding ones of the one or more first memory bank input / output line groups and the one or more second memory bank input / output line groups.

4. The high-bandwidth memory of any one of claims 1-3, wherein, The first processing element controller includes: a first processing element unit configured to generate a first processing element input / output gating signal and perform a first processing operation based on a first memory bank address applied with a first processing command; and one or more first processing element input / output gating units configured to isolate connections between the one or more first memory bank input / output line groups and a first global input / output line group based on the first processing element input / output gating signal, wherein the second processing element controller includes: a second processing element unit configured to generate a second processing element input / output gating signal and perform the second processing operation based on a second memory bank address applied with the first processing command; and one or more second processing element input / output gating units configured to connect connections between the one or more second memory bank input / output line groups and a second global input / output line group based on the second processing element input / output gating signal.

5. The high-bandwidth memory of claim 4, wherein, when the memory bank address applied with the first processing command specifies one of the one or more first memory banks, one of the one or more second memory banks, the first processing element controller, and the second processing element controller, the one of the one or more first memory banks and the one of the one or more second memory banks generate first data and second data simultaneously based on a common row address and a common column address.

6. The high-bandwidth memory of claim 4, wherein, Each of the first processing element unit and the second processing element unit includes an instruction storage configured to store a plurality of instructions including a first instruction or a second instruction, wherein each of the first processing element unit and the second processing element unit is configured to generate a first processing element control signal based on the first instruction or generate a second processing element control signal based on the second instruction based on the first processing command, wherein each of the first processing element unit and the second processing element unit includes a processing element processor configured to perform a first processing operation based on the first processing element control signal or perform a second processing operation based on the second processing element control signal.

7. The high-bandwidth memory of claim 6, wherein, the instruction storage is configured to store the first instruction or the second instruction based on a second processing command in a processing setup operation and output the first instruction or the second instruction based on a first processing command in a processing operation, wherein each of the first instruction and the second instruction includes index information, operator information, destination location information, at least one operand location information, and mask information, wherein the mask information specifies an operation of at least one of the at least one first processing element memory bank group or the at least one second processing element memory bank group.

8. The high-bandwidth memory of claim 7, wherein, Each of the first processing element unit and the second processing element unit is further configured to generate an input selection signal based on the first operand location information and the second operand location information, generate a processing element execution control signal based on the operator information, and generate a register control signal and an output selection signal based on the destination location information.

9. The high-bandwidth memory of claim 8, wherein, The processing element processor includes: an input selector configured to select the first data, the second data, and / or the register output data based on the input selection signal; a calculator configured to perform the first processing operation based on the processing element execution control signal and output processing operation result data; a register configured to store the processing operation result data based on the register control signal; and an output selector configured to transmit the processing operation result data to one of the one or more first bank input / output line groups, the first global input / output line group, and the input selector based on the output selection signal.

10. The high-bandwidth memory of claim 9, wherein, The operator information in the first instruction includes a multiply-accumulate operator, a multiplication operator, or an addition / subtraction operator, wherein, based on the operator information in the first instruction being the multiplication operator or the addition / subtraction operator, the input selector selects the first data and the second data, and the calculator performs a multiplication operation or an addition / subtraction operation on the first data and the second data as the first processing operation, wherein the first instruction further includes third operand information, wherein, based on the operator information in the first instruction being the multiply-accumulate operator, the input selector selects the first data, the second data, and the register output data, and the calculator performs a multiply-accumulate operation of adding the register output data to result data obtained by multiplying the first data and the second data as the first processing operation, wherein the operator information in the second instruction is a move operator.

11. A high-bandwidth memory, the high-bandwidth memory comprising: a buffer die; and a plurality of memory dies stacked on the buffer die, wherein each of the plurality of memory dies includes at least one first processing element bank group and at least one second processing element bank group, wherein the at least one first processing element bank group includes one or more first banks connected to one or more first bank input / output line groups connected to a first global input / output line group, and a first processing element controller connected to the first global input / output line group and a data bus, wherein the first processing element controller is configured to receive first data output through one of the one or more first bank input / output line groups based on a first instruction, receive second data through the data bus, and perform a first processing operation on the first data and the second data based on a first processing command, the first instruction being received based on the first processing command, The at least one second processing element bank includes one or more second banks connected to one or more second bank input / output line groups connected to a second global input / output line group, and a second processing element controller connected to the second global input / output line group and the data bus, wherein the second processing element controller is configured to perform a second processing operation of obtaining second data from one of the one or more second bank input / output line groups and transmitting the second data to the data bus through the second global input / output line group based on a second instruction different from the first instruction, the second instruction being received based on the first processing command.

12. The high-bandwidth memory of claim 11, wherein, Each of the plurality of memory dies includes at least two memory channels, wherein each of the at least two memory channels includes at least two pseudo memory channels, wherein each of the at least two pseudo memory channels includes one of the at least one first processing element bank and one of the at least one second processing element bank, wherein each of the at least two pseudo memory channels includes a command and address generator configured to receive a command and an address applied from the buffer die and to generate a command signal, a bank address, a row address, and a column address.

13. The high-bandwidth memory of claim 12, wherein, Each of the one or more first banks and the one or more second banks includes: a row decoder configured to generate a plurality of word line selection signals based on the row address; a column decoder configured to generate a plurality of column selection signals based on the column address; a memory cell array including a plurality of memory cells partially selected based on the plurality of word line selection signals and the plurality of column selection signals; an input / output sense amplifier configured to amplify and output data output from the memory cell array; a write driver configured to drive input data and output the driven data to the memory cell array; and a global input / output gating unit configured to control connection between the input / output sense amplifier and the write driver and a corresponding bank input / output line group of the one or more first bank input / output line groups and the one or more second bank input / output line groups.

14. The high-bandwidth memory of any one of claims 11-13, wherein, The first processing element controller includes: a first processing element unit configured to generate a first processing element input / output gating signal based on a first bank address applied with the first processing command and to perform a first processing operation; and one or more first processing element input / output gating units configured to cut off connection between the first global input / output line group and the data bus in response to the first processing element input / output gating signal, wherein the second processing element controller includes: a second processing element unit configured to generate a second processing element input / output gate signal and perform a second processing operation based on a second memory bank address applied with the first processing command; and one or more second processing element input / output gate units configured to establish a connection between a second global input / output line group and the data bus in response to the second processing element input / output gate signal.

15. The high-bandwidth memory of claim 14, wherein, when a memory bank address applied with the first processing command specifies one of the one or more first memory banks, one of the one or more second memory banks, the first processing element controller, and the second processing element controller, the one of the one or more first memory banks and the one of the one or more second memory banks generate first data and second data based on a common row address and based on a common column address simultaneously.

16. The high-bandwidth memory of claim 14, wherein, each of the first processing element unit and the second processing element unit includes an instruction storage configured to store a plurality of instructions including a first instruction or a second instruction, wherein each of the first processing element unit and the second processing element unit is configured to generate a first processing element control signal based on the first instruction or generate a second processing element control signal based on the second instruction based on the first processing command, wherein each of the first processing element unit and the second processing element unit includes a processing element processor configured to perform a first processing operation based on the first processing element control signal or perform a second processing operation based on the second processing element control signal.

17. The high-bandwidth memory of claim 16, wherein, the instruction storage is configured to store the first instruction or the second instruction based on a second processing command in a processing setup operation and output the first instruction or the second instruction based on a first processing command in a processing operation, wherein each of the first instruction and the second instruction includes index information, operator information, destination location information, at least one operand location information, and mask information, wherein the mask information specifies an operation of at least one of the at least one first processing element memory bank group or the at least one second processing element memory bank group.

18. The high-bandwidth memory of claim 17, wherein, each of the first processing element unit and the second processing element unit is further configured to generate an input selection signal based on the first operand location information and the second operand location information, generate a processing element execution control signal based on the operator information, and generate a register control signal and an output selection signal based on the destination location information.

19. The high-bandwidth memory of claim 18, wherein, the processing element processor includes: an input selector configured to select the first data, the second data, and / or the register output data based on the input selection signal; a calculator configured to perform the first processing operation based on the processing element execution control signal and output processing operation result data; a register configured to store the processing operation result data based on the register control signal; and an output selector configured to transfer the processing operation result data to one of the one or more first memory bank input / output line groups, the data bus, and the input selector based on the output selection signal.

20. The high-bandwidth memory of claim 19, wherein, The operator information in the first instruction includes a multiply-accumulate operator, a multiplication operator, or an addition / subtraction operator, wherein, based on the operator information in the first instruction being the multiplication operator or the addition / subtraction operator, the input selector selects the first data and the second data, and the calculator performs a multiplication operation or an addition / subtraction operation on the first data and the second data as the first processing operation, wherein, the first instruction further includes third operand information, wherein, based on the operator information in the first instruction being the multiply-accumulate operator, the input selector selects the first data, the second data, and the register output data, and the calculator performs a multiply-accumulate operation of adding the register output data to result data obtained by multiplying the first data and the second data as the first processing operation, wherein, the operator information in the second instruction is a move operator.

21. A system comprising: a high bandwidth memory including a buffer die and a plurality of memory dies stacked on the buffer die; and a controller configured to transmit commands and addresses to the buffer die and receive output data from the buffer die, wherein each of the plurality of memory dies includes at least one first processing element bank group and at least one second processing element bank group, wherein the at least one first processing element bank group includes one or more first banks connected to one or more first bank input / output line groups and a first processing element controller connected to the one or more first bank input / output line groups and a first global input / output line group, wherein the first processing element controller is configured to perform a first processing operation on first data output from one of the one or more first bank input / output line groups and second data transmitted through the first global input / output line group based on a first instruction, the first instruction being generated based on a first processing command, wherein the at least one second processing element bank group includes one or more second banks connected to one or more second bank input / output line groups and a second processing element controller connected to the one or more second bank input / output line groups and a second global input / output line group, wherein the second processing element controller is configured to perform a second processing operation of retrieving second data from one of the one or more second banks and transmitting the second data to the second global input / output line group based on a second instruction, the second instruction being different from the first instruction generated based on the first processing command, wherein the first global input / output line group and the second global input / output line group are commonly connected to a data bus.

22. The system of claim 21, wherein, each of the plurality of memory dies includes at least two memory channels, wherein each of the at least two memory channels includes at least two pseudo memory channels, wherein each of the at least two pseudo memory channels includes one of the at least one first processing element memory bank group and one of the at least one second processing element memory bank group, wherein each of the at least two pseudo memory channels includes a command and address generator configured to receive commands and addresses from the buffer die and generate command signals, bank addresses, row addresses, and column addresses.

23. The system of claim 22, wherein, The first processing element controller includes: a first processing element unit configured to generate a first processing element input / output gating signal and perform a first processing operation based on a first bank address applied with a first processing command; and one or more first processing element input / output gating units configured to isolate a connection between the one or more first bank input / output line groups and a first global input / output line group based on the first processing element input / output gating signal, wherein the second processing element controller includes: a second processing element unit configured to generate a second processing element input / output gating signal and perform a second processing operation based on a second bank address applied with a first processing command; and one or more second processing element input / output gating units configured to connect a connection between the one or more second bank input / output line groups and a second global input / output line group based on the second processing element input / output gating signal.

Images