Volatile memory device, memory controller, and memory system

Abstract

A volatile memory device, a memory controller, and a memory system are provided. A memory device includes: a memory cell array including a first space and a second space; a control logic circuit configured to control an operation of the memory cell array, in which the control logic circuit is configured to control the operation of the memory cell array in response to receiving a first read control command targeting the first space, reading model data from first model data stored at a first start address in a first space to second model data stored at a first end address in the first space in an order of addresses, and the control logic circuit is configured to read second data in a non-sequential order of addresses in response to receiving a second read control command targeting the second data stored in the second space.

Description

[0001] Cross-reference of related applications

[0002] This application claims priority and benefit to Korean Patent Application No. 10-2024-0110651, filed on August 19, 2024, with the Korean Intellectual Property Office, the entire disclosure of which is incorporated herein by reference. Technical Field

[0003] This disclosure relates to volatile memory devices, memory controllers, and memory systems. Background Technology

[0004] With the development of technologies such as artificial intelligence (AI), big data, and edge computing, there is a need to process larger volumes of data faster on devices. In other words, applications that perform complex calculations require faster data processing.

[0005] Furthermore, the expansion of generative AI models requires considerable resources and time to learn or infer these models. This process involves the exchange of large amounts of data between host devices such as CPUs and GPUs and memory. Therefore, research is underway to improve memory read and write performance in the user experience related to generative AI models. Summary of the Invention

[0006] The purpose of this disclosure is to provide a memory device with improved read performance.

[0007] Another object of this disclosure is to provide a memory controller that can improve the read performance of a memory device.

[0008] Another object of the present invention is to provide a memory system with improved read performance.

[0009] According to some embodiments of this disclosure, a volatile memory device is provided, comprising: a memory cell array including a first space and a second space; and control logic circuitry configured to control the operation of the memory cell array, wherein the control logic circuitry is configured to, in response to receiving a first read control command targeting model data stored in the first space, read model data from first model data stored at a first start address in the first space to second model data stored at a first end address in the first space in address order, and in response to receiving a second read control command targeting second data stored in the second space, read second data from the second space in address non-order.

[0010] According to some embodiments of this disclosure, a memory controller is provided, comprising: processing circuitry configured to control the operation of a volatile memory device, the volatile memory device including a first space allocated for storing a first model dataset and a second space allocated for storing second data; and a clock generator configured to provide a clock signal for operation of the volatile memory device to the volatile memory device, wherein the clock generator is configured to generate a first data clock signal having a first frequency and a second data clock signal having a second frequency lower than the first frequency, wherein the processing circuitry is configured to, in response to receiving a first read command targeting the first model dataset, provide the volatile memory device with: a first high-speed read control command instructing the volatile memory device to read the first model dataset in address order, a first data clock signal, a first start address as a first address of the first model dataset, and a first end address as a last address of the first model dataset, and wherein the processing circuitry is configured to, in response to receiving a second read command targeting the second data, provide the volatile memory device with: a second read control command instructing the volatile memory device to read the second data in address non-order, a second data clock signal, and an address of the second data.

[0011] According to some embodiments of this disclosure, a memory system is provided, comprising: a memory cell array including a first space and a second space; a volatile memory device including control logic circuitry configured to control the memory cell array; and a memory controller including processing circuitry configured to control the operation of the volatile memory device, wherein the memory controller further includes a clock generator configured to provide a clock signal for the operation of the volatile memory device to the volatile memory device, wherein the clock generator is configured to generate a first data clock signal having a first frequency and a second data clock signal having a second frequency lower than the first frequency, wherein the control logic circuitry is configured to, in response to receiving a first read control command targeting the first space, read at least some of a plurality of model data stored in the first space in address order based on the first data clock signal, and wherein the control logic circuitry is configured to, in response to receiving a second read control command targeting second data stored in the second space, read the second data in address non-order based on the second data clock signal.

[0012] The purposes of this disclosure are not limited to those mentioned above, and those skilled in the art will clearly understand from the following description of this disclosure other purposes not mentioned herein. Attached Figure Description

[0013] Figure 1 This is a block diagram showing the memory system.

[0014] Figure 2 It is shown Figure 1 A block diagram of the memory device.

[0015] Figure 3 It is shown Figure 2 A block diagram of a memory cell array.

[0016] Figure 4 It is shown Figure 3 A block diagram of the first artificial neural network model space.

[0017] Figure 5 It is shown Figure 3 A block diagram of the normal space.

[0018] Figure 6 This is a flowchart illustrating the process of reading normal data stored in the first space.

[0019] Figure 7 This is a flowchart illustrating the process of reading model data stored in the second space.

[0020] Figure 8 It is shown Figure 7 A detailed flowchart of the process of reading data from multiple models.

[0021] Figure 9 This is a flowchart illustrating the process of writing model data by allocating a third space.

[0022] Figure 10 It is shown Figure 9 A diagram of the third space.

[0023] Figure 11 It is shown Figure 1 Block diagram of the memory controller.

[0024] Figure 12 This is a block diagram illustrating the process of reading normal data stored in normal space.

[0025] Figure 13 This is a block diagram illustrating the process of reading the first model dataset stored in the first artificial neural network model space.

[0026] Figure 14 This is a block diagram showing the memory system.

[0027] Figure 15 This is a diagram showing the memory module.

[0028] Figure 16 This is a diagram showing a semiconductor package.

[0029] Figure 17This is a diagram illustrating an implementation example of a semiconductor package.

[0030] Figure 18 This is a diagram showing a semiconductor package. Detailed Implementation

[0031] In the following description, embodiments based on the technical spirit of this disclosure will be described with reference to the accompanying drawings.

[0032] Figure 1 This is a block diagram showing the memory system.

[0033] Reference Figure 1 The memory system may include a host device 30 and a memory storage device 1. The memory storage device 1 may include a memory device 20 and a memory controller 10.

[0034] The memory controller 10 can control the overall operation of the memory device 20. For example, the memory controller 10 can control the data exchange between the external host device 30 and the memory device 20. For example, the memory controller 10 can control the memory device 20 according to the request of the host device 30, thereby writing or reading data.

[0035] The memory controller 10 and the memory device 20 can communicate with each other via the memory interface MEM I / F. Furthermore, the memory controller 10 and the external host device 30 can communicate with each other via a host interface. That is, the memory controller 10 can relay signals between the memory device 20 and the host device 30. The memory controller 10 can control the operation of the memory device 20 by applying commands (CMD) to control the memory device 20. In this case, the memory device 20 may include dynamic memory cells. For example, the memory device 20 may include dynamic random access memory (DRAM), double data rate 4 (DDR4), synchronous DRAM (SDRAM), low power DDR4 (LPDDR4) SDRAM, or LPDDR5 SDRAM, but embodiments according to the spirit of this disclosure are not limited thereto. The memory device 20 may include a non-volatile memory device. However, in this embodiment, the memory device 20 will be described as a volatile memory device.

[0036] The memory controller 10 can send clock signals CLK, commands CMD, address ADDR signals, etc., to the memory device 20. The memory controller 10 can provide data DQ to the memory device 20 and can receive data DQ from the memory device 20. The memory device 20 may include a memory cell array 280 for storing data DQ, control logic circuitry 210, and a data input / output (I / O) buffer 295.

[0037] Figure 2 It is shown Figure 1 A block diagram of the memory device.

[0038] Reference Figure 2 The memory device 20 may include control logic circuitry 210, voltage generator 213, address register 220, memory bank control logic circuitry 230, row address multiplexer (RA MUX) 240, refresh counter 242, refresh address generator 244, column address latch 250, row decoder 260, column decoder 270, memory cell array 280, sense amplifier 285, input / output gating circuitry 290, and data input / output buffer 295.

[0039] The memory cell array 280 may include multiple memory bank arrays 280a to 280h. Although Figure 2 The memory cell array 280 shown includes eight memory bank arrays 280a to 280h, but this disclosure is not limited thereto.

[0040] Each of the multiple memory bank arrays 280a to 280h may include multiple word lines WL, multiple bit lines BL, and multiple memory cells MC formed at the points where the word lines WL and bit lines BL intersect each other.

[0041] The row decoder 260 may include multiple memory bank row decoders 260a to 260h, each connected to a plurality of memory bank arrays 280a to 280h. The column decoder 270 may include multiple column decoders 270a to 270h, each connected to a plurality of memory bank arrays 280a to 280h. The sense amplifier 285 may include multiple sense amplifiers 285a to 285h, each connected to a plurality of memory bank arrays 280a to 280h.

[0042] Address register 220 can be obtained from the memory controller ( Figure 1 10) Receive address ADDR, which includes the bank address BANK_ADDR, the row address ROW_ADDR, and the column address COL_ADDR. Address register 220 can provide the received bank address BANK_ADDR to the bank control logic circuit 230, the received row address ROW_ADDR to the row address multiplexer 240, and the received column address COL_ADDR to the column address latch 250.

[0043] The memory bank control logic circuit 230 can generate a memory bank control signal in response to the memory bank address BANK_ADDR. In response to the memory bank control signal, the memory bank row decoder corresponding to the memory bank address BANK_ADDR among the multiple memory bank row decoders 260a to 260h can be activated, and the column decoder corresponding to the memory bank address BANK_ADDR among the multiple column decoders 270a to 270h can also be activated.

[0044] The refresh counter 242 can sequentially output the count row address CRA under the control of the control logic circuit 210. For example, the control logic circuit 210 can generate a refresh count signal in response to a normal refresh command. The refresh counter 242 can perform a counting operation in response to the refresh count signal and can output the count row address CRA. That is, the refresh counter 242 can output the refresh address used to perform the normal refresh operation.

[0045] The refresh address generator 244 can receive the bank address BANK_ADDR and the row address ROW_ADDR. The refresh address generator 244 can count the activated values ​​of the bank address BANK_ADDR and the row address ROW_ADDR based on the count value. The refresh address generator 244 can generate a hammer address based on the count value, corresponding to a word line that has been activated more than a predetermined number of times, or the row address corresponding to an adjacent word line. In other words, the refresh address generator 244 can output a refresh address for performing a target row refresh operation.

[0046] The refresh address generator 244 can output one of the count row address CRA and the hammer address as the refresh row address RRA.

[0047] The refresh counter 242 and the refresh address generator 244 can be implemented as separate elements as shown, or they can be implemented as a single element. Alternatively, the refresh counter 242 and the refresh address generator 244 can be included in the control logic circuitry 210.

[0048] The row address multiplexer 240 can receive the row address ROW_ADDR from the address register 220 and the refresh row address RRA from the refresh address generator 244. The row address multiplexer 240 can selectively output either the row address ROW_ADDR or the refresh row address RRA as the row address RA. The row address RA output from the row address multiplexer 240 can be applied to each of the multiple memory bank row decoders 260a to 260h.

[0049] The memory bank row decoder 260a to 260h, activated by the memory bank control logic circuit 230, can decode the row address RA output from the row address multiplexer 240 to activate the word line corresponding to the row address. For example, the activated memory bank row decoder can apply a word line drive voltage to the word line corresponding to the row address.

[0050] Column address latch 250 can receive column address COL_ADDR from address register 220 and can temporarily store the received column address COL_ADDR. Column address latch 250 can gradually increment the column address COL_ADDR received in burst mode. Column address latch 250 can apply the temporarily stored column address COL_ADDR or the gradually incremented column address COL_ADDR to each of the multiple column decoders 270a to 270h.

[0051] The memory bank column decoder among the multiple column decoders 270a to 270h, activated by the memory bank control logic circuit 230, can activate the sense amplifier corresponding to the memory bank address BANK_ADDR and the column address COL_ADDR through the corresponding input / output gating circuit 290.

[0052] The input / output gating circuit 290 may include input data masking logic, a read data latch for storing data output from multiple memory bank arrays 280a to 280h, a write driver for writing data into the multiple memory bank arrays 280a to 280h, and circuitry for gating the input / output data.

[0053] Data DQ to be read from one of the multiple memory bank arrays 280a to 280h can be read by the sense amplifier (one of 285a to 285h) corresponding to that memory bank array and stored in a read data latch. The data DQ stored in the read data latch can be provided to the memory controller 10 through the data input / output buffer 295.

[0054] Data DQ to be written to one of the multiple memory bank arrays 280a to 280h can be provided to the input / output gate circuit 290, and the input / output gate circuit 290 can write the data into the memory bank array through the write driver.

[0055] Control logic circuitry 210 can control the operation of memory device 20. For example, control logic circuitry 210 can generate control signals that cause memory device 20 to perform write or read operations. Control logic circuitry 210 may include command decoder 211 for decoding commands (CMD) received from memory controller 10 and mode register 212 for setting the operating mode of memory device 20.

[0056] According to some embodiments, the memory controller ( Figure 1 10) Information stored in mode register 212 can be read to check whether memory device 20 supports high-speed (Turbo) read mode (e.g., configured to read data in address order and address non-order in response to a corresponding read control command, as described herein).

[0057] For example, the command CMD may include an activation command for switching the memory cell array 280 to an active state for writing or reading data, a precharge command for switching the memory cell array 280 to a standby state, a refresh command for controlling the refresh operation of the memory cell array 280, and a command for reading information stored in the mode register 212.

[0058] Voltage generator 213 can be obtained from an external device (e.g., Figure 1 The memory controller 10) or the host device ( Figure 1 (30) Receives voltages to generate various voltages sufficient for the operation of the memory device 20. For example, voltage generator 213 may receive a first input voltage VDD1 and a second input voltage VDD2H to generate and provide various voltages sufficient for the operation of the control logic circuit 210, the multiple bank row decoders 260a to 260h, and the sense amplifiers 285a to 285h of the memory device 20. However, voltage generator 213 may also receive input voltages other than the first input voltage VDD1 and the second input voltage VDD2H, and may also generate and provide voltages sufficient for other components in the memory device 20 besides the control logic circuit 210, the multiple bank row decoders 260a to 260h, and the sense amplifiers 285a to 285h.

[0059] Memory cells MC can be, for example, DRAM memory cells. Each memory cell MC can be connected to a word line WL and a bit line BL. Memory cells MC can store charge through cell capacitors. Due to the structure of memory cells MC, leakage current is generated in the memory cells MC, so the data stored in the cell capacitors may be corrupted.

[0060] Therefore, the memory device 20 can perform a refresh operation to recharge the data in the memory cell MC to prevent the data stored in the memory cell MC from being changed due to leakage current.

[0061] Figure 3 It is shown Figure 2 A block diagram of a memory cell array.

[0062] Reference Figure 3 A portion of the memory cell array 280 may be allocated to the first artificial neural network model space MA1. The remaining portion not allocated to a specific space may be the normal space NA. As described above, since the memory cell array 280 may include multiple memory bank arrays 280a to 280h, the first artificial neural network model space MA1 may be allocated to a portion of each of the multiple memory bank arrays 280a to 280h.

[0063] Figure 4 It is shown Figure 3 A block diagram of the first artificial neural network model space.

[0064] Reference Figure 4 Multiple model data sets MD1_1 to MD1_n can be stored in a first artificial neural network model space MA1. Each of the multiple stored model data sets MD1_1 to MD1_n can have a physical address based on its location stored in the memory cell array 280. For example, among the multiple model data sets MD1_1 to MD1_n, the first model data set MD1_1 can have a first physical address PA1_1 based on its location stored in the memory cell array 280. Among the multiple model data sets MD1_1 to MD1_n, the nth model data set MD1_n can have an nth physical address PA1_n based on its location stored in the memory cell array 280.

[0065] Memory interleaving is a technique that stores data with adjacent addresses in different memory banks, thereby allowing simultaneous access to the stored data. Since memory interleaving is well known to those skilled in the art, its detailed description will be omitted.

[0066] According to some embodiments, based on memory interleaving technology, multiple model data MD1_1 to MD1_n stored in the first artificial neural network model space MA1 can be stored sequentially (e.g., in sequentially ordered addresses) in multiple memory storage arrays according to address order (e.g., from the address with the smallest address number to the address with the largest address number). Figure 3In the first artificial neural network model space MA1 of each of (280a to 280h). For example, according to memory interleaving technology, multiple model data MD1_1 to MD1_n can be sequentially stored in multiple memory storage arrays ( ) in the address order from the first model data MD_1 stored at the first physical address PA1_1 as the first address to the nth model data MD1_n stored at the nth physical address PA1_n as the last address. Figure 3 In the first artificial neural network model space MA1 of each of 280a to 280h. The first address may be, for example, the physical address with the smallest address number among the physical addresses storing multiple model data MD1_1 to MD1_n. The last address may be, for example, the physical address with the largest address number among the physical addresses storing multiple model data MD1_1 to MD1_n.

[0067] Figure 5 It is shown Figure 3 A block diagram of the normal space.

[0068] Reference Figure 5 Normal data ND1 can be stored in normal space NA. Normal data ND1 can have a first physical address PA0_k based on its location stored in memory cell array 280. Normal data ND1 can be stored in a non-sequential (e.g., random) location within memory cell array 280, and the first physical address PA0_k storing normal data ND1 can be used to access normal data ND1.

[0069] Figure 6 This is a flowchart illustrating the process of reading normal data stored in the first space.

[0070] Reference Figures 1 to 6 Describe the process of reading normal data stored in the first space.

[0071] Receive a normal read control command for normal data (S110). For example, the memory device 20 can receive a normal read control command for normal data ND1 stored in the first space from the memory controller 10 via the command decoder 211. The first space can be the normal space NA, but is not limited thereto.

[0072] In order for memory device 20 to access normal data ND1 stored in the first space, memory device 20 needs to know the location of normal data ND1 stored in memory cell array 280. That is, in order to access normal data ND1, memory device 20 may need information about the physical address of normal data ND1. Therefore, normal read control commands for normal data ND1 may include information about the physical address PA0_k of normal data ND1.

[0073] Receive the first data clock signal (S120). For example, the memory device 20 may receive the first data clock signal WCK1 from the memory controller 10 as a clock signal for reading normal data ND1.

[0074] Generate a first read voltage (S130). For example, the voltage generator 213 of the memory device 20 may generate the first read voltage V based on a normal read control command received by the control logic circuit 210 by using the received first input voltage VDD1 and the received second input voltage VDD2H. r1 First reading voltage V r1 It can be the voltage used to read normal data ND1 based on the first data clock signal WCK1.

[0075] Normal data is read by non-sequential (e.g., random) access (S140). For example, the control logic circuit 210 of the memory device 20 may read normal data ND1 stored in a first space (e.g., normal space NA) of the memory cell array 280 based on a first data clock signal WCK1. During the read process, normal data ND1 may be output in the form of multiple data segments DQ equivalent to a preset burst length. To access normal data ND1, the control logic circuit 210 of the memory device 20 may read normal data ND1 by non-sequential (e.g., random) access based on information about the physical address PA0_k where normal data ND1 is stored, included in the normal read control command.

[0076] Figure 7 This is a flowchart illustrating the process of reading model data stored in the second space.

[0077] Reference Figures 1 to 5 and Figure 7 The system receives high-speed (turbo) read control commands for the second space (S210). For example, the memory device 20 can receive high-speed read control commands from the memory controller 10 via the command decoder 211. The second space can be the artificial neural network model space described above, but is not limited thereto.

[0078] According to some embodiments, with Figure 6Unlike S110, the high-speed read control command may not include information about the physical addresses PA1_1 to PA1_n corresponding to the multiple model data MD1_1 to MD1_n stored in the second space (e.g., the first artificial neural network model space MA1). For example, the high-speed read control command may include information about the second space (e.g., the first artificial neural network model space MA1) instead of the physical addresses PA1_1 to PA1_n corresponding to the multiple model data MD1_1 to MD1_n respectively. In this case, the high-speed read control command may instruct the memory device 20 to read all the data (e.g., the multiple model data MD1_1 to MD1_n) stored in the second space (e.g., the first artificial neural network model space MA1).

[0079] Unlike the example above, when reading a first model dataset MDS1 (e.g., MD1_2 to MD_(n-1)) that is part of a plurality of model data MD1_1 to MD1_n, the high-speed read control command may include information about the first address (e.g., PA1_2) among the physical addresses (e.g., PA1_2 to PA1_(n-1)) corresponding to the first model dataset MDS1 to be read, and information about the last address (e.g., PA1_(n-1)). In this case, the high-speed read control command may instruct the memory device 20 to read the first model dataset MDS1 (e.g., MD1_2 to MD_(n-1)) that is part of a plurality of model data MD1_1 to MD1_N stored in a second space (e.g., the first artificial neural network model space MA1).

[0080] Receive a second data clock signal (S220). For example, the memory device 20 may receive a second data clock signal WCK2 from the memory controller 10. The second data clock signal WCK2 is a clock signal used to read at least a portion of a plurality of model data MD1_1 to MD1_n.

[0081] According to some embodiments, the second data clock signal WCK2 may have a ratio of Figure 6The first data clock signal WCK1 of S120 has a higher frequency. Typically, the maximum bandwidth of the memory device 20 is determined by multiplying the frequency of the data clock signal by the size of the memory bus. Since the frequency of the second data clock signal WCK2 is higher than the frequency of the first data clock signal WCK1, the maximum bandwidth of the memory device 20 is greater when reading data based on the second data clock signal WCK2 than when reading data based on the first data clock signal WCK1. For example, the maximum bandwidth can be approximately 10.7 Gbps when the memory device 20 reads data based on the second data clock signal WCK2, and approximately 9.6 Gbps when reading data based on the first data clock signal WCK1. In other words, data can be read faster when reading data based on the second data clock signal WCK2 than when reading data based on the first data clock signal WCK1.

[0082] A second read voltage is generated (S230). For example, the voltage generator 213 of the memory device 20 can generate the second read voltage V based on a high-speed read control command received by the control logic circuit 210, using the received first input voltage VDD1 and the received second input voltage VDD2H. r2 The second reading voltage V r2 It can be a voltage used to read at least a portion of multiple model data MD1_1 to MD1_n based on the second data clock signal WCK2.

[0083] According to some embodiments, since the second data clock signal WCK2 has a higher... Figure 6 The first data clock signal WCK1 of S120 has a high frequency, therefore the second read voltage V used to read data based on the second data clock signal WCK2 is... r2 Can be higher Figure 6 The first read voltage V of S130 for reading data based on the first data clock signal WCK1 r1 .

[0084] At least some of the multiple model data are read in address order (e.g., by sequential access) (S240). For example, the control logic circuit 210 of the memory device 20 may read at least some of the multiple model data MD1_1 to MD1_n stored in the second space (e.g., the first artificial neural network model space MA1) of the memory cell array 280 based on the second data clock signal WCK2. During the read process, each of the multiple model data MD1_1 to MD1_n may be output in the form of multiple data segments DQ equivalent to a preset burst length.

[0085] According to some embodiments, the high-speed read control command may include information about a second space (e.g., a first artificial neural network model space MA1). In this case, the memory device 20 may access and read all data (e.g., multiple model data MD1_1 to MD1_n) stored in the second space (e.g., the first artificial neural network model space MA1) in address order. For example, in response to receiving a high-speed read control command including information about the second space (e.g., the first artificial neural network model space MA1), the memory device 20 may access multiple model data MD1_1 to MD1_n sequentially from the first model data MD1_1 corresponding to the first start address PA1_1 as the first address to the nth model data MD1_n corresponding to the first end address PA1_n as the last address, and may read all data stored in the second space (e.g., the first artificial neural network model space MA1) in address order.

[0086] According to some embodiments, in the physical address (e.g., PA1_2 to PA1_(n-1)) corresponding to a first model dataset MDS (e.g., MD1_2 to MD_(n-1)) which is part of a plurality of model data MD1_1 to MD1_n, the high-speed read control command may include information about a first start address (e.g., PA1_2) as a first address and information about a first end address (e.g., PA1_(n-1)) as a last address. In this case, the memory device 20 can read the first model dataset MDS (e.g., MD1_2 to MD_(n-1)) which is part of a plurality of model data (e.g., MD1_1 to MD1_n) in address order by sequentially accessing the model data (e.g., MD1_2) corresponding to the first start address (e.g., PA1_1) to the model data (e.g., MD_(n-1)) corresponding to the first end address (e.g., PA1_(n-1)).

[0087] Figure 8 It is shown Figure 7 A detailed flowchart of the process of reading data from multiple models.

[0088] Reference Figure 8The system accesses and reads the first model data corresponding to the first starting address (S241). According to some embodiments, when the high-speed read control command includes information about a second space (e.g., the first artificial neural network model space MA1), the first address of the second space (e.g., the first artificial neural network model space MA1) can be the first starting address. In other words, the first address (e.g., PA1_1 to PA1_n) among the physical addresses (e.g., PA1_1 to PA1_n) corresponding to all stored data (e.g., multiple model data MD1_1 to MD1_n) can be the first starting address. The first model data (e.g., MD1_1) to be read can be provided to the memory controller 10.

[0089] Furthermore, according to some embodiments, when the high-speed read control command includes information about a first start address (e.g., PA1_2) as a first address among the physical addresses (e.g., PA1_2 to PA1_(n-1)) corresponding to a first model dataset MDS (e.g., MD1_2 to MD_(n-1)) which is part of a plurality of model data MD1_1 to MD1_n, and information about a first end address (e.g., PA1_(n-1)) as a last address, the received first start address (e.g., PA1_2) can be used as the first start address as is. The first model data (e.g., MD1_1) to be read can be provided to the memory controller 10.

[0090] Access and read the model data corresponding to the next address of the read model data (S242). For example, after reading the first model data (e.g., MD1_1), the model data (e.g., MD1_2) corresponding to the next address (e.g., PA1_2) of the first model data (e.g., MD1_1) can be accessed and read. The read model data (e.g., MD1_2) can be provided to the memory controller 10.

[0091] Determine whether the next address of the read model data is the first end address (S243). According to some embodiments, for example, when the high-speed read control command includes information about a second space (e.g., the first artificial neural network model space MA1), the last address of the second space (e.g., the first artificial neural network model space MA1) can be the first end address.

[0092] Furthermore, according to some embodiments, when the high-speed read control command includes information about a first start address (e.g., PA1_2) among the physical addresses (e.g., PA1_2 to PA1_(n-1)) corresponding to a first model dataset MDS (e.g., MD1_2 to MD_(n-1)) which is part of a plurality of model data MD1_1 to MD1_n, and information about a first end address (e.g., PA1_(n-1)) as the last address, the received first end address (e.g., PA1_(n-1)) can be used as the first end address as is.

[0093] When it is determined that the next address of the read model data is the first end address (S243 - Yes), the reading operation can be terminated. When it is determined that the next address of the read model data is not the first end address (S243 - No), the current step returns to step S242, and the reading operation can continue by accessing the model data corresponding to the next address of the read model data.

[0094] Based on current trends, mobile communication devices such as smartphones and laptops are using various artificial intelligence models on-device to internalize AI applications. One of the key factors determining processing speed in the user experience of on-device applications using Large Language Models (LLMs) (an AI model) is the average time to generate tokens (data units of characters recognized by the LLM). The generation time of tokens generated after the first token is primarily affected by the read bandwidth of main memory (e.g., Dynamic Random Access Memory (DRAM)). In other words, when high read bandwidth can be guaranteed, the user experience of on-device applications using generative AI models can be greatly improved.

[0095] According to some embodiments, the memory device of this disclosure can store model data in an artificial neural network model space by allocating the model space separately from the normal space. Then, when reading the stored model data, the model data can be read based on a momentarily high data clock signal, thereby obtaining a momentarily high read bandwidth.

[0096] According to some embodiments, the memory device according to this disclosure can read model data based on a higher data clock signal only in a specific state by receiving a specific command (e.g., the high-speed read control command described above), thereby significantly improving the user experience of device-side applications that use generated AI models, while minimizing the increase in power consumption by using the high data clock signal.

[0097] According to some embodiments, the memory device according to this disclosure achieves stable overclocking by simultaneously increasing the internal voltage according to the use of a high data clock signal to ensure the operating margin of the memory device.

[0098] According to some embodiments, considering the memory interleaving in the artificial neural network model space, the memory device according to this disclosure writes data to addresses ordered in sequence and reads data in the order of addresses, thus achieving optimal read performance.

[0099] Figure 9 This is a flowchart illustrating the process of writing model data by allocating a third space. Figure 10 It is shown Figure 9 A diagram of the third space.

[0100] Reference Figure 9 and Figure 10 The system receives a space allocation control command (S310) instructing the allocation of space for storing model data. For example, the memory device 20 may receive a space allocation control command from the memory controller 10, which instructs the allocation of space for storing multiple model data that are different from the multiple model data stored in the second space (e.g., the first artificial neural network model space MA1). The third space may be a second artificial neural network model space MA2, which is different from the first artificial neural network model space MA1 described above, but is not limited thereto.

[0101] The third space is allocated to the memory cell array (S320). For example, the memory device 20 may allocate the third space (e.g., the second artificial neural network model space MA2) to the memory cell array 280 in response to receiving a space allocation control command from the memory controller 10. The memory device 20 may convert a portion of the first space (e.g., the normal space NA) into the third space (e.g., the second artificial neural network model space MA2), but the allocation method is not limited to this, and the third space (e.g., the second artificial neural network model space MA2) may be allocated in various ways.

[0102] The system receives a first write control command (S330) instructing the third to fourth model data to be written into a third space. For example, the memory device 20 may receive the first write control command and a plurality of model data including the third model data MD2_1 and the fourth model data MD2_n from the memory controller 10. The first write control command instructs the third model data MD2_1 to the fourth model data MD2_n to be written into a third space (e.g., the second artificial neural network model space MA2).

[0103] The third model data to the fourth model data are written to the third space in sequential address order (S340). For example, the memory device 20 may write the third model data MD2_1 to the fourth model data MD2_n sequentially in the third space (e.g., the second artificial neural network model space MA2) in address order according to memory interleaving technology.

[0104] Multiple model data MD2_1 to MD2_n stored in the third space (e.g., the second artificial neural network model space MA2) can be read using the method described above. For example, in response to receiving a high-speed read control command for the third space (e.g., the second artificial neural network model space MA2) from the memory controller 10, the memory device 20 can sequentially access and read the fifth model data (e.g., MD2_2) corresponding to the second starting address (e.g., PA2_2) to the sixth model data (e.g., MD2_(n-1)) corresponding to the second ending address (e.g., PA2_(n-1)).

[0105] Figure 11 It is shown Figure 1 Block diagram of the memory controller.

[0106] Reference Figure 11 The memory controller 10 may include processing circuitry 110 and a clock generator 120. Processing circuitry 110 controls the overall operation of the memory controller 10. For example, processing circuitry 110 may... Figure 1 The host device 30 receives a data read command and controls the memory device 20 so that the memory device 20, which stores the data requested by the host device 30, can read the data requested by the host device 30.

[0107] Clock generator 120 can generate various clock signals sufficient to operate memory device 20. Memory device 20 can perform data write or read operations based on the clock signals received from clock generator 120. The higher the frequency of the received clock signal, the faster the data write or read operation may be. According to some embodiments, clock generator 120 can generate a first data clock signal WCK1 and a second data clock signal WCK2. The second data clock signal WCK2 may have a higher frequency than the first data clock signal WCK1. When memory device 20 reads data stored in normal space, the first data clock signal WCK1 can be a reference clock signal. When memory device 20 reads data stored in a first artificial neural network model space (and a second artificial neural network model space), the second data clock signal WCK2 can be a reference clock signal.

[0108] Figure 12 This is a block diagram illustrating the process of reading normal data stored in normal space.

[0109] Memory controller 10 may receive a first read command CMD1 from an external device (e.g., host device 30) regarding normal data stored in the normal space. Memory controller 10 may, in response to the first read command CMD1, provide memory device 20 with a first data clock signal WCK1 generated by clock generator 120. Memory controller 10 may, in response to the first read command CMD1, provide memory device 20 with the address of the normal data. The address of the normal data may represent the physical address corresponding to the normal data stored in the normal space NA of memory cell array 280. In response to the first read command CMD1, memory controller 10 may provide memory device 20 with a normal read control command instructing the normal data to be read by accessing the normal data non-sequentially (e.g., randomly). Memory device 20 may, according to the normal read control command, access the normal data non-sequentially (e.g., randomly) based on the address of the normal data using the first data clock signal WCK1. Memory device 20 may provide the read normal data to memory controller 10.

[0110] Figure 13 This is a block diagram illustrating the process of reading the first model dataset stored in the first artificial neural network model space.

[0111] The memory controller 10 can be accessed from an external device (e.g., Figure 1 The host device 30 receives a second read command CMD2 for a first model dataset among multiple model data stored in the first artificial neural network model space. The memory controller 10 may, in response to the second read command CMD2, provide the memory device 20 with a second data clock signal WCK2 generated by the clock generator 120. The memory controller 10 may, in response to the second read command CMD2, provide the memory device 20 with a first start address and a first end address of the first model dataset. The first start address may represent the physical address corresponding to the model data with the lowest address number in the first model dataset stored in the first artificial neural network model space MA1 of the memory cell array 280. The first end address may represent the physical address corresponding to the model data with the highest address number in the first set of model data stored in the first artificial neural network model space MA1 of the memory cell array 280. In response to the second read command CMD2, the memory controller 10 may provide the memory device 20 with a high-speed read control command instructing the access and reading of the first model dataset sequentially according to address order. The memory device 20 can sequentially access the first start address to the first end address based on the second data clock signal WCK2 according to a high-speed read control command to read the first model dataset. The memory device 20 can provide the read first model dataset to the memory controller 10.

[0112] According to some embodiments, the memory controller according to this disclosure can specify only a specific storage space in the memory device (e.g., the first artificial neural network model space described above), and does not need to specify the address of each piece of data to be read separately. Therefore, the address mapping process of converting the system address of the data to be read into the physical address of the memory device can be omitted. Thus, resources of the memory controller and the host device can be saved.

[0113] According to some embodiments, the memory controller according to this disclosure can specify only the first and last addresses of the plurality of model data to be read in the memory device. That is, the addresses between the first and last addresses of the plurality of model data to be read do not need to be provided to the memory device. Therefore, resources of the memory controller and the host device can be saved.

[0114] Figure 14 This is a block diagram showing the memory system.

[0115] Reference Figure 14 The host device 30 may include a memory controller 10. That is, in reference to... Figure 1 In the described memory system, the memory controller 10 is located external to the host device 30, which may include the memory controller 10 according to embodiments of the present disclosure. The host device 30 can control the memory device 20 via the memory controller 10. In this case, the host device 30 may perform communication with the memory device 20 based on one of the standards such as Double Data Rate (DDR), Low Power Double Data Rate (LPDDR), Graphics Double Data Rate (GDDR), Wide I / O, High Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), or Computer Ethernet Link (CXL).

[0116] Figure 15 This is a diagram showing the memory module.

[0117] Reference Figure 15 The memory module 1a may include a controller 110_1 and a plurality of memory devices 20_1 to 20_8. The memory module 1a may be installed in an electronic device.

[0118] Multiple memory devices 20_1 to 20_8 may correspond to the aforementioned memory device 20. Controller 110_1 may perform some of the functions of the aforementioned memory controller 10.

[0119] Host (e.g., Figure 1The host device 30 can control the memory module 1a according to a communication protocol such as Double Data Rate (DDR) or Low Power DDR (LPDDR). For example, in order to read data stored in the memory module 1a, the host (e.g., Figure 1 The host device 30) can send commands and addresses to the memory module 1a.

[0120] Multiple memory devices 20_1 to 20_8 can write data or output written data under the control of the CPU. Each of the multiple memory devices 20_1 to 20_8 can be at least one of dynamic random access memory (DRAM) and SDRAM.

[0121] Multiple memory devices 20_1 to 20_8 can exchange data DQ in response to signals provided from controller 110_1. The multiple memory devices 20_1 to 20_8 may also include data buffers for data communication, and the data buffers can be synchronized with a data strobe signal DQS to exchange data DQ with host device 30. Unlike the example shown, the multiple memory devices 20_1 to 20_8 can communicate with a host (e.g., via controller 110_1) Figure 1 The host device 30) performs communication of data DQ.

[0122] According to some embodiments, controller 110_1 may perform communication with memory devices 20_1 to 20_8 according to one of the standards of memory modules such as dual in-line memory modules (DIMM), registered DIMM (RDIMM), load-reduced DIMM (LRDIMM), and UDIMM.

[0123] The controller 110_1 can receive the command / address CA and clock signal CK of the memory module 1a through the memory input / output pins, and can provide the received signals to the memory devices 20_1 to 20_8.

[0124] Figure 16 This is a diagram showing a semiconductor package.

[0125] Reference Figure 16 The semiconductor package may include a stacked memory device 1100, a system-on-a-chip (SoC) 1200, an interposer 1300, and a packaging substrate 1400. The stacked memory device 1100 may include a buffer chip 1110 and core chips 1120 to 1150. Core chips 1120 to 1150 may include reference... Figures 1 to 15The memory device 20 is described. Buffer chip 1110 may include a physical layer (PHY) 1111 and a direct access area (DAB) 1112. Physical layer 1111 is electrically connected to physical layer 1210 of system-on-chip 1200 via insert 1300. Stacked memory device 1100 may receive signals from or transmit signals to system-on-chip 1200 via physical layer 1111.

[0126] Direct access area 1112 provides an access path for testing stacked memory devices 1100 without going through system-on-chip 1200. Direct access area 1112 may include conductive components (e.g., ports or pins) that can directly perform communication with external test equipment. Test signals and data received through direct access area 1112 can be transmitted to core wafers 1120 to 1150 via TSV. Data read from core wafers 1120 to 1150 to test core wafers 1120 to 1150 can be transmitted to test equipment via TSV and direct access area 1112. Therefore, direct access testing of core wafers 1120 to 1150 can be performed.

[0127] Buffer chip 1110 and core chips 1120 to 1150 can be electrically connected to each other via TSV 1101 and bump 1102. Buffer chip 1110 can receive signals from system-on-chip 1200 provided to each channel via bump 1102 assigned to each channel. For example, bump 1102 can be a microbump.

[0128] The system-on-chip 1200 can execute applications supported by the semiconductor package 1000 by using the stacked memory device 1100. For example, the system-on-chip 1200 may include at least one processor selected from a central processing unit (CPU), application processor (AP), graphics processing unit (GPU), neural processing unit (NPU), tensor processing unit (TPU), vision processing unit (VPU), image signal processor (ISP), and digital signal processor (DSP) to perform specialized calculations.

[0129] The system-on-chip 1200 may include a physical layer 1210 and a memory controller 1220. The physical layer 1210 may include input / output circuitry for sending and receiving signals to and from the physical layer 1111 of the stacked memory device 1100. The system-on-chip 1200 may provide various signals to the physical layer 1111 via the physical layer 1210. Signals provided to the physical layer 1111 may be transmitted to the core wafers 1120 to 1150 via the interface circuitry of the TSV 1101 and the physical layer 1111.

[0130] The memory controller 1220 controls the overall operation of the stacked memory device 1100. The memory controller 1220 can send signals for controlling the stacked memory 1100 to the stacked memory 1100 via the physical layer 1210. The memory controller 1220 can correspond to... Figure 1 The memory controller 10.

[0131] Insert 1300 connects stacked memory device 1100 to system-on-chip 1200. Insert 1300 connects physical layer 1111 of stacked memory device 1100 to physical layer 1210 of system-on-chip 1200 and provides physical paths formed using conductive materials. Therefore, stacked memory device 1100 and system-on-chip 1200 can be stacked on insert 1300 to send and receive signals to each other.

[0132] Bump 1103 may be attached to the upper portion of package substrate 1400, and solder ball 1104 may be attached to the lower portion of package substrate 1400. For example, bump 1103 may be a flip-chip bump. Insert 1300 may be stacked on package substrate 1400 via bump 1103. Semiconductor package 1000 may send signals to and receive signals from other external packages or semiconductor devices via solder ball 1104. For example, package substrate 1400 may be a printed circuit board (PCB).

[0133] Figure 17 This is a diagram illustrating an implementation example of a semiconductor package.

[0134] Reference Figure 17 The semiconductor package 2000 may include multiple stacked memory devices 2100 and system-on-chip 2200. The stacked memory devices 2100 and system-on-chip 2200 may be stacked on an insert 2300, and the insert 2300 may be stacked on a package substrate 2400. The semiconductor package 2000 may send signals to and receive signals from other external packages or semiconductor devices via solder balls 2001 attached to the lower part of the package substrate 2400.

[0135] Each of the stacked memory devices 2100 may be implemented based on the HBM standard, but this disclosure is not limited thereto. Each of the stacked memory devices 2100 may be implemented based on GDDR, HMC, or wide I / O standards. Each of the stacked memory devices 2100 may correspond to Figure 16 Stacked memory device 1100.

[0136] The system-on-a-chip 2200 may include at least one processor such as a CPU, AP, GPU, and NPU, and multiple memory controllers for controlling multiple stacked memory devices 2100. The system-on-a-chip 2200 can send signals to and receive signals from corresponding stacked memory devices via the memory controllers. Figure 16 The System-on-Chip 1200.

[0137] Figure 18 This is a diagram showing a semiconductor package.

[0138] Reference Figure 18 The semiconductor package 3000 may include a stacked memory device 3100, a host chip 3200, and a package substrate 3300. The stacked memory device 3100 may include a buffer chip 3110 and core chips 3120 to 3150. The buffer chip 3110 may include a physical layer 3111 for performing communication with the host chip 3200, and each of the core chips 3120 to 3150 may include a memory cell array.

[0139] The host chip 3200 may include a physical layer 3210 for performing communication with the stacked memory device 3100, and a memory controller 3220 for controlling the overall operation of the stacked memory device 3100. The host chip 3200 may also include a processor for controlling the overall operation of the semiconductor package 3000 and executing applications supported by the semiconductor package 3000. For example, the host chip 3200 may include at least one processor, such as a CPU, AP, GPU, and NPU.

[0140] The stacked memory device 3100 can be mounted on the host chip 3200 based on the TSV 3001 and stacked vertically on the host chip 3200. Therefore, the buffer chip 3110, core chips 3120 to 3150, and host chip 3200 can be electrically connected to each other via the TSV 3001 and bumps 3002 without inserts. For example, bumps 3002 can be microbumps.

[0141] Bump 3003 may be attached to the upper part of package substrate 3300, and solder ball 3004 may be attached to the lower part of package substrate 3300. For example, bump 3003 may be a flip-chip bump. Host chip 3200 may be stacked on package substrate 3300 via bump 3003. Semiconductor package 3000 may send signals to and receive signals from other external packages or semiconductor devices via solder ball 3004.

[0142] Although embodiments of the present disclosure have been described with reference to the accompanying drawings, the present disclosure is not limited to the above embodiments, but can be implemented in various different forms. Those skilled in the art will understand that the present disclosure can be implemented in other specific forms without changing the technical spirit or essential characteristics of the present disclosure. Therefore, it should be understood that the above embodiments are illustrative in all respects rather than limiting.

Claims

1. A volatile memory device, comprising: A memory cell array, comprising a first space and a second space; as well as Control logic circuitry configured to control the operation of the memory cell array. The control logic circuit is configured as follows: In response to receiving a first read control command targeting model data stored in the first space, the model data is read in address order from the first model data stored at the first start address in the first space to the second model data stored at the first end address in the first space. In response to receiving a second read control command targeting second data stored in the second space, the second data is read from the second space in non-order order of addresses.

2. The volatile memory device as claimed in claim 1, wherein, The control logic circuit is further configured to: Receive a first data clock signal having a first frequency and a second data clock signal having a second frequency lower than the first frequency. In response to receiving the first read control command, the model data is read from the first model data to the second model data based on the first data clock signal, and In response to receiving the second read control command, the second data is read based on the second data clock signal.

3. The volatile memory device as claimed in claim 2, wherein, The volatile memory device also includes a voltage generator configured to generate a voltage sufficient to read data. The voltage generator is configured to generate a first read voltage with a first amplitude in response to receiving the first read control command, so as to perform a read operation based on the first data clock signal. In response to receiving the second read control command, a second read voltage with a second amplitude lower than the first amplitude is generated to perform a read operation based on the second data clock signal.

4. The volatile memory device as claimed in claim 1, wherein, The control logic circuit is configured to allocate a third space in the memory cell array in response to receiving a space allocation control command that instructs the control logic circuit to allocate space for storing model data.

5. The volatile memory device as claimed in claim 1, wherein, The memory cell array also includes a third space, and The control logic circuit is configured to, in response to receiving a first write control command instructing the control logic circuit to write the third model data to the fourth model data into the third space, write the third model data to the fourth model data into the third space according to the sequentially ordered addresses.

6. The volatile memory device as claimed in claim 5, wherein, The control logic circuit is configured to, in response to receiving a third read control command targeting the third space, read from the fifth model data stored at the second starting address in the third space to the sixth model data stored at the second ending address in the third space in address order.

7. The volatile memory device of claim 1, wherein, The first read control command includes the first start address and the first end address.

8. The volatile memory device of claim 1, wherein, The first starting address is the first address of the first space, and The first end address is the last address of the first space.

9. The volatile memory device as claimed in claim 1, wherein, The control logic circuitry includes a mode register configured to store information indicating the operating mode of the volatile memory device, and The mode register is configured to provide the memory controller with information indicating whether the volatile memory device is configured to read data in address order and out of address order in response to a corresponding read control command.

10. A memory controller, comprising: The processing circuitry is configured to control the operation of a volatile memory device, the volatile memory device including a first space allocated for storing a first model dataset and a second space allocated for storing second data; as well as A clock generator is configured to provide a clock signal to the volatile memory device for operation of the volatile memory device. The clock generator is configured to generate a first data clock signal having a first frequency and a second data clock signal having a second frequency lower than the first frequency. The processing circuitry is configured to, in response to receiving a first read command targeting the first model dataset, provide the following to the volatile memory device: A first read control command instructs the volatile memory device to read the first model dataset in address order. The first data clock signal, The first starting address is the first address of the first model dataset, and A first end address, which is the last address of the first model dataset, and wherein the processing circuitry is configured to provide the volatile memory device with the following in response to receiving a second read command targeting the second data: A second read control command instructs the volatile memory device to read the second data in non-sequential address order. The second data clock signal, and The address of the second data.

11. The memory controller of claim 10, wherein, The processing circuitry is also configured to provide a space allocation control command to the volatile memory device, the space allocation control command instructing the volatile memory device to allocate space for storing the second model dataset.

12. The memory controller of claim 10, wherein, The volatile memory device also includes a third space allocated for storing the second model dataset, and The processing circuitry is further configured to, in response to receiving a first write command instructing the memory controller to write the second model dataset, provide the volatile memory device with a first write control command instructing the volatile memory device to write the second model dataset into the third space according to the sequentially ordered addresses.

13. The memory controller of claim 12, wherein, The processing circuitry is configured to, in response to receiving a second read command instructing the memory controller to read the second model dataset, provide the following to the volatile memory device: The second read control command instructs the volatile memory device to read the second model dataset sequentially according to the address order. The first data clock signal, The second starting address is the first address of the second model dataset, and The second end address is the last address of the second model dataset.

14. A memory system, comprising: A memory cell array, comprising a first space and a second space; A volatile memory device, comprising control logic circuitry configured to control the array of memory cells; as well as A memory controller includes processing circuitry configured to control the operation of the volatile memory device, and the memory controller further includes a clock generator configured to provide a clock signal to the volatile memory device for the operation of the volatile memory device. The clock generator is configured to generate a first data clock signal having a first frequency and a second data clock signal having a second frequency lower than the first frequency. The control logic circuit is configured to, in response to receiving a first read control command targeting the first space, read at least some model data from a plurality of model data stored in the first space according to address order based on the first data clock signal, and The control logic circuit is configured to, in response to receiving a second read control command targeting second data stored in the second space, read the second data stored in the second space in a non-sequential order of addresses based on the second data clock signal.

15. The memory system of claim 14, wherein, The control logic circuit is configured to, in response to receiving a first read control command targeting the first space, read from the first model data stored at the first starting address in the first space to the second model data stored at the first ending address in the first space, in address order.

16. The memory system of claim 15, wherein, The first read control command includes the first start address and the first end address.

17. The memory system of claim 15, wherein, The first starting address is the first address of the first space, and The first end address is the last address of the first space.

18. The memory system of claim 14, wherein, The volatile memory device also includes a voltage generator configured to generate a voltage sufficient to read data. The voltage generator is configured to generate a first voltage based on the first data clock signal in response to receiving the first read control command, and to perform a read operation. The voltage generator is configured to generate a second voltage lower than the first voltage based on the second data clock signal in response to receiving the second read control command, so as to perform a read operation.

19. The memory system of claim 14, wherein, The control logic circuit is configured to allocate a third space in the memory cell array in response to receiving a space allocation control command from the memory controller, which instructs the volatile memory device to allocate space for storing model data.

20. The memory system of claim 14, wherein, The memory cell array also includes a third space, and The control logic circuit is configured to, in response to receiving a first write control command from the memory controller instructing the volatile memory device to write the third model data to the fourth model data into the third space, write the third model data to the fourth model data into the third space according to the sequentially ordered addresses.

Images