Memory device status push within the memory subsystem

Abstract

This application relates to a memory device status push within a memory subsystem. A local media controller of a first memory device receives a first cycle count broadcast by a second memory device via a bus connecting the first and second memory devices. The local media controller initializes a counter associated with the first memory device. In response to determining that the value of the counter matches the first cycle count, the local media controller transmits the status of the first memory device via the bus. Furthermore, in response to determining that the status is ready, the local media controller sends the status of the memory regions of the first memory device to a memory subsystem controller managing the first memory device.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to a memory subsystem, and more specifically, to pushing the state of memory devices within a memory subsystem. Background Technology

[0002] The memory subsystem may include one or more memory devices for storing data. These memory devices may be, for example, non-volatile memory devices and volatile memory devices. Generally, a host system can utilize the memory subsystem to store data at the memory devices and retrieve data from the memory devices. Summary of the Invention

[0003] This disclosure provides a method comprising: receiving, via a bus, a first cycle number broadcast by a second memory device from a local media controller of a first memory device, the bus connecting the first memory device and the second memory device; initializing a counter associated with the first memory device; in response to determining that the value of the counter matches the first cycle number, transmitting via the bus a state of the first memory device, wherein the state includes a second cycle number associated with the first memory device, wherein the second cycle number is determined based on the state of the first memory device; and in response to determining that the state of the first memory device is ready, sending a state of a memory region of the first memory device to a memory subsystem controller managing the first memory device.

[0004] Another aspect of this disclosure provides a system comprising: a plurality of memory devices; and a processing means operatively coupled to the plurality of memory devices to perform operations including: sending a synchronization clock command to the plurality of memory devices via a single bit serial bus during a power-on event; receiving a state of a first memory device from a local media controller of a first memory device among the plurality of memory devices via a sideband channel; and sending an input / output operation to the local media controller via an input / output channel in response to determining that the state of the first memory device is ready.

[0005] Another aspect of this disclosure provides a non-transitory computer-readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations including: receiving, via a bus, a first cycle number broadcast by a second memory device from a local media controller of a first memory device, the bus connecting the first memory device and the second memory device; initializing a counter associated with the first memory device; transmitting, via the bus, a state of the first memory device in response to determining that the value of the counter matches the first cycle number, wherein the state includes a second cycle number associated with the first memory device, wherein the second cycle number is determined based on the state of the first memory device; and sending the state of a memory region of the first memory device to a memory subsystem controller managing the first memory device in response to determining that the state of the first memory device is ready. Attached Figure Description

[0006] This disclosure will be more fully understood from the detailed description and the accompanying drawings of various embodiments of the present disclosure given below. However, the drawings should not be construed as limiting this disclosure to the specific embodiments, but are for explanation and understanding only.

[0007] Figure 1A An example computing system including a memory subsystem is shown according to some embodiments of the present disclosure.

[0008] Figure 1B More detailed embodiments according to this disclosure are shown. Figure 1A An example computing system with a memory subsystem.

[0009] Figure 2 Examples of control timing for a token ring cyclic architecture for bus arbitration in multiple memory devices according to some embodiments of the present disclosure are shown.

[0010] Figure 3 An example protocol for transferring the state of an associated memory device according to some embodiments of this disclosure is shown.

[0011] Figure 4 This is a flowchart of an example method for implementing state push arbitration according to some embodiments of this disclosure.

[0012] Figure 5 This is a flowchart of an example method for receiving state information via a sideband channel according to some embodiments of the present disclosure.

[0013] Figure 6 This is a block diagram of an example computer system in which embodiments of the present disclosure may operate. Detailed Implementation

[0014] Various aspects of this disclosure relate to pushing the status of memory devices within a memory subsystem. The memory subsystem may be a storage device, a memory module, or a combination of both. Examples of storage devices and memory modules are described below with reference to Figure 1. Generally, a host system may utilize a memory subsystem comprising one or more components (e.g., a memory device for storing data). The host system can provide data to be stored in the memory subsystem and can request data to be retrieved from the memory subsystem.

[0015] The memory subsystem may contain high-density non-volatile memory devices, in which data needs to be retained when no power is supplied to the memory devices. An example of a non-volatile memory device is a NAND flash memory device. The following section combines... Figure 1A Other examples of non-volatile memory devices are described. A non-volatile memory device is a package of one or more dies. Each die may contain one or more planes. For some types of non-volatile memory devices (e.g., NAND devices), each plane contains a set of physical blocks. Each block contains a set of pages. Each page contains a set of memory cells (“cells”). A cell is an electronic circuit that stores information. Depending on the cell type, a cell may store one or more bits of binary information and has various logical states associated with the number of bits being stored. Logical states may be represented as binary values, such as “0” and “1”, or combinations of such values.

[0016] Memory access operations can be performed by the memory subsystem. These operations can be initiated by the host or by the memory subsystem controller. For example, the host system can initiate memory access operations (e.g., write, read, erase, etc.) on the memory subsystem. The host system can send memory access commands (e.g., write commands, read commands) to the memory subsystem to store data on and read data from memory devices within the memory subsystem. The data to be read or written, as specified by a host request, is referred to hereinafter as "host data". The host request may contain logical address information (e.g., logical block address (LBA), namespace) of the host data, which is the location associated between the host system and the host data. The logical address information (e.g., LBA, namespace) may be part of the metadata of the host data. The metadata may also include error handling data (e.g., ECC codeword, parity check code), data version (e.g., expiration date for distinguishing written data), validity bitmap (whose LBA or logical transfer unit contains valid data), etc. Memory access operations initiated by the memory subsystem controller may involve maintenance operations, such as garbage collection, wear leveling, bad block management, and block refresh operations.

[0017] Non-volatile memory devices can comprise several individual dies. The memory subsystem controller can include several input / output (I / O) ports and channels, which the memory subsystem controller can use to communicate with the individual dies. For example, there can be eight communication channels between the memory subsystem controller and the non-volatile memory device, where each channel can be enabled using a separate chip enable (CE) signal. Each communication channel can support a certain number of memory dies. For example, there can be 16 memory dies accessible via each channel. Each individual memory die can be configured as a separate logical unit (LU) identified by a unique logical unit number (LUN). Therefore, a system with eight communication channels and 16 LUs per channel can contain 128 individual LUs.

[0018] During certain operations, the memory subsystem controller typically checks the status of various LUs within the memory subsystem. For example, after submitting a read command to the memory device, the memory subsystem controller may repeatedly poll the memory device to check if the requested I / O data is ready to be read. In some systems, the memory controller sends a status polling command to a single LU. In response, the LU returns the status of one or more parameters of the LU and / or the memory die. For example, one bit of the status indicates whether the I / O data is ready to be read, and the remaining bits are associated with other states. Such status polling operations can be performed periodically on a given LU. In memory subsystems with a large number of LUs (e.g., 128 LUs) and under heavy workloads, the number and frequency of these status polling operations performed on individual LUs introduce considerable system overhead and consume significant bandwidth in the communication channel between the memory subsystem controller and the memory device within the memory subsystem.

[0019] This disclosure addresses the aforementioned drawbacks and others by adding a status push sideband channel between the memory subsystem controller and the memory device, thereby decoupling status polling commands from the I / O channel. The sideband channel is separated from the primary communication channel between the memory subsystem controller and the memory device. In an embodiment, a single-bit serial bus is added between the memory subsystem controller and the memory device. The bus implements a sideband channel for which the memory device takes turns acting as the master, and the memory subsystem controller as the slave. A token ring architecture can be used to enable arbitration of the new bus. When the memory device controller holds a token, it can transmit the state of the memory device to the memory subsystem controller. Upon receiving the state of the memory device, the memory subsystem controller can act accordingly. That is, if the memory device is busy, the memory subsystem controller can wait to perform I / O operations. However, if the memory device is ready, the memory subsystem controller can use the existing I / O channel to perform I / O operations. Therefore, the memory subsystem controller can avoid using the I / O channel for status polling commands.

[0020] In some embodiments, a separate channel is added between the memory subsystem controller and the memory devices to transmit clock synchronization. For example, clock synchronization can be transmitted during power-on events or at set time intervals (e.g., once a day). The clock can be used to implement token ring architecture bus arbitration, which provides each memory device within the memory subsystem with an equal opportunity to transmit status information to the memory subsystem controller.

[0021] The advantages of this disclosure include, but are not limited to, eliminating the burden of polling memory devices from the memory subsystem controller, thereby reducing power consumption associated with sending status polling commands. By decoupling the I / O channels from status polling commands, the I / O channels can be used for I / O operations where the memory device has transmitted a ready state, thereby increasing the rate at which the memory subsystem controller performs operations (e.g., input / output operations (IOPs) per second). This can potentially lead to improved performance of the memory subsystem and reduced latency associated with I / O operations within the memory subsystem.

[0022] Figure 1A An example computing system 100 including a memory subsystem 110 is illustrated according to some embodiments of the present disclosure. The memory subsystem 110 may include media, such as one or more volatile memory devices (e.g., memory device 140), one or more non-volatile memory devices (e.g., memory device 130), or a combination thereof.

[0023] The memory subsystem 110 may be a storage device, a memory module, or a combination of a storage device and a memory module. Examples of storage devices include solid-state drives (SSDs), flash drives, universal serial bus (USB) flash drives, embedded multimedia controller (eMMC) drives, universal flash memory (UFS) drives, secure digital cards (SD cards), and hard disk drives (HDDs). Examples of memory modules include dual in-line memory modules (DIMMs), small form factor DIMMs (SO-DIMMs), and various types of non-volatile dual in-line memory modules (NVDIMMs).

[0024] The computing system 100 may be a computing device, such as a desktop computer, laptop computer, web server, mobile device, vehicle (e.g., an airplane, drone, train, car or other means of transport), Internet of Things (IoT) enabled device, embedded computer (e.g., an embedded computer contained in a vehicle, industrial equipment or networked business device), or such computing device containing memory and processing power.

[0025] The computing system 100 may include a host system 120 coupled to one or more memory subsystems 110. In some embodiments, the host system 120 is coupled to multiple memory subsystems 110 of different types. Figure 1A An example of a host system 120 coupled to a memory subsystem 110 is shown. As used herein, “coupled to” or “coupled with” generally refers to a connection between components, which may be an indirect communication connection or a direct communication connection (e.g., without an intermediate component), whether wired or wireless, and includes connections such as electrical, optical, magnetic, etc.

[0026] Host system 120 may include a processor chipset and a software stack executed by the processor chipset. The processor chipset may include one or more cores, one or more cache memories, a memory controller (e.g., an NVDIMM controller), and a storage protocol controller (e.g., a PCIe controller, a SATA controller). Host system 120 uses memory subsystem 110, for example, to write data to memory subsystem 110 and read data from memory subsystem 110.

[0027] Host system 120 can be coupled to memory subsystem 110 via a physical host interface. Examples of physical host interfaces include, but are not limited to, Serial Advanced Technology Attachment (SATA) interfaces, Peripheral Component Interconnect High Speed ​​(PCIe) interfaces, Universal Serial Bus (USB) interfaces, Fibre Channel, Serial Attached SCSI (SAS), Dual Data Rate (DDR) memory bus, Small Computer System Interface (SCSI), Dual In-line Memory Module (DIMM) interfaces (e.g., DIMM sockets supporting Dual Data Rate (DDR)), etc. The physical host interface can be used to transfer data between host system 120 and memory subsystem 110. When memory subsystem 110 is coupled to host system 120 via a physical host interface (e.g., a PCIe bus), host system 120 can further utilize an NVM High Speed ​​(NVMe) interface to access components (e.g., memory device 130). The physical host interface can provide an interface for transferring control, address, data, and other signals between memory subsystem 110 and host system 120. Figure 1A Memory subsystem 110 is shown as an example. Generally, host system 120 can access multiple memory subsystems via the same communication connection, multiple separate communication connections, and / or combinations of communication connections.

[0028] Memory devices 130 and 140 may comprise any combination of different types of non-volatile memory devices and / or volatile memory devices. Volatile memory devices (e.g., memory device 140) may be, but are not limited to, random access memory (RAM), such as dynamic random access memory (DRAM) and synchronous dynamic random access memory (SDRAM).

[0029] Some examples of non-volatile memory devices (e.g., memory device 130) include NAND flash memory and in-place write memory, such as three-dimensional crosspoint (“3D crosspoint”) memory devices, which are crosspoint arrays of non-volatile memory cells. Crosspoint arrays of non-volatile memory cells can perform bit storage based on variations in volume resistance in conjunction with stackable cross-grid data access arrays. Furthermore, compared to many flash-based memories, crosspoint non-volatile memories can perform in-place write operations, where non-volatile memory cells can be programmed without prior erasing. NAND flash memories include, for example, two-dimensional NAND (2D NAND) and three-dimensional NAND (3D NAND).

[0030] Each of the memory devices 130 may include one or more arrays of memory cells. One type of memory cell, such as a single-level cell (SLC), may store one bit per cell. Other types of memory cells, such as multi-level cell (MLC), three-level cell (TLC), four-level cell (QLC), and five-level cell (PLC), may store multiple bits per cell. In some embodiments, each of the memory devices 130 may include one or more arrays of memory cells, such as SLC, MLC, TLC, QLC, PLC, or any combination thereof. In some embodiments, a particular memory device may include an SLC portion of memory cells, as well as an MLC portion, a TLC portion, a QLC portion, or a PLC portion. The memory cells of the memory device 130 may be grouped into pages that refer to logical units of the memory device used for storing data. For some types of memory (e.g., NAND), pages may be grouped to form blocks.

[0031] Although non-volatile memory components, such as 3D cross-point arrays of non-volatile memory cells and NAND flash memory (e.g., 2D NAND, 3D NAND), are described, memory device 130 may be based on any other type of non-volatile memory, such as read-only memory (ROM), phase-change memory (PCM), auto-select memory, other chalcogenide-based memory, ferroelectric transistor random access memory (FeTRAM), ferroelectric random access memory (FeRAM), magnetic random access memory (MRAM), spin-transfer torque (STT)-MRAM, conductive bridged RAM (CBRAM), resistive random access memory (RRAM), oxide-based RRAM (OxRAM), NOR flash memory, or electrically erasable programmable read-only memory (EEPROM).

[0032] The memory subsystem controller 115 (or, for simplicity, controller 115) can communicate with the memory device 130 to perform operations such as reading data, writing data, or erasing data at the memory device 130, and other such operations. The memory subsystem controller 115 may include hardware such as one or more integrated circuits and / or discrete components, buffer memories, or combinations thereof. The hardware may include a digital circuit system having dedicated (i.e., hard-decoded) logic for performing the operations described herein. The memory subsystem controller 115 may be a microcontroller, a dedicated logic circuit system (e.g., a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc.), or other suitable processor.

[0033] The memory subsystem controller 115 may include a processing means comprising one or more processors (e.g., processor 117) configured to execute instructions stored in local memory 119. In the illustrated example, the local memory 119 of the memory subsystem controller 115 includes embedded memory configured to store instructions for performing various processes, operations, logical flows, and routines that control the operation of the memory subsystem 110, including handling communication between the memory subsystem 110 and the host system 120.

[0034] In some embodiments, local memory 119 may include memory registers storing memory pointers, retrieved data, etc. Local memory 119 may also include read-only memory (ROM) for storing microcode. Although Figure 1A The example memory subsystem 110 has been shown to include a memory subsystem controller 115, but in another embodiment of this disclosure, the memory subsystem 110 does not include a memory subsystem controller 115 and may instead rely on external control (e.g., provided by an external host or by a processor or controller separate from the memory subsystem).

[0035] Generally, the memory subsystem controller 115 can receive commands or operations from the host system 120 and can translate these commands or operations into instructions or appropriate commands to achieve the desired access to the memory device 130. The memory subsystem controller 115 may be responsible for other operations, such as wear leveling, garbage collection, error detection and error correction (ECC) operations, encryption, caching, and address translation between logical addresses (e.g., logical block addresses, namespaces) and physical addresses (e.g., physical block addresses) associated with the memory device 130. The memory subsystem controller 115 may further include a host interface circuitry for communicating with the host system 120 via a physical host interface. The host interface circuitry can translate commands received from the host system into command instructions to access the memory device 130 and translate responses associated with the memory device 130 into information for the host system 120.

[0036] The memory subsystem 110 may also include additional circuitry or components not shown. In some embodiments, the memory subsystem 110 may include a cache memory or buffer (e.g., DRAM) and an address circuitry (e.g., a row decoder and a column decoder) that can receive and decode addresses from the memory subsystem controller 115 to access the memory device 130.

[0037] In some embodiments, memory device 130 includes a local media controller 135 that operates in conjunction with memory subsystem controller 115 to perform operations on one or more memory cells of memory device 130. An external controller (e.g., memory subsystem controller 115) may externally manage memory device 130 (e.g., perform media management operations on memory device 130). In some embodiments, memory subsystem 110 is a managed memory device, which is the original memory device 130 having on-die control logic (e.g., local media controller 135) and a controller (e.g., memory subsystem controller 115) for media management within the same memory device package. An example of a managed memory device is a managed NAND (MNAND) device.

[0038] The memory subsystem 110 includes a memory interface component 113 capable of receiving status information from a memory device via a sideband channel. In some embodiments, the memory subsystem controller 115 includes at least a portion of the memory interface component 113. In some embodiments, the memory interface component 113 is part of the host system 110, an application, or an operating system. In other embodiments, the local media controller 135 includes at least a portion of the memory interface component 113 and is configured to perform the functionality described herein.

[0039] In some embodiments, memory device 130 includes a status push component 150 that can transmit the state of the memory device to a memory subsystem controller via a sideband channel. In one embodiment, a local media controller 135 of memory device 130 includes at least a portion of the status push component 150. In this embodiment, the status push component 150 can be implemented using hardware or firmware stored on memory device 130, which is executed by control logic (e.g., local media controller 135) to perform operations related to power budget arbitration for a plurality of concurrent access operations described herein. In some embodiments, memory subsystem controller 115 includes at least a portion of the status push component 150. For example, memory subsystem controller 115 may include a processor 117 (e.g., a processing device) configured to execute instructions stored in local memory 119 for performing the operations described herein.

[0040] In some embodiments, the memory subsystem 110 employs a token-based protocol, wherein tokens are rotated among multiple push state components 150 (e.g., in a cyclic manner, after a number of cycles of a shared clock signal). When a push state component 150 holds a token, it may transmit the state of the associated memory device 130 and the state of one or more regions within the memory device 130. The push state component 150 may transmit the state to other push state components of other memory devices in the memory subsystem 110. The transmitted state may indicate to other push state components in the memory subsystem 110 the number of bits that the memory device wishes to send to the memory subsystem controller 115, thereby indicating the amount of time the state push component wishes to hold the token (e.g., the number of cycles of a shared clock signal).

[0041] In some embodiments, the memory interface component 113 can transmit a synchronization clock command to each state push component. Therefore, each state push component 150 can track cycle counts, and each state push component 150 can predict when it will receive a token based on the number of bits transmitted by the previous state push component in the cyclic ring architecture arbitration. Regarding Figure 2 and 3 Further describe the timing of the status push cycle.

[0042] Memory interface component 113 may receive status information from status push component 150. In some embodiments, the status information may indicate whether an associated memory device is busy or ready, whether a region within the associated memory device is busy or ready, and / or whether certain operations performed within said region have succeeded / failed. If the received status information indicates that memory device 130 is ready, memory interface component 113 may perform input / output operations with memory device 130. However, if the received status information indicates that memory device 130 is busy, memory interface component 113 may decide not to perform I / O operations with memory device 130, thereby opening the I / O channel for I / O operations with respect to other memory devices within memory subsystem 110.

[0043] In some embodiments, the status information received from the status push component 150 may include additional information, such as cell distribution information, soft decision information, outliers in performance metrics, thermal alarms, interrupts initiated by the memory device, etc. The status push component 150 can determine which status information to include in the protocol without assistance from the memory subsystem controller 115. In some embodiments, if the memory device 130 is busy, the push status component 150 of the memory device 130 may send an indication that it is busy and may not include any other information in the transmission. Alternatively, if the memory device 130 is ready, it may include additional status information in the transmission.

[0044] Further details regarding the operation of the memory interface component 113 and the status push component 150 are described below.

[0045] Figure 1B More detailed embodiments according to this disclosure are shown. Figure 1A This is an example computing system with a memory subsystem 110. In some embodiments, the memory subsystem 110 may have multiple memory devices 130A-N. In one embodiment, the memory devices 130A-N may share a clock signal received via a clock signal line bus 180. After a power-on event, and / or at certain time intervals (e.g., once a day, or once an hour), the memory interface component 113 may transmit a synchronization clock signal CLK to the memory devices 130A-N via the bus 180. The status push component 150A-N of the memory devices 130A-N may receive the synchronization clock signal and maintain clock cycle counting. The bus 180 may be a serial single-bit bus for clock synchronization between the memory subsystem controller 115 and the memory devices 130A-N.

[0046] In some embodiments, memory devices 130A-N can be selectively enabled in response to a chip enable signal (e.g., via a control link) and can communicate via a separate single-bit serial bus 182. Memory devices 130A-N can take turns acting as the master of bus 182, as determined by a token ring architecture bus arbitration scheme. Memory subsystem controller 115 can be a slave device of bus 182. In one embodiment, each of memory devices 130A-N includes an example of a status push component 150 that receives both a clock signal CLK and a token. In an embodiment, two additional pins are mounted on memory subsystem controller 115 and each memory device 130A-N to enable buses 180 and 182.

[0047] In one embodiment, a token-based ring architecture protocol is implemented, wherein tokens are circulated through memory devices 130A-N to transmit the state of associated memory devices 130A-N via bus 182. The period during which a given state push component 150A-N holds a token can be referred to as a state cycle of the associated memory device. The length of each state cycle can be variable, depending on the state of the associated memory device. At the end of a state cycle, the token is sequentially passed to the next memory device. Finally, the token is received again by the same state push component 150, which signals the start of a new state cycle for the associated memory device. Regarding... Figure 2 Further describe the length of the state cycle, and regarding Figure 3 Further describe the status push protocol.

[0048] Status push components 150A-N can track the status of associated memory devices 130A-130N. In some embodiments, the status of associated memory devices 130A-N can be busy or ready. In some embodiments, the status of memory devices 130A-N can include additional status information, such as read calibration data, performance metrics, soft decision information, thermal alarms, and / or interrupts initiated by the memory devices. Local memory media controllers 135A-N for each memory device 130A-N can store certain status metrics, and status push components 150A-N can determine which status metrics will be included in the status information of the memory devices 130A-N. For example, status push components 150A can determine that the transmission of status information for memory devices 130A includes status metrics that exceed certain thresholds, and / or includes outlier status metrics. In other embodiments, status push components 150A can include all stored status metrics. For example, status push components 150A can include all thermal alarms associated with memory devices 130A, and can include only performance metrics that are outliers or above / below certain thresholds. The status push component 150A can transmit the status of associated memory device 130A to the memory subsystem controller 115 and other memory devices 130B-N via bus 182.

[0049] Since the information on bus 182 is transparent to all memory devices 130A-130N, status push components 150A-150N can continuously monitor bus 182 to obtain token signals (which may include status updates) from other status push components 150A-150N. Furthermore, each status push component 150A-150N can maintain a clock synchronized with the clocks of other status push components 150A-150N. Using the synchronized clock and status information, the status push component 150A-150N can generate a token signal for bus 182. Each status push component 150A-150N can use the synchronized clock and information from the token signals monitored from bus 182 to determine when a token will be allocated to the associated memory device 130A-130N. About Figure 2 The state cycle and token allocation process are further described. When the state push component 150A of the memory device 130A stores the token, it can transmit the state associated with the memory device 130A to the memory interface component 113 of the memory subsystem controller 115.

[0050] The memory interface component 113 of the memory subsystem controller 115 can receive status information transmitted from the status push components 150A-N and act accordingly. Therefore, if the status push component 150A indicates that the memory device 130A is busy, the memory subsystem controller 115 can avoid sending any operation to the memory device 130A. However, if the status push component 150A sends a ready status to the memory interface component 113, the memory subsystem controller 115 can determine to send an operation (e.g., an I / O operation) to the memory device 130A. The memory subsystem controller 115 can send the operation via an existing communication channel (not shown). The memory subsystem controller 115 can also receive additional status information from the status push component 150A and act accordingly. For example, if the status push component 150A contains a thermal alarm, the memory subsystem controller 115 can perform appropriate operations in response to the thermal alarm. As another example, if the status push component 150A contains read calibration data above a certain threshold, the memory subsystem controller 115 can perform specific memory management operations.

[0051] Figure 2 Examples of control timing for a token ring loop architecture for bus arbitration among multiple memory devices 201-205 according to some embodiments of the present disclosure are shown. In some embodiments, the token ring loop architecture can be arbitrated among multiple dies in a multi-die package.

[0052] CLK 220 illustrates the clock signal. In some embodiments, Figure 1A -B's memory subsystem controller 115 can, for example, via... Figure 1B The bus 180 shown continuously transmits a synchronization clock signal to each memory device 201-205. In some embodiments, the CLK 220 signal can be synchronized (or resynchronized) among all memory devices 201-205 at 223, for example, on a power-on event, at fixed time intervals (e.g., once a day, or once an hour), and / or at another predefined time interval. Synchronization (or resynchronization) of the CLK 220 signal among all memory devices 201-205 can be performed to clear all in-process tokens. After synchronization (or resynchronization) of the CLK 220 signal, a token can be initialized to memory device 0201. Synchronization at operation 223 can occur on each power-on event. Additionally, resynchronization of the CLK 220 signal can be performed in response to… Figure 1A -B's memory subsystem controller 115 detects an error in the token ring protocol. Alternatively, resynchronization of the CLK 220 signal can occur periodically (e.g., at fixed time intervals) to prevent potential errors in token ring control.

[0053] Each memory device 201-205 can store a token for a certain number of cycles of a continuous synchronization clock. The time period during which a memory device stores a token can be referred to as the state cycle of the associated memory device. Cycle 210 shows the state cycle of memory device 0 201, cycle 213 shows the state cycle of memory device 1 203, and state cycle 215 shows the state cycle of memory device N 205. Cycle 230 shows the delay experienced by memory device 0 201, i.e., the amount of time it takes for the token to cycle through each memory device and back to memory device 0 201.

[0054] During state cycle 210, memory device 0 201 is the bus that arbitrates between memory devices 201-205 (e.g., Figure 1B The associated memory device 1 203 is the master device of the bus (182). During state cycle 213, the associated memory device 1 203 is the master device of the bus. During state cycle 215, the associated memory device N 205 is the master device of the bus. Furthermore, during state cycle 210, the associated memory device 0 201 can transmit status information via the bus to other memory devices 203-205 and the memory subsystem controller. The memory subsystem controller is a slave device of the bus. Memory devices 203-205 can receive the transmitted status information from memory device 0 201 and can determine the length of state cycle 210 based on the status information. (See also: Regarding...) Figure 3 Further described, if memory device 0 201 transmits a busy state, then state period 210 is of minimum length, during which memory device 0 201 does not transmit additional state information. In some embodiments, if memory device 0 201 transmits a signal other than ready (e.g., busy, standby, idle, etc.), then state period 210 is of minimum length. Therefore, memory device 1 203 (i.e., the next memory device in the sequential ring architecture) can determine that it will receive the token within a minimum amount of time (e.g., two clicks on a shared clock). On the other hand, if memory device 0 201 transmits a ready state (e.g., if memory device 0 201 is transitioning from busy to ready), then state period 210 can be longer, depending on the amount of state information pushed by memory device 0 201. Therefore, each cycle 210-215 can be variable, depending on the amount of data transmitted by each memory device. The length of each state period depends on the state information data transmitted by the memory device, and regarding... Figure 3 Further description is required.

[0055] Figure 3This is an example protocol for transferring the state of an associated memory device according to some embodiments of this disclosure. It should be noted that this is an example of a protocol that can be used to implement the token ring architecture described herein. Other protocols not shown herein can be used to implement the token ring architecture described herein. In some embodiments, Figure 1A -B status push component 150 use Figure 3 The protocol is used to transmit the status of the associated memory device. In some embodiments, the protocol begins with a cycle count 301 field. The cycle count 301 field contains a 1-bit notifier 319. In some embodiments, if the associated memory device is busy, the notifier 319 may be set to "1", in which case no other data is transmitted. When the memory device transmits its busy status by setting the notifier 319 to "1", the memory device's status cycle is shorter because it only transmits 1 bit of the notifier 319. In some embodiments, when the memory device has a status other than ready, such as an idle or standby state, the memory device sets the notifier 319 to "1". If the associated die is ready, the notifier 319 may be set to "0". The last 6 bits of the cycle count 301 field can indicate the amount of data to be transmitted, shown as 317.

[0056] The bits following the 1-bit notification 319 included in cycle count 301 can be transmitted to other memory devices in the token ring architecture, thereby enabling variable state cycles for each memory device. For example, the transmission of notification 319, which is set to "1" by memory device 0, can indicate to the next memory device in the architecture that it will acquire a token with a minimum number of clock signals (e.g., 2 clock signals). As another example, the transmission of notification 319, which is set to "0" indicating the number of data bits to be transmitted, can indicate to the next memory device in the architecture that it will acquire a token with the number of clock signals required to transmit the number of data bits.

[0057] In an embodiment, die information 303 includes a die identification number associated with the memory device. In some embodiments, the status push component can determine the status of each plane within the die and can transmit information about the status of each plane of the associated die. For example, the protocol may include a plane number 305 field, a pass / fail (P / F) 307 field, and / or a ready / busy (R / B) 309 field. For example, the P / F 307 field may indicate whether the operation associated with the plane has passed or failed. The R / B 309 field may indicate whether the plane is ready or busy. These data fields may be repeated for each plane in the die (e.g., in the next plane information 311 field).

[0058] As the token is passed sequentially to the next memory device, the state push component of the memory device that stores the token can add its die state information to the protocol, for example, in the next die information 313.

[0059] Figure 4 This is a flowchart of an example method 400 for implementing state push arbitration according to some embodiments of this disclosure. Method 400 can be executed by processing logic, which may include hardware (e.g., processing device, circuit system, dedicated logic, programmable logic, microcode, device hardware, integrated circuit, etc.), software (e.g., instructions that run or execute on the processing device), or a combination thereof. In some embodiments, by Figure 1A-1B The status push component 150 executes method 400. Although shown in a specific order or sequence, the order of processes can be modified unless otherwise specified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes can be executed in different orders, and some processes can be executed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are also possible.

[0060] At operation 410, the processing logic receives a first cycle count (i.e., cycle count) broadcast by the second memory device via a bus connecting the first and second memory devices. The bus may connect memory devices in a token ring architecture that arbitrates sideband channels used for transmitting state information of memory devices within the architecture. The second memory device may be a memory device preceding the first memory device in the ring architecture. The first cycle count may indicate the length of time the second memory device will hold the token. Therefore, the processing logic can use the first cycle count to determine when to receive the token.

[0061] At operation 420, the processing logic may initialize a counter associated with the first memory device. In an embodiment, the processing logic maintains a counter synchronized with counters of other memory devices in the token ring architecture (e.g., a second counter associated with a second memory device). Initializing the counter may involve marking a time on the counter and counting the number of cycles from that time to identify when a token will be received.

[0062] In some embodiments, the processing logic may receive a synchronization clock command via a single-bit serial bus during a power-on event. Alternatively, the processing logic may receive the synchronization clock command at fixed time intervals, and / or in response to determining an error within the token ring architecture. In an embodiment, in response to receiving a synchronization clock command, the processing logic may determine that a token has been allocated to a first memory device. That is, the number of cycles (i.e., cycle count) associated with the previous memory device (i.e., the second memory device) in the token ring architecture has elapsed, and the first memory device now holds the token. When the processing logic holds the token, it acts as the master of the bus arbitrating among the multiple memory devices, and therefore the processing logic can transmit its status information.

[0063] At operation 430, in response to determining that the value of the counter matches the first cycle number, the processing logic transmits the state of the first memory device via the bus. In some embodiments, the first memory device maintains its state, and at operation 430, the processing logic identifies the state of the first memory device. The state of the first memory device may include whether the memory device is busy or ready. In some embodiments, the state of the first memory device may further include the ready / busy state of a memory region within the first memory device (e.g., the ready / busy state of a plane within the memory die), and / or the pass / fail state of a memory region within the first memory device (e.g., the pass / fail state of a plane within the memory die). In some embodiments, the state information of the first memory device may also include performance metrics, thermal alarms, interrupts, read calibration data, soft decision (SD) information, or any other data that the memory device can determine about itself.

[0064] As mentioned above, at operation 430, the processing logic transmits the state of the first memory device via the bus. The state of the first memory device may include a second number of cycles associated with the first memory device. In some embodiments, in response to determining that the state of the first memory device is busy, the processing logic may determine that the second number of cycles is a minimum number of cycles. For example, the minimum number of cycles may be 2, in which case, if the first memory device is busy, the processing logic may update the token to include 2 cycles associated with the first memory device. This instructs other memory devices in the token ring architecture that the first memory device will hold the token for 2 cycles of continuously sharing the synchronization clock. When a memory device transitions from busy to ready, the number of cycles may exceed the minimum number of cycles; otherwise, the number of cycles is the minimum number of cycles.

[0065] If the first memory device is in a ready state, the processing logic can further determine the state of a memory region within the memory device (ready / busy, and / or pass / fail). The number of second cycles can be determined based on this additional state information. That is, the processing logic can determine the number of bits required to transmit the ready / busy state and / or pass / fail state of the memory region. The number of second cycles can represent the number of cycles required to transmit additional data. For example, it might take one cycle to transmit 8 bits of data, in which case transmitting state information containing 24 bits of data (e.g., where the state information indicates the ready / busy state and pass / fail state of a memory plane within the first memory device) might take 3 cycles. In some embodiments, the processing logic can further include additional information in the state of the first memory device (e.g., outliers in performance metrics, soft decision information, thermal alarms, interrupts, or read calibration data), and the number of second cycles is determined based on the additional information included in the state information of the first memory device.

[0066] In some embodiments, the state of the first memory device is sent to the memory subsystem controller via a sideband channel. In some embodiments, additional pins are added to the memory subsystem controller and to the first and second memory devices to enable the sideband channel and to enable a single-bit serial bus for sending synchronous clock commands.

[0067] In an embodiment, in response to sending a ready state to the memory subsystem controller, the processing logic receives input / output (I / O) operations from the memory subsystem controller via an input / output channel.

[0068] At operation 440, in response to determining that the state of the first memory device is ready, the processing logic sends the state of the memory region of the first memory device to the memory subsystem controller managing the first memory device. The state may include at least one of the following: a ready / busy state of a memory region within the first memory device, or a pass / fail state of a memory region within the first memory device. In some embodiments, the processing logic may further send at least one of the following: an outlier in a performance metric, soft decision information, a thermal alarm, an interrupt, or read calibration data, and wherein the number of second cycles is determined based on the outlier in the performance metric, soft decision information, thermal alarm, interrupt, or read calibration data.

[0069] Figure 5This is a flowchart of an example method 500 for receiving state information from a sideband channel according to some embodiments of this disclosure. Method 500 can be executed by processing logic, which may include hardware (e.g., processing device, circuit system, dedicated logic, programmable logic, microcode, device hardware, integrated circuit, etc.), software (e.g., instructions that run or execute on the processing device), or a combination thereof. In some embodiments, by Figure 1A-1B The memory interface component 113 executes method 500. Although shown in a specific order or sequence, the order of processes may be modified unless otherwise specified. Therefore, the illustrated embodiments should be understood as examples only, and the illustrated processes may be executed in different orders, and some processes may be executed in parallel. In addition, one or more processes may be omitted in various embodiments. Therefore, not all processes are required in every embodiment. Other process flows are also possible.

[0070] At operation 510, the processing logic sends a synchronization clock command to multiple memory devices via a single-bit serial bus during a power-on event. In an embodiment, the processing logic may send the synchronization clock command at predefined time intervals (e.g., once a day, once an hour, or at another time interval), and / or send the synchronization clock command in response to receiving an indication of an error from one of the multiple memory devices. That is, if one of the memory devices detects an error within the token ring architecture, the memory device may transmit the error to the processing logic. The processing logic may then reset the clocks of the multiple memory devices by sending the synchronization clock command.

[0071] At operation 520, the processing logic receives the status of a first memory device from the local media controller of a first memory device among a plurality of memory devices via a sideband channel. The status of the first memory device can be ready or busy. In some embodiments, the processing logic receives status information of the first memory device, such as the ready / busy status of a memory region within the first memory device (e.g., a plane within the die of the first memory device), and the pass / fail status of a region within the first memory device. In some embodiments, the processing logic receives interrupts, thermal alarms, read calibration data, soft decision information, and / or performance anomaly data from the first memory device. The processing logic can process the received information and react accordingly.

[0072] At operation 530, in response to determining that the state of the first memory device is ready, the processing logic sends an input / output operation to the local media controller via the input / output channel. By waiting for the ready status from the memory device via the sideband channel, the processing logic uses the I / O channel for I / O operations and no longer needs to use the I / O channel for status polling commands.

[0073] In some embodiments, the local media controller of the first memory device is the master controller of the sideband channel, and the memory subsystem controller is the master controller of the input / output channel. In some embodiments, additional pins are added to the memory subsystem controller and to the first and second memory devices to enable the sideband channel and enable a single-bit serial bus for sending synchronous clock commands.

[0074] Figure 6 An example machine of computer system 600 is shown, within which an instruction set is executable to cause the machine to perform any or more of the methods discussed herein. In some embodiments, computer system 600 may correspond to a host system (e.g., Figure 1A The host system 120 includes, is coupled to, or utilizes a memory subsystem (e.g., Figure 1A The memory subsystem 110), or may be used to perform controller operations (e.g., to execute an operating system, thereby executing commands corresponding to...). Figure 1A (Operation of the memory interface component 113 and / or the status push component 150). In an alternative embodiment, the machine may be connected (e.g., networked) to other machines in a LAN, intranet, extranet, and / or the Internet. The machine may operate as a peer machine in a peer-to-peer (or distributed) network environment or as a server or client machine in a client-server network environment as a server or client machine in a cloud computing infrastructure or environment.

[0075] The machine may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), cellular phone, network appliance, server, network router, switch, or bridge, or any machine capable of executing (sequentially or otherwise) a set of instructions specifying actions to be taken by the machine. Furthermore, although a single machine is shown, the term "machine" should also be considered to include any set of machines that individually or collectively execute a set of instructions (or multiple sets of instructions) to perform any or more of the methods discussed herein.

[0076] Example computer system 600 includes processing devices 602 that communicate with each other via bus 630, main memory 604 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM), such as synchronous DRAM (SDRAM) or RDRAM), static memory 606 (e.g., flash memory, static random access memory (SRAM), etc.), and data storage system 618.

[0077] Processing device 602 represents one or more general-purpose processing devices, such as microprocessors, central processing units, etc. More specifically, the processing device may be a Complex Instruction Set Computing (CISC) microprocessor, a Reduced Instruction Set Computing (RISC) microprocessor, a Very Long Instruction Word (VLIW) microprocessor, or a processor implementing other instruction sets or a combination of instruction sets. Processing device 602 may also be one or more special-purpose processing devices, such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), network processors, etc. Processing device 602 is configured to execute instructions 626 for performing the operations and steps discussed herein. Computer system 600 may further include a network interface device 608 communicating via network 620.

[0078] Data storage system 618 may include machine-readable storage medium 624 (also referred to as computer-readable medium) on which one or more instruction sets 626 or software embodying any one or more of the methods or functions described herein are stored. Instructions 626 may also reside wholly or at least partially within main memory 604 and / or processing device 602 during execution by computer system 600, which also constitute machine-readable storage medium. Machine-readable storage medium 624, data storage system 618, and / or main memory 604 may correspond to... Figure 1A The memory subsystem 110.

[0079] In one embodiment, instruction 626 includes instructions for implementing a memory interface component and / or a status push component (e.g., Figure 1A The memory interface component 113 and / or status push component 150) contain functional instructions. Although the machine-readable storage medium 624 is shown as a single medium in the example embodiment, the term "machine-readable storage medium" should be considered to include a single medium or multiple media storing one or more sets of instructions. The term "machine-readable storage medium" should also be considered to include any medium capable of storing or encoding a set of instructions for machine execution and causing a machine to perform any one or more methods of this disclosure. Therefore, the term "machine-readable storage medium" should be considered to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0080] Some parts of the previously described descriptions have been presented regarding the algorithms and symbolic representations of operations on data bits within computer memory. These algorithmic descriptions and representations are the means by which those skilled in the art of data processing most effectively communicate the essence of their work to others skilled in the art. Algorithms are, and generally are, considered as a self-consistent sequence of operations that produce the desired result. These operations are those requiring physical manipulation of physical quantities. These quantities are typically, but not necessarily, in the form of electrical or magnetic signals that can be stored, combined, compared, and otherwise manipulated. Sometimes, primarily for general reasons, it has proven convenient to refer to these signals as bits, values, elements, symbols, characters, terms, quantities, etc.

[0081] However, it should be remembered that all these and similar terms should be associated with appropriate physical quantities and are merely convenient notations applied to those quantities. This disclosure can refer to the actions and processes of a computer system or similar electronic computing device that manipulate data represented as physical (electronic) quantities in the registers and memories of the computer system and transform said data into other data similarly represented as physical quantities in the memory or registers or other such information storage systems of the computer system.

[0082] This disclosure also relates to apparatus for performing the operations described herein. Such apparatus may be specifically constructed for its intended purpose, or the apparatus may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in a computer. Such computer programs may be stored in computer-readable storage media, such as, but not limited to, any type of disk, including floppy disks, optical disks, CD-ROMs and magneto-optical disks, read-only memory (ROM), random access memory (RAM), EPROM, EEPROM, magnetic cards or optical cards, or any type of media suitable for storing electronic instructions, each coupled to a computer system bus.

[0083] The algorithms and displays presented herein are not inherently related to any particular computer or other device. Various general-purpose systems can be used with the programs taught herein, or it may be convenient to construct more specialized devices to perform the methods described herein. The structures of various such systems will be presented as described below. Furthermore, this disclosure is described without reference to any particular programming language. It should be understood that the teachings of this disclosure as described herein can be implemented using various programming languages.

[0084] This disclosure can be provided as a computer program product or software, which may include a machine-readable medium on which instructions are stored for programming a computer system (or other electronic device) to perform processes according to this disclosure. The machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). In some embodiments, the machine-readable (e.g., computer-readable) medium includes machine-readable storage media, such as read-only memory (“ROM”), random access memory (“RAM”), disk storage media, optical storage media, flash memory components, etc.

[0085] In the foregoing description, embodiments of the present disclosure have been described with reference to specific exemplary embodiments thereof. It will be apparent that various modifications can be made to the present disclosure without departing from the broader spirit and scope of the embodiments set forth in the appended claims. Therefore, the description and drawings should be regarded as illustrative rather than restrictive.

Claims

1. A method for a memory system, comprising: receiving, by a local media controller of a first memory device, a first number of cycles broadcasted by a second memory device via a bus connecting the first memory device and the second memory device; initializing a counter associated with the first memory device; in response to determining that a value of the counter matches the first number of cycles, transmitting a status of the first memory device via the bus, wherein the status comprises a second number of cycles associated with the first memory device, wherein the second number of cycles is determined in accordance with the status of the first memory device; and in response to determining that the status of the first memory device is ready, sending a status of a memory region of the first memory device to a memory sub-system controller managing the first memory device.

2. The method of claim 1, further comprising: in response to determining that the status of the first memory device is busy, determining that the second number of cycles is a minimum number of cycles.

3. The method of claim 1, wherein the local media controller of the first memory device keeps the counter synchronized with a second counter associated with the second memory device.

4. The method of claim 1, wherein the status of the memory region within the first memory device comprises at least one of a ready / busy status of the memory region, or a pass / fail status of the memory region, and wherein the second number of cycles is further determined in accordance with the ready / busy status of the memory region or the pass / fail status of the memory region.

5. The method of claim 1, in response to determining that the status of the first memory device is ready, the local media controller of the first memory device further sends at least one of an outlier of a performance metric, soft decision information, a hot alert, an interrupt, or read calibration data to the memory sub-system controller managing the first memory device, and wherein the second number of cycles is determined in accordance with the outlier of the performance metric, the soft decision information, the hot alert, the interrupt, or the read calibration data.

6. The method of claim 1, receiving a synchronization clock command during at least one of a power-on event, a predefined time interval, or in response to detecting an error on the first memory device via a single bit serial bus.

7. The method of claim 1, wherein the status of the memory region within the first memory device is sent to the memory sub-system controller via a sideband channel.

8. The method of claim 1, further comprising: in response to sending the status of the memory region within the first memory device, receiving an input / output (I / O) operation from the memory sub-system controller via an input / output (I / O) channel.

9. A memory system, comprising: a plurality of memory devices; and a memory sub-system controller managing the plurality of memory devices. a processing device operatively coupled to the plurality of memory devices to perform operations comprising: receiving, via a local media controller of a first memory device of the plurality of memory devices, a first number of cycles broadcast by a second memory device of the plurality of memory devices over a bus connecting the first memory device and the second memory device; initializing a counter associated with the first memory device; in response to determining that a value of the counter matches the first number of cycles, transmitting a status of the first memory device via the bus, wherein the status comprises a second number of cycles associated with the first memory device, wherein the second number of cycles is determined from the status of the first memory device; and in response to determining that the status of the first memory device is ready, sending a status of a memory region of the first memory device to a memory sub-system controller managing the first memory device.

10. The memory system of claim 9, wherein the operations further comprise: in response to determining that the status of the first memory device is busy, determining that the second number of cycles is a minimum number of cycles.

11. The memory system of claim 9, wherein the local media controller of the first memory device keeps the counter synchronized with a second counter associated with the second memory device.

12. The memory system of claim 9, wherein the status of the memory region within the first memory device comprises at least one of: a ready / busy status of the memory region, or a pass / fail status of the memory region, and wherein the second number of cycles is further determined from the ready / busy status of the memory region or the pass / fail status of the memory region.

13. The memory system of claim 9, wherein in response to determining that the status of the first memory device is ready, the local media controller of the first memory device further sends at least one of: an outlier of a performance metric, soft decision information, a hot alert, an interrupt, or read calibration data, to the memory sub-system controller managing the first memory device, and wherein the second number of cycles is determined from the outlier of the performance metric, the soft decision information, the hot alert, the interrupt, or the read calibration data.

14. A non-transitory computer-readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising: receiving, by a local media controller of a first memory device, a first number of cycles broadcast by a second memory device via a bus connecting the first memory device and the second memory device; initializing a counter associated with the first memory device; in response to determining that the value of the counter matches the first number of cycles, transmitting a status of the first memory device via the bus, wherein the status comprises a second number of cycles associated with the first memory device, wherein the second number of cycles is determined from the status of the first memory device; and in response to determining that the status of the first memory device is ready, sending a status of a memory region of the first memory device to a memory sub-system controller that manages the first memory device.

15. The non-transitory computer-readable storage medium of claim 14, further comprising: in response to determining that the status of the first memory device is busy, determining the second number of cycles to be a minimum number of cycles.

16. The non-transitory computer-readable storage medium of claim 14, wherein the local media controller of the first memory device keeps the counter synchronized with a second counter associated with the second memory device.

17. The non-transitory computer-readable storage medium of claim 16, wherein the status of the memory region within the first memory device comprises at least one of a ready / busy status of the memory region, or a pass / fail status of the memory region, and wherein the second number of cycles is further determined from the ready / busy status of the memory region or the pass / fail status of the memory region.

18. The non-transitory computer-readable storage medium of claim 14, in response to determining that the status of the first memory device is ready, the local media controller of the first memory device further sends at least one of an outlier value of a performance metric, soft decision information, a hot alert, an interrupt, or read calibration data to the memory sub-system controller that manages the first memory device, and wherein the second number of cycles is determined from the outlier value of the performance metric, the soft decision information, the hot alert, the interrupt, or the read calibration data.

19. The non-transitory computer-readable storage medium of claim 14, receiving a synchronization clock command during at least one of a power-on event, a predefined time interval, or in response to detecting an error on the first memory device via a single bit serial bus.

20. The non-transitory computer-readable storage medium of claim 14, wherein the status of the memory region within the first memory device is sent to the memory sub-system controller via a sideband channel.

Images