Storage system

Abstract

A storage system capable of reducing power consumption with multiple links is implemented. According to an embodiment, the storage system is provided with a first controller, a second controller, and a third controller. The first controller sets a first link between a first host and the storage system to either an active state or a low power state. The second controller sets a second link between a second host and the storage system to either an active state or a low power state. The third controller sends a busy signal to the first controller and the second controller in a case where data packet transmission via either of the first link and the second link is predicted. The first controller maintains the first link in the low power state by deactivating the busy signal in a case where the first link is in the low power state and no data packet containing data to be transmitted to the third controller is received via the first link.

Description

[0001] This application claims priority to Japanese Patent Application No. 2022-014219 (filed on February 1, 2022). This application incorporates the entire contents of that basic application by reference. Technical Field

[0002] Embodiments of the present invention relate to storage systems. Background Technology

[0003] The PCI Express (PCIe) standard (registered trademark) is one of the interface standards used to connect information processing devices to equipment. Through an interface conforming to the PCIe standard, information processing devices and equipment are connected via a transmission path called a link. On the link, data is transmitted using packets. Data transmitted using packets includes, for example, requests from the information processing device to the device, responses from the device to the information processing device, or user data.

[0004] The PCIe standard specifies the ability to set the link to a low-power state even when the device is in an active state. This function is called Active State Power Management (ASPM).

[0005] If no data packets are transmitted through the link during a specific period, the device uses the ASPM function to transition the link from a normal operating state to a low-power state. In the PCIe standard, the normal operating state is, for example, defined as Link Power State L0. Conversely, the low-power state is, for example, defined as Link Power State L1.

[0006] When a link returns to its normal operating state from a low-power state, a delay of a few microseconds (μs) may occur. This delay can degrade the performance of data packets transmitted over the link. Therefore, in anticipation of data packet transmission over the link, control measures are implemented within the device to prevent the link from transitioning to a low-power state.

[0007] In addition, devices sometimes have multiple ports that conform to the PCIe standard. For example, a device with two ports can transmit data packets between two information processing devices via two separate links. Summary of the Invention

[0008] One implementation provides a storage system capable of reducing power consumption when utilizing multiple links for packet transmission.

[0009] According to one implementation, the storage system can be connected to multiple hosts via serial interfaces. The storage system includes non-volatile memory, a first controller, a second controller, and a third controller. The first controller sets a first link between the first host and the storage system to either an active state or a low-power state (where power consumption is lower than the active state). The second controller sets a second link between the second host and the storage system to either an active state or a low-power state. The third controller processes requests for the non-volatile memory using data contained in data packets received via the first and second links. When the third controller anticipates data packet transmission via either the first or second link, it sends a busy signal to the first and second controllers. When the first link is in a low-power state and no data packet containing data to be transmitted to the third controller is received via the first link, the first controller maintains the first link in a low-power state by invalidating the busy signal. When the first link is in an active state and no data packet containing data to be transmitted to the third controller is received via the first link, the first controller disables the busy signal and, based on the condition that no data packet is transmitted via the first link during the first period, causes the first link to switch to the low-power state. Attached Figure Description

[0010] Figure 1 This is a block diagram illustrating an example of the configuration of an information processing system, including the storage system involved in the implementation.

[0011] Figure 2 This is a diagram illustrating an example of the operation of the storage system involved in the first comparative example, where the link to the host is maintained in a normal operating state (link power state L0).

[0012] Figure 3 This is a diagram illustrating an example of how the links to the two hosts in the storage system involved in the second comparative example are maintained in a normal working state.

[0013] Figure 4 This is a block diagram illustrating an example of the configuration for controlling the link power state in the storage system of this embodiment.

[0014] Figure 5 This diagram illustrates an example of a storage system in this embodiment where one link transitions from a low-power state to a normal operating state while the other link remains in a low-power state (link power state L1).

[0015] Figure 6 This diagram illustrates an example of a storage system according to this embodiment, where one link is maintained in a normal operating state and the other link transitions from a normal operating state to a low-power state.

[0016] Label Explanation

[0017] 1. Information processing system; 2-1 First host; 2-2 Second host; 21-1, 21-2 PCIe PHY; 3. Storage system; 4. Controller; 5. NAND flash memory; 6. DRAM; 11. Host I / F; 12-1, 12-2 PCIe PHY; 13-1, 13-2 PCIe link controller; 14. NVMe controller; 15. CPU; 16. NAND I / F; 17. DRAM I / F; 131-1, 131-2 ASPM; L1 timer / controller; 132-1, 132-2 busy signal control circuit; 31. First link; 32. Second link. Detailed Implementation

[0018] The embodiments will now be described with reference to the accompanying drawings.

[0019] First, refer to Figure 1 The configuration of the information processing system 1, including the storage system according to the embodiments, will be described. The information processing system 1 includes a first host device 2-1, a second host device 2-2, and a storage system 3. Hereinafter, the first host device 2-1 will be referred to as the first host 2-1, and the second host device 2-2 will be referred to as the second host 2-2. Alternatively, without distinguishing between the first host 2-1 and the second host 2-2, these will be referred to as host 2. Furthermore, the information processing system 1 may include three or more host 2s.

[0020] Host 1 2-1 and Host 2-2 can each be either a storage server that stores large amounts of diverse user data in storage system 3, or a personal computer.

[0021] Storage system 3 is a semiconductor storage device configured to write user data to and read user data from non-volatile memory such as NAND flash memory. Storage system 3 is also referred to as a storage device. Storage system 3 is implemented, for example, as a solid-state drive (SSD).

[0022] Storage system 3 can be used as a storage device for the first host 2-1 and the second host 2-2. Storage system 3 is connected to the first host 2-1 and the second host 2-2 respectively.

[0023] The interfaces used to connect the first host 2-1 and the second host 2-2 to the storage system 3 respectively conform to standards such as PCI Express (PCIe) (registered trademark) and NVM Express (NVMe) (registered trademark).

[0024] Host 1 2-1 has a PCIe PHY 21-1. Host 2-2 has a PCIe PHY 21-2. PCIe PHY 21-1 and 21-2 are circuits equivalent to the physical layer defined by the PCIe standard. PCIe PHY 21-1 and 21-2 have physical connection forms that conform to the PCIe standard.

[0025] The storage system 3 includes, for example, a controller 4, a NAND flash memory 5, and a dynamic random access memory (DRAM) 6.

[0026] NAND flash memory 5 includes one or more memory chips. Each memory chip includes multiple blocks. A block functions as the smallest unit for data erasure operations. A block is sometimes also called an "erase block" or a "physical block." Each block includes multiple pages. Each page includes multiple memory cells connected to a single word line. A page functions as the unit for data write and data read operations. In addition, a word line can also function as the unit for data write and data read operations.

[0027] There is an upper limit to the number of program / erase cycles (P / E cycles) for each block, known as the maximum number of P / E cycles. One P / E cycle for a block includes an erase operation to erase all memory cells within the block and a write operation to write data to the pages of the block.

[0028] DRAM6 is a volatile memory. RAM such as DRAM6 contains areas for storing firmware (FW) and cache areas for logical-physical address translation tables.

[0029] Controller 4 can be implemented using circuitry such as a SoC. Controller 4 can also have built-in SRAM. In this case, DRAM 6 may not be present in storage system 3. Alternatively, DRAM 6 can be integrated into controller 4. Controller 4 functions as a storage controller configured to control NAND flash memory 5.

[0030] Controller 4 can also function as a flash translation layer (FTL) that performs data management and block management for the NAND flash memory 5. Data management performed by this FTL includes (1) managing mapping information representing the correspondence between logical addresses and physical addresses of the NAND flash memory 5, and (2) handling to hide the differences between page-based data read / write operations and block-based data erase operations. Block management includes the management of bad blocks, wear leveling, and garbage collection.

[0031] Logical addresses are used by host 2 to assign addresses to storage areas in storage system 3. A logical address is, for example, a logical block address (LBA).

[0032] The management of the mapping between logical addresses and physical addresses is performed, for example, using a logical-physical address translation table (LPTD). Controller 4 uses the LPTD to manage the mapping between logical addresses and physical addresses in units of a specific management size. The physical address corresponding to a logical address indicates the physical storage location within the NAND flash memory 5 where user data has been written to that logical address. The LPTD can also be loaded from the NAND flash memory 5 into the DRAM 6 during the startup of the storage system 3.

[0033] Writing data to a page can be done only once per P / E cycle. Therefore, instead of writing the updated user data corresponding to a certain logical address to the physical storage location that stores the previous user data corresponding to that logical address, controller 4 writes it to another physical storage location. Furthermore, controller 4 invalidates the previous user data by updating the logical physical address translation table to associate the logical address with that other physical storage location.

[0034] Controller 4 includes, for example, a host interface (host I / F) 11, a CPU 15, a NAND interface (NAND I / F) 16, and a DRAM interface (DRAM I / F) 17. These host I / F 11, CPU 15, NAND I / F 16, and DRAM I / F 17 can also be connected via bus 10.

[0035] CPU 15 is a processor configured to control host I / F 11, NAND I / F 16, and DRAM I / F 17. CPU 15 performs various processes by executing file systems (FWs) loaded from NAND flash memory 5 into DRAM 6. The FW is a control program that includes a set of commands for causing CPU 15 to perform various processes. CPU 15 is capable of executing command processing, etc., to process various commands from host 2. The operation of CPU 15 is controlled by the FWs executed by CPU 15.

[0036] The NAND I / F16 electrically connects the controller 4 to the NAND flash memory 5. The NAND I / F16 is compatible with interface standards such as Toggle DDR and Open NAND Flash Interface (ONFI).

[0037] The NAND I / F16 functions as a NAND control circuit that controls the NAND flash memory 5. The NAND I / F16 can also be connected to multiple memory chips within the NAND flash memory 5 via multiple channels (Ch). By driving multiple memory chips in parallel, the access bandwidth for the entire NAND flash memory 5 can be widened.

[0038] DRAM I / F17 functions as a DRAM control circuit that controls access to DRAM6.

[0039] The host I / F11 is a circuit that functions as an interface for communication between the storage system 3 and each of the multiple hosts 2-1 and 2-2. The host I / F11 includes circuitry for receiving data packets from the first host 2-1 and the second host 2-2, respectively. These data packets are, for example, data packets conforming to the PCIe standard. The received data packets include various commands, such as input / output (I / O) commands and various control commands.

[0040] Additionally, host I / F11 includes circuitry for sending data packets to host 2-1 and host 2-2, respectively. The data packets being sent may include, for example, responses to commands and user data.

[0041] The host I / F11 may include PCIe PHY12-1, 12-2, PCIe link controller 13-1, 13-2 and NVMe controller 14.

[0042] PCIe PHY12-1 and 12-2 are circuits equivalent to the physical layer defined by the PCIe standard. PCIe PHY12-1 and 12-2, for example, have physical connection forms that conform to the PCIe standard.

[0043] PCIe PHY12-1 is connected to PCIe PHY21-1 of host 2-1 via a serial interface. This serial interface includes a link 31 (hereinafter referred to as the first link 31) that enables PCIe PHY12-1 and PCIe PHY21-1 to be interconnected. PCIe PHY12-1 and PCIe PHY21-1 perform interface operations for physically sending and receiving data via the first link 31.

[0044] PCIe PHY12-2 is connected to PCIe PHY21-2 of a second host 2-2 via a serial interface. This serial interface includes a link 32 (hereinafter referred to as the second link 32) that enables interconnection between PCIe PHY12-2 and PCIe PHY21-2. PCIe PHY12-2 and PCIe PHY21-2 perform physical data transmission and reception interface operations via the second link 32. Data is transmitted in packet form in both the first link 31 and the second link 32.

[0045] PCIe link controller 13-1 is circuitry that manages the first link 31 and performs processing for exchanging data between PCIe PHY 12-1 and NVMe controller 14. More specifically, PCIe link controller 13-1 receives data packets from the first host 2-1 via the first link 31 and PCIe PHY 12-1. PCIe link controller 13-1 processes the data packets, for example, obtaining data that should be sent to NVMe controller 14. PCIe link controller 13-1 sends the obtained data to NVMe controller 14. The data to be sent to NVMe controller 14 is, for example, data associated with access to NAND flash memory 5. More specifically, the data to be sent to NVMe controller 14 is, for example, write requests, read requests, and vendor-defined messages (VDMs) for NAND flash memory 5.

[0046] PCIe link controller 13-2 is circuitry that manages the second link 32 and performs processing for exchanging data between PCIe PHY 12-2 and NVMe controller 14. More specifically, PCIe link controller 13-2 receives data packets from the second host 2-2 via the second link 32 and PCIe PHY 12-2. PCIe link controller 13-2 processes the data packets, for example, retrieving data that should be sent to NVMe controller 14. PCIe link controller 13-2 then sends the retrieved data to NVMe controller 14.

[0047] The NVMe controller 14 is a circuit that processes transactions such as write requests, read requests, or VDM requests for NAND flash memory 5. The NVMe controller 14 performs operations corresponding to requests contained in data received via PCIe PHY 12-1 and PCIe link controller 13-1, operations corresponding to requests contained in data received via PCIe PHY 12-2 and PCIe link controller 13-2, operations for sending data containing responses to requests via PCIe PHY 12-1 and PCIe link controller 13-1, and operations for sending data containing responses to requests via PCIe PHY 12-2 and PCIe link controller 13-2. The operation of the NVMe controller 14, for example, conforms to the NVMe standard.

[0048] Thus, the PCIe PHY 12-1, PCIe link controller 13-1, and NVMe controller 14 control the data transmission between the first host 2-1 and the storage system 3. Additionally, the PCIe PHY 12-2, PCIe link controller 13-2, and NVMe controller 14 control the data transmission between the second host 2-2 and the storage system 3. That is, the storage system 3 has a port (hereinafter referred to as port 0) for data transmission with the first host 2-1 and a port (hereinafter referred to as port 1) for data transmission with the second host 2-2.

[0049] Furthermore, the functions of each part within controller 4 can be implemented either by dedicated hardware within controller 4 or by executing FW through CPU 15.

[0050] Here, we will explain the link power states set for a link. Link power states are the power states set for a link. Link power states are set, for example, through the ASPM function specified by the PCIe standard. The ASPM function is a function that allows the link to be set to a low-power state even when the device (e.g., storage system 3) is in an active state. For example, there are link power states L0 and L1. Link power state L0 is the normal operating state (active state). Link power state L1 is the low-power state (inactive state). Link power state L1 is also referred to as ASPM L1.

[0051] If a link that has transitioned to link power state L1 reverts to link power state L0, for example, a delay of a few microseconds (exit latency) will occur. Due to this latency, the performance of data packet transmission over the link may be degraded. Therefore, in the device, control is implemented to prevent the link from transitioning to link power state L1 when data packet transmission over the link is predicted.

[0052] The control example of link power state is illustrated using the first and second comparison examples.

[0053] (Comparative Example 1)

[0054] The first comparative example involves a storage system that performs data transfer with a host.

[0055] Figure 2 This represents an example in which the link 31A between the storage system 3A and the host 2A in the first comparative example is maintained in link power state L0 (normal operating state).

[0056] Storage system 3A has a port for connecting to a host 2A. Specifically, storage system 3A includes a PCIe PHY 12A, a PCIe link controller 13A, and an NVMe controller 14A.

[0057] The PCIe PHY12A is connected to the PCIe PHY21A of the host 2A via a serial interface. This serial interface includes a link 31A that enables the PCIe PHY12A and PCIe PHY21A to be interconnected.

[0058] PCIe link controller 13A manages link 31A between PCIe PHY12A and PCIe PHY21A, and performs processing for exchanging data between PCIe PHY12A and NVMe controller 14A.

[0059] The NVMe controller 14A performs operations corresponding to commands contained in data received via the PCIe PHY 12A and PCIe link controller 13A, and operations for sending data containing responses to the commands via the PCIe PHY 12A and PCIe link controller 13A.

[0060] In this way, the PCIe PHY12A, PCIe link controller 13A, and NVMe controller 14A control the data transmission between host 2A and storage system 3A.

[0061] Here, the configuration of the control link power state in storage system 3A will be explained.

[0062] The PCIe link controller 13A features an ASPM L1 timer / controller 131A.

[0063] The ASPM L1 timer / controller 131A is a circuit with ASPM functionality. The ASPM L1 timer / controller 131A controls the link power state of link 31A through the ASPM function. More specifically, the ASPM L1 timer / controller 131A functions as a timer to measure the duration of data packets not transmitted via link 31A when link 31A is set to link power state L0. When the measured time exceeds a threshold (i.e., when the timer expires), the ASPM L1 timer / controller 131A causes link 31A to transition from link power state L0 to link power state L1. Thus, link 31A enters a low-power state.

[0064] When the NVMe controller 14A anticipates a data packet transmission via link 31A, it sends a busy signal to the ASPM L1 timer / controller 131A. The busy signal indicates that the NVMe controller 14A is in a busy state. The busy NVMe controller 14A anticipates data packet transmissions via link 31A.

[0065] When the ASPM L1 timer / controller 131A sends a busy signal from the NVMe controller 14A while link 31A is set to link power state L1, it causes link 31A to transition to link power state L0. Furthermore, during the period when link 31A is set to link power state L0 and the NVMe controller 14A sends a busy signal (i.e., the period when the busy signal is active), the ASPM L1 timer / controller 131A does not measure the duration of the state where no data packets have been transmitted via link 31A. Therefore, during the period when the NVMe controller 14A sends a busy signal, the ASPM L1 timer / controller 131A will not cause link 31A to transition from link power state L0 to link power state L1.

[0066] By operating the ASPM L1 timer / controller 131A in response to such a busy signal, when the transmission of data packets via link 31A is predicted, link 31A does not transition from link power state L0 to link power state L1. Therefore, during the operation of the ASPM L1 timer / controller 131A while receiving a busy signal, there is no transition from link 31A set to link power state L0 to link power state L1, nor is there a return to link power state L0 when link 31A is set to link power state L1. Consequently, no delay occurs due to the transition from link power state L1 to link power state L0. Therefore, performance degradation in data packet transmission via link 31A can be prevented when the transmission of data packets via link 31A is predicted.

[0067] (Comparative Example 2)

[0068] The second comparative example involves a storage system that performs data transfers with each of the two hosts.

[0069] Figure 3 This represents an example in which the link 31B with host 2-1B and the link 31B with host 2-2B in the storage system 3B involved in the second comparative example are both maintained in the link power state L0 (normal operating state).

[0070] Storage system 3B has two ports for connecting to two hosts 2-1B and 2-2B respectively. Specifically, storage system 3B includes PCIe PHYs 12-1B and 12-2B, PCIe link controllers 13-1B and 13-2B, and an NVMe controller 14B.

[0071] The PCIe PHY12-1B is connected to the PCIe PHY21-1B of the host 2-1B via a serial interface. This serial interface includes a link 31B (hereinafter referred to as the first link 31B) that enables the PCIe PHY12-1B and PCIe PHY21-1B to be interconnected.

[0072] PCIe PHY12-2B is connected to PCIe PHY21-2B of host 2-2B via a serial interface. This serial interface includes link 32B (hereinafter referred to as second link 32B) which enables the interconnection of PCIe PHY12-2B and PCIe PHY21-2B.

[0073] PCIe link controller 13-1B manages the first link 31B and performs processing for exchanging data between PCIe PHY 12-1B and NVMe controller 14B.

[0074] PCIe link controller 13-2B manages the second link 32B and performs processing for exchanging data between PCIe PHY 12-2B and NVMe controller 14B.

[0075] The NVMe controller 14B performs operations corresponding to commands contained in data received via any one of the PCIe PHY 12-1B, PCIe link controller 13-1B, PCIe PHY 12-2B, and PCIe link controller 13-2B, and operations for sending data containing responses to the commands via any one of the PCIe PHY 12-1B, PCIe link controller 13-1B, PCIe PHY 12-2B, and PCIe link controller 13-2B.

[0076] Thus, the PCIe PHY12-1B, PCIe link controller 13-1B, and NVMe controller 14B control the data transmission between host 2-1B and storage system 3B. The PCIe PHY12-2B, PCIe link controller 13-2B, and NVMe controller 14B control the data transmission between host 2-2B and storage system 3B.

[0077] Here, the configuration of the control link power state in storage system 3B will be explained.

[0078] The PCIe link controller 13-1B features an ASPM L1 timer / controller 131-1B.

[0079] The ASPM L1 timer / controller 131-1B is a circuit with ASPM functionality that controls the link power state of the first link 31B. More specifically, the ASPM L1 timer / controller 131-1B functions as a timer to measure the duration of a state in which no data packets are transmitted via the first link 31B when the first link 31B is set to link power state L0. If the measured time exceeds a threshold, the ASPM L1 timer / controller 131-1B changes the link power state of the first link 31B from link power state L0 to link power state L1. Thus, the first link 31B enters a low-power state.

[0080] PCIe link controller 13-2B includes ASPM L1 timer / controller 131-2B. ASPM L1 timer / controller 131-2B controls the link power status of the second link 32B in the same way as ASPM L1 timer / controller 131-1B.

[0081] When the NVMe controller 14B predicts a data packet transmission via link 31B or 32B, it sends a busy signal to both the ASPML1 timer / controller 131-1B and the ASPM L1 timer / controller 131-2B. The busy signal indicates that the NVMe controller 14B is in a busy state. The busy NVMe controller 14B then predicts the data packet transmission via link 31B or 32B.

[0082] When the ASPM L1 timer / controller 131-1B sends a busy signal from the NVMe controller 14B while the first link 31B is set to link power state L1, it causes the first link 31B to transition to link power state L0. Furthermore, the ASPM L1 timer / controller 131-1B does not measure the duration of the state where no data packets have been transmitted via the first link 31B during the period when the first link 31B is set to link power state L0 and the NVMe controller 14B sends a busy signal. Therefore, during the period when the NVMe controller 14B sends a busy signal, the ASPM L1 timer / controller 131-1B will not cause the first link 31B to transition from link power state L0 to link power state L1.

[0083] When the second link 32B is set to link power state L1 and a busy signal is sent by the NVMe controller 14B, the ASPM L1 timer / controller 131-2B causes the second link 32B to transition to link power state L0. Furthermore, during the period when the second link 32B is set to link power state L0 and a busy signal is sent by the NVMe controller 14B, the ASPM L1 timer / controller 131-2B does not measure the duration of the state where no data packets have been transmitted via the second link 32B. Therefore, during the period when a busy signal is sent by the NVMe controller 14B, the ASPM L1 timer / controller 131-2B will not cause the second link 32B to transition from link power state L0 to link power state L1.

[0084] By operating the ASPM L1 timers / controllers 131-1B and 131-2B corresponding to such busy signals, when data packet transmission via link 31B or 32B is predicted, neither link 31B nor 32B transitions from link power state L0 to link power state L1. Therefore, during the operation of the ASPM L1 timers / controllers 131-1B corresponding to the busy signal, the process of link 31B, which was set to link power state L0, transitioning to link power state L1 and then reverting to link power state L0 is avoided. Similarly, during the operation of the ASPM L1 timers / controllers 131-2B corresponding to the busy signal, the process of link 32B, which was set to link power state L0, transitioning to link power state L1 and then reverting to link power state L0 is avoided. Thus, no delay occurs due to the transition from link power state L1 to link power state L0. Therefore, performance degradation during data packet transmission via link 31B or 32B can be prevented when data packet transmission via link 31B or 32B is predicted.

[0085] Furthermore, the NVMe controller 14B is configured not to determine which link, the first link 31B or the second link 32B, the predicted data packet transmission is transmitted via. Therefore, even if the NVMe controller 14B predicts that the predicted data packet transmission is transmitted via either the first link 31B or the second link 32B, it still sends a busy signal to both the ASPM L1 timer / controller 131-1B and the ASPM L1 timer / controller 131-2B.

[0086] Busy signals are sent to both ASPM L1 timer / controller 131-1B and ASPM L1 timer / controller 131-2B. Therefore, in storage system 3B, if it is predicted that the data packet transmission will occur via one of the two links 31B and 32B and not via the other link, not only the link of one link but also the link of the other link will not change from link power state L0 to link power state L1.

[0087] For example, if it is predicted that a data packet transmission will occur via link 31B and not via link 32B, the ASPM L1 timer / controller 131-2B corresponding to the busy signal will not cause link 32B to transition from link power state L0 to link power state L1, even though no data packets are transmitted via link 32B during a specific period. Therefore, the power consumption reduction achieved by transitioning link 32B to link power state L1 cannot be obtained.

[0088] In contrast, in the storage system 3 of this embodiment, each of the two PCIe link controllers 13-1 and 13-2 is equipped with a circuit (hereinafter referred to as the busy signal control circuit) that enables or disables a busy signal sent from the NVMe controller 14 based on whether a data packet containing data transmitted from the two hosts 2-1 and 2-2 to the NVMe controller 14 is detected. In the storage system 3, the link power states of the two links 31 and 32 are controlled based on the busy signal enabled or disabled by the busy signal control circuit. Therefore, in the storage system 3, when it is predicted that a data packet transmission will occur via one of the links 31 and 32, and not via the other link, the other link can transition from link power state L0 to link power state L1. Thus, by transitioning the other link to link power state L1, power consumption can be reduced.

[0089] Reference Figure 4 Here, we will give a specific description of an example of the configuration of the control link power state in the storage system 3 of this embodiment.

[0090] PCIe link controller 13-1 includes ASPM L1 timer / controller 131-1 and busy signal control circuit 132-1. PCIe link controller 13-2 includes ASPM L1 timer / controller 131-2 and busy signal control circuit 132-2.

[0091] When NVMe controller 14 anticipates packet transmission via either link 31 or 32, it sends a busy signal to PCIe link controllers 13-1 and 13-2. Sending a busy signal is also referred to as activating the busy signal. Conversely, not sending a busy signal is referred to as deactivating the busy signal. For example, NVMe controller 14 sends a busy signal if processing of all requests received from hosts 2-1 and 2-2 has not been completed (e.g., if no responses to all requests have been returned). The busy signal sent by NVMe controller 14 is not received by ASPML1 timers / controllers 131-1 and 131-2, but rather by busy signal control circuits 132-1 and 132-2. The busy signal indicates that NVMe controller 14 is in a busy state. While in a busy state, NVMe controller 14 anticipates packet transmission via either link 31 or 32.

[0092] The NVMe controller 14, for example, predicts packet transmissions via either link 31 or 32 when it receives data contained in a packet transmitted from the first host 2-1 via link 31 from the PCIe link controller 13-1, or when it receives data contained in a packet transmitted from the second host 2-2 via link 32 from the PCIe link controller 13-2. Furthermore, the NVMe controller 14 does not distinguish which link, first link 31 or second link 32, the predicted packet transmission occurred via.

[0093] If no packet transmissions via link 31 or link 32 are anticipated, NVMe controller 14 will not send a busy signal to either PCIe link controller 13-1 or PCIe link controller 13-2. For example, NVMe controller 14 will stop sending busy signals once processing of all requests received from hosts 2-1 and 2-2 is complete (e.g., all responses to the requests have been returned). That is, the busy signal will be changed from active to inactive.

[0094] Busy signal control circuit 132-1 is a circuit that enables or deactivates a busy signal sent by NVMe controller 14 based on whether a data packet containing data transmitted from the first host 2-1 to the NVMe controller 14 via the first link 31 is detected. In the following, a data packet containing data transmitted from either the first host 2-1 or the second host 2-2 to the NVMe controller 14 is also referred to as an object data packet.

[0095] The busy signal control circuit 132-1 detects target data packets. More specifically, when the PCIe link controller 13-1 receives a data packet via the first link 31 and the PCIe PHY 12-1, the busy signal control circuit 132-1 determines, for example, whether the data packet contains data that should be sent to the NVMe controller 14. If the data packet contains data that should be sent to the NVMe controller 14, the busy signal control circuit 132-1 detects the data packet as the target data packet.

[0096] Alternatively, the busy signal control circuit 132-1 can also determine whether a data packet contains data associated with access to the NAND flash memory 5 when the PCIe link controller 13-1 receives a data packet. In this case, the busy signal control circuit 132-1 detects the data packet containing data associated with access to the NAND flash memory 5 as the target data packet.

[0097] Data associated with access to the NAND flash memory 5 is, for example, data that subsequently causes the generation of data packets transmitted via the first link 31. Data that subsequently causes the generation of data packets transmitted via the first link 31 is, for example, a write request, read request, or VDM for the NAND flash memory 5. The busy signal control circuit 132-1 can also detect data packets containing data that subsequently causes the generation of data packets transmitted via the first link 31 as target data packets. In this case, the busy signal control circuit 132-1 can activate the busy signal only if a data packet transmission via the first link 31 subsequently occurs.

[0098] When a target data packet is detected, the busy signal control circuit 132-1 activates the busy signal. That is, when a target data packet is detected, the busy signal control circuit 132-1 sends the busy signal sent by the NVMe controller 14 to the ASPM L1 timer / controller 131-1 unchanged.

[0099] More specifically, the busy signal control circuit 132-1 activates the busy signal after detecting an object data packet, when the busy signal sent by the NVMe controller 14 changes from inactive to active. Additionally, the busy signal control circuit 132-1 also activates the busy signal when the busy signal sent by the NVMe controller 14 is already active, upon detecting an object data packet. Then, if the busy signal changes from active to inactive, the busy signal control circuit 132-1 resets its state. The reset busy signal control circuit 132-1 then deactivates the busy signal.

[0100] On the other hand, if no target data packet is detected, the busy signal control circuit 132-1 invalidates the busy signal. That is, if no target data packet is detected, the busy signal control circuit 132-1 does not send the busy signal sent by the NVMe controller 14 to the ASPM L1 timer / controller 131-1.

[0101] The ASPM L1 timer / controller 131-1 is a circuit with ASPM functionality. The ASPM L1 timer / controller 131-1 controls the link power status of the first link 31 through the ASPM function.

[0102] Specifically, the ASPM L1 timer / controller 131-1 functions as a timer to measure the duration of a state in which no data packets are transmitted via the first link 31 when the first link 31 is set to link power state L0. If the measured time exceeds a threshold, the ASPM L1 timer / controller 131-1 causes the first link 31 to transition from link power state L0 to link power state L1. Thus, the first link 31 enters a low-power state.

[0103] Furthermore, at least a portion of the components within the storage system 3 that manage and control the first link 31 operate in a low-power state. Additionally, enabling the first link 31 to operate in a normal operating state involves the components within the storage system 3 that manage and control the first link 31 operating in a normal operating state. The components within the storage system 3 that manage and control the first link 31 include, for example, a PCIe PHY 12-1 and a PCIe link controller 13-1.

[0104] If the ASPM L1 timer / controller 131-1 receives a busy signal from the busy signal control circuit 132-1 during the period when the first link 31 is set to link power state L1, it causes the first link 31 to transition to link power state L0. Furthermore, if the ASPM L1 timer / controller 131-1 does not receive a busy signal during the period when the first link 31 is set to link power state L1, the first link 31 is maintained in link power state L1. The ASPM L1 timer / controller 131-1 does not accept busy signals if the NVMe controller 14 does not send a busy signal or if the busy control circuit 132-1 invalidates the busy signal sent by the NVMe controller 14.

[0105] The ASPM L1 timer / controller 131-1 does not measure the duration of the state in which no data packets have been transmitted via the first link 31 during the period when the first link 31 is set to link power state L0 and receives a busy signal from the busy signal control circuit 132-1. Therefore, during the period when the busy signal control circuit 132-1 sends out a busy signal, the ASPM L1 timer / controller 131-1 will not cause the first link 31 to transition from link power state L0 to link power state L1.

[0106] The ASPM L1 timer / controller 131-1 measures the duration of the period during which no data packets are transmitted via the first link 31 while the first link 31 is set to link power state L0 and no busy signal is received from the busy signal control circuit 132-1. If the measured time exceeds a threshold, the ASPM L1 timer / controller 131-1 causes the first link 31 to transition from link power state L0 to link power state L1. Thus, not only when the NVMe controller 14 does not send a busy signal, but also when the busy signal control circuit 132-1 invalidates the busy signal sent by the NVMe controller 14, the ASPM L1 timer / controller 131-1 can also cause the first link 31 to transition to link power state L1 if the duration of the period during which no data packets are transmitted via the first link 31 exceeds the threshold.

[0107] Therefore, in the absence of a target data packet detected by the busy signal control circuit 132-1, i.e., when it is predicted that the data packet transmission will occur via the second link 32 and not via the first link 31, the period during which the first link 31 is in link power state L1 can be extended, and the probability of the first link 31 transitioning to link power state L1 can be increased. Thus, power consumption can be reduced when utilizing multiple links 31 and 32.

[0108] Furthermore, the operation of the second link 32 performed by the busy signal control circuit 132-2 in the PCIe link controller 13-2, as well as the ASPM L1 timer / controller 131-2 and the NVMe controller 14, is the same as the operation of the first link 31 performed by the busy signal control circuit 132-1 in the aforementioned PCIe link controller 13-1, as well as the ASPM L1 timer / controller 131-1 and the NVMe controller 14.

[0109] Reference Figure 5 and Figure 6 A specific example of controlling the link power status of two links 31 and 32 in storage system 3 is illustrated.

[0110] Figure 5 This represents an example of operation where link 31 (port 0) transitions from link power state L1 to link power state L0, while link 32 (port 1) remains in link power state L1. At the start of this operation, both link 31 and link 32 are initially set to link power state L1. Figure 5 (1)). In addition, the busy signal sent by the NVMe controller 14 at this start time can be either active or inactive.

[0111] Based on the fact that data has been generated that should be sent to the storage system 3, host 2-1 executes a transition sequence to change link 31 to link power state L0. Figure 5 (2)). The timing of this transition includes, for example, training processes to make the first link 31 available, such as the synchronization of clock signals between the first host 2-1 and the storage system 3.

[0112] PCIe link controller 13-1 transitioned link 31 to link power state L0 by executing the transition timing. Figure 5 (3) in the middle.

[0113] After the first host 2-1 transitions to link power state L0 on the first link 31, it sends a data packet containing the data to be sent to the storage system 3 via the first link 31. Figure 5 (4) in the middle.

[0114] PCIe link controller 13-1 receives data packets via PCIe PHY 12-1. PCIe link controller 13-1 processes the received data packets, for example, to obtain the data that should be sent to NVMe controller 14. PCIe link controller 13-1 sends the obtained data to NVMe controller 14.

[0115] Busy signal control circuit 132-1 detects data packets (object data packets) containing data transmitted from host 2-1 to NVMe controller 14 via link 31. Figure 5 (5)). Busy signal control circuit 132-1 detects the data packet as an object data packet, for example, when the data packet received by PCIe link controller 13-1 contains data that should be sent to NVMe controller 14.

[0116] The NVMe controller 14 receives data from data packets sent by the PCIe link controller 13-1. Based on the received data, the NVMe controller 14 determines whether data packet transmission via link 31 or 32 is predicted. Here, we assume the case where data packet transmission via link 31 or 32 is predicted. In this case, the NVMe controller 14 activates the busy signal. Figure 5 (6)). Furthermore, when the busy signal is already active, the NVMe controller 14 maintains the busy signal as active. The busy signal being active is received by the busy signal control circuit 132-1 and the busy signal control circuit 132-2.

[0117] The busy signal control circuit 132-1 detected the target data packet, and therefore, activated the busy signal received from the NVMe controller 14. Figure 5 (7-1)). That is, the busy signal control circuit 132-1 sends a busy signal to the ASPM L1 timer / controller 131-1.

[0118] ASPM L1 timer / controller 131-1 receives a busy signal, therefore, the first link 31 is maintained in link power state L0. Figure 5 (8-1) in the middle.

[0119] Thus, the busy signal control circuit 132-1 activates the busy signal upon detecting a target data packet. Therefore, through the operation of the ASPM L1 timer / controller 131-1, the first link 31 does not transition to link power state L1. Consequently, delays caused by the first link 31 reverting from link power state L1 to link power state L0 can be prevented.

[0120] On the other hand, the busy signal control circuit 132-2 did not detect the target data packet, therefore, it invalidated the busy signal received from the NVMe controller 14. Figure 5 (7-2)). That is, the busy signal control circuit 132-2 does not send a busy signal to the ASPM L1 timer / controller 131-2.

[0121] ASPM L1 timer / controller 131-2 did not receive a busy signal, therefore, the second link 32 is maintained in link power state L1. Figure 5 (8-2) in the middle.

[0122] Thus, the busy signal control circuit 132-2 invalidates the busy signal when no target data packet is detected. Therefore, through the operation of the ASPM L1 timer / controller 131-2, the second link 32 does not transition to link power state L0. That is, the second link 32 is maintained in link power state L1. Therefore, it is possible to prevent the second link 32 from transitioning to link power state L0 due to the NVMe controller 14 sending a busy signal after predicting data packet transmission via link 31 or 32. As a result, compared to the case where the busy signal control circuit 132-2 is not provided to invalidate the busy signal, the period during which the second link 32 is maintained in link power state L1 can be extended. As a result, power consumption can be reduced.

[0123] Figure 6 This example illustrates the operation where link 31 is maintained in link power state L0, and link 32 transitions from link power state L0 to link power state L1. At the start of this operation, both link 31 and link 32 are set to link power state L0, and the busy signal sent by the NVMe controller 14 is inactive. Figure 6 (1) in the middle.

[0124] The first host 2-1 sends a data packet containing the data to be sent to the storage system 3 via the first link 31. Figure 6 (2) in the middle.

[0125] PCIe link controller 13-1 receives data packets via PCIe PHY 12-1. PCIe link controller 13-1 processes the received data packets, for example, to obtain the data that should be sent to NVMe controller 14. PCIe link controller 13-1 sends the obtained data to NVMe controller 14.

[0126] Busy signal control circuit 132-1 detects data packets (object data packets) containing data transmitted from host 2-1 to NVMe controller 14 via link 31. Figure 6 (3)). Busy signal control circuit 132-1 detects the data packet as an object data packet, for example, when the data packet received by PCIe link controller 13-1 contains data that should be sent to NVMe controller 14.

[0127] The NVMe controller 14 receives data from data packets sent by the PCIe link controller 13-1. Based on the received data, the NVMe controller 14 determines whether data packet transmission via link 31 or 32 is predicted. Here, we assume the case where data packet transmission via link 31 or 32 is predicted. In this case, the NVMe controller 14 activates the busy signal. Figure 6 (4)). Busy signal that has been set to active is received by busy signal control circuit 132-1 and busy signal control circuit 132-2.

[0128] The busy signal control circuit 132-1 detected the target data packet, and therefore, activated the busy signal received from the NVMe controller 14. Figure 6 (5-1)). That is, the busy signal control circuit 132-1 sends a busy signal to the ASPM L1 timer / controller 131-1.

[0129] ASPM L1 timer / controller 131-1 received a busy signal, therefore, the first link 31 is maintained in link power state L0. Figure 6 (6-1) in the middle.

[0130] Thus, the busy signal control circuit 132-1 activates the busy signal upon detecting a target data packet. Therefore, through the operation of the ASPM L1 timer / controller 131-1, the first link 31 does not transition to link power state L1. Consequently, delays caused by the first link 31 reverting from link power state L1 to link power state L0 can be prevented.

[0131] On the other hand, the busy signal control circuit 132-2 did not detect the target data packet, therefore, it was able to invalidate the busy signal received from the NVMe controller 14. Figure 6 (5-2)). That is, the busy signal control circuit 132-2 does not send a busy signal to the ASPML1 timer / controller 131-2.

[0132] ASPM L1 timer / controller 131-2 did not receive a busy signal, therefore, it measured the time during which no data packets were transmitted via the second link 32. Furthermore, if the measured time exceeded a threshold, ASPM L1 timer / controller 131-2 caused the second link 32 to transition from link power state L0 to link power state L1. Figure 6 (6-2) in the middle.

[0133] In this way, the busy signal control circuit 132-2 invalidates the busy signal when no target data packet is detected. Therefore, the ASPM L1 timer / controller 131-2 can transition the second link 32 to link power state L1. That is, when the NVMe controller 14 predicts that a data packet has been transmitted via link 31 or 32 and sends a busy signal, the ASPM L1 timer / controller 131-2 can also transition the second link 32 to link power state L1. Thus, compared to the case where there is no busy signal control circuit 132-2 to invalidate the busy signal, the probability of the second link 32 transitioning to link power state L1 is increased, thereby reducing power consumption.

[0134] As explained above, according to this embodiment, power consumption can be reduced when utilizing multiple links for packet transmission. PCIe link controller 13-1 sets the first link 31 between the first host 2-1 and the storage system 3 to either an operating state or a low-power state with lower power consumption than the operating state. PCIe link controller 13-2 sets the second link 32 between the second host 2-2 and the storage system 3 to either an operating state or a low-power state. NVMe controller 14 processes requests for non-volatile memory (e.g., NAND flash memory 5) using data contained in packets received via the first link 31 and the second link 32. NVMe controller 14 sends a busy signal to PCIe link controller 13-1 and PCIe link controller 13-2 when it anticipates packet transmission via either the first link 31 or the second link 32. When the first link 31 is in a low-power state and no data packet containing data to be transmitted to the NVMe controller 14 is received via the first link 31, the PCIe link controller 13-1 maintains the first link 31 in a low-power state by disabling the busy signal. When the first link 31 is in an active state and no data packet containing data to be transmitted to the NVMe controller 14 is received via the first link 31, the PCIe link controller 13-1 disabling the busy signal, based on the condition that no data packet is transmitted via the first link 31 during the first period, transitions the first link 31 into a low-power state.

[0135] With the above configuration, for example, if a data packet transmission via the second link 32 is predicted, even if the NVMe controller 14 sends a busy signal to the PCIe link controller 13-1 to set it to an operating state, the PCIe link controller 13-1 (more specifically, the busy signal control circuit 132-1) will invalidate the busy signal. Therefore, the period during which the first link 31 is in a low-power state can be extended, and the likelihood of the first link 31 transitioning to a low-power state can be increased. Thus, power consumption can be reduced when utilizing multiple links for data packet transmission.

[0136] The various functions described in this embodiment can each be implemented by a circuit (processing circuit). Examples of processing circuits include a programmed processor such as a central processing unit (CPU). This processor executes the various functions described by executing a computer program (command set) stored in memory. The processor can also be a microprocessor that includes electronic circuitry. Examples of processing circuits also include digital signal processors (DSPs), application-specific integrated circuits (ASICs), microcontrollers, controllers, and other electronic circuit components. Other components besides the CPU described in this embodiment can also be implemented separately by processing circuitry.

[0137] Several embodiments of the present invention have been described above, but these embodiments are merely illustrative and not intended to limit the scope of the invention. These new embodiments can be implemented in a wide variety of other ways, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and / or variations thereof are included within the scope and spirit of the invention, and are included within the scope of the invention as described in the claims and its equivalents.

Claims

1. A storage system capable of connecting to multiple hosts via serial interfaces, the storage system comprising: Non-volatile memory; The first controller sets the first link between the first host and the storage system to either an operating state or a low-power state with lower power consumption than the operating state. The second controller sets the second link between the second host and the storage system to either the operating state or the low-power state. as well as The third controller processes requests for the non-volatile memory using data contained in packets received via the first and second links. If packet transmission via either the first or second link is predicted, it sends a busy signal to both the first and second controllers. The first controller, When the first link is in the low-power state and no data packet containing data to be transmitted to the third controller is received via the first link, the first link is maintained in the low-power state by invalidating the busy signal. When the first link is in the operating state and no data packet containing data to be transmitted to the third controller is received via the first link, the first link is switched to the low-power state by invalidating the busy signal, based on the condition that no data packet was transmitted via the first link during the first period.

2. The storage system according to claim 1, When the second controller receives a data packet containing data to be transmitted to the third controller via the second link while the second link is in the low-power state, the second controller causes the second link to switch from the low-power state to the working state according to the busy signal.

3. The storage system according to claim 1 or 2, When the second controller is in the operating state of the second link and receives a data packet containing data to be transmitted to the third controller via the second link, it activates the busy signal and maintains the second link in the operating state.

4. The storage system according to claim 1 or 2, When the first link is in the operating state and the busy signal sent by the third controller has ended, the first controller causes the first link to switch to the low-power state based on the condition that no data packets are transmitted via the first link during the first period.

5. The storage system according to claim 1 or 2, The first controller includes circuitry that enables the busy signal when a data packet containing data to be transmitted to the third controller is received via the first link, and deactivates the busy signal when no data packet containing data to be transmitted to the third controller is received via the first link.

6. The storage system according to claim 1 or 2, Upon receiving a data packet containing first data to be transmitted to the third controller via the first link, the first controller sends the first data to the third controller. Upon receiving a data packet containing second data to be transmitted to the third controller via the second link, the second controller sends the second data to the third controller. The third controller sends a busy signal to the first controller and the second controller based on the fact that it has received either the first data or the second data.

7. The storage system according to claim 6, The third controller If a busy signal is sent to the first controller and the second controller without returning responses to all requests received from the first host or the second host, the busy signal is sent to the first controller and the second controller. When the response to all requests received from the first host or the second host is returned to the first host or the second host, the sending of the busy signal to the first controller and the second controller is terminated.

8. The storage system according to claim 6, The first data and the second data each include any one of a write request, a read request, and a vendor-defined message (VDM) for the non-volatile memory.

9. The storage system according to claim 1 or 2, The serial interface is a serial interface that conforms to the PCI Express standard.

Images