Method and apparatus for power analysis of storage system

Abstract

A storage system includes one or more storage devices, a power source to supply power to the storage devices, a processor; the processor, in response to determining that a total power consumption of the one or more storage devices is less than a first percentage threshold of a load of an active power supply, performs: deactivating one or more of the active power supplies until the total power consumption is equal to or greater than the first percentage threshold of the load of each of the active power supplies, and in response to determining that the total power consumption is equal to or greater than a second percentage threshold of the load of each of the activated power sources, activating one or more of the deactivated power sources of the power sources until the total power consumption is less than the second percentage threshold of the load of each of the activated power sources.

Description

[0001] This application is a divisional application of an invention patent application with an application date of March 4, 2019, application number 201910159333.1, and invention name “Method and device for power analysis of a storage system”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 713,466, filed on August 1, 2018, the entire contents of which are incorporated herein by reference.

[0004] This application is also a continuation-in-part of U.S. patent application No. 15 / 975,463, filed on May 9, 2018, entitled “METHOD AND APPARATUS FOR SELF-REGULATING POWER USAGE AND POWER CONSUMPTION IN ETHERNET SSD STORAGE SYSTEMS,” which claims priority to and the benefit of U.S. Provisional Application No. 62 / 638,035, filed on March 2, 2018, and the entire contents of these two applications are incorporated herein by reference. Background Art

[0005] Many companies provide cloud-based storage to end users so that end users will have the ability to access their stored data remotely. Such companies generally utilize Ethernet-attached solid-state drives (SSDs) to meet their storage requirements. Specifically, Ethernet-attached non-volatile memory high-speed NVMe (Non-Volatile Memory Express) SSDs (e.g., NVMe Over Fabrics (NVMe-oF) storage devices) are considered to be an emerging and disruptive technology in the field.

[0006] Cloud-based storage providers typically charge users for storing data on a monthly or annual basis based on the total storage space allocated to the user, and either the average cost of energy consumed by all users or the user based on the maximum power consumption that the system can consume. For example, for two users who have purchased the same amount of cloud storage space, the user who only stores a small amount of data relative to the total storage space purchased and only stores data infrequently will be charged the same fee as the user who regularly removes and adds new data and uses most of the storage space he / she purchased. Ideally, users should be charged for storage based on the actual energy resources consumed. However, there is no accurate method to calculate the power consumption of individual users, or to calculate power consumption in real time.

[0007] The above information disclosed in the Background Technology section herein is only for enhancing understanding of the background of the disclosure and therefore it may contain information that does not constitute the prior art. Summary of the invention

[0008] Various aspects of embodiments of the present invention are directed to a storage system and a method of operating the storage system that is capable of managing (eg, optimizing) the operation of power supplies of the storage system by dynamically monitoring the operation of power supplies of the storage system and ensuring that active power supplies operate within their high power efficiency range.

[0009] Various aspects of embodiments of the present invention are directed to a storage system and a method of operating the storage system that are capable of managing (e.g., optimizing) power usage of storage devices of a storage bank of the storage system by dynamically adjusting a maximum power cap of the storage devices of the storage bank based on a workload of the storage bank.

[0010] According to some embodiments of the present invention, a storage system is provided, comprising: one or more storage devices; multiple power supplies configured to supply power to the storage devices; a processor; and a memory having instructions stored thereon, which, when executed by the processor, cause the processor to execute: determining whether multiple power supplies among the multiple power supplies are activated; in response to determining that multiple power supplies are activated: determining the total power consumption of the one or more storage devices; in response to determining that the total power consumption is less than a first percentage threshold of the load of the activated power supplies among the power supplies, deactivating the activated power supplies among the power supplies one by one until the total power consumption is equal to or greater than the first percentage threshold of the load of each of the activated power supplies among the power supplies; and in response to determining that the total power consumption is equal to or greater than a second percentage threshold of the load of each of the activated power supplies among the power supplies, activating the deactivated power supplies among the power supplies one by one until the total power consumption is less than the second percentage threshold of the load of each of the activated power supplies among the power supplies.

[0011] In some embodiments, determining the total power consumption of the one or more storage devices includes: obtaining the actual power consumption of each of the one or more storage devices from the storage device or a corresponding power meter; and summing the actual power consumption of each storage device to obtain the total power consumption.

[0012] In some embodiments, obtaining the actual power consumption of each storage device includes: retrieving power measurement information from a power log corresponding to the storage device, wherein the power measurement information is measured by the corresponding power meter and recorded in the power log.

[0013] In some embodiments, the corresponding power meter is internal to the storage device.

[0014] In some embodiments, the corresponding power meter is external to and coupled to the storage device.

[0015] In some embodiments, the first percentage threshold of the load of each of the activated ones of the power supplies is 40% of the load of each of the activated ones of the power supplies.

[0016] In some embodiments, the second percentage threshold of the load of each of the activated ones of the power supplies is 90% of the load of each of the activated ones of the power supplies.

[0017] In some embodiments, the instructions further cause the processor to perform: determining whether only one of the multiple power supplies is in high availability mode; and in response to determining that only one of the multiple power supplies is in high availability mode, generating a warning message indicating that only this power supply is in high availability mode.

[0018] In some embodiments, deactivating the activated power supplies in the power supplies one by one includes: deactivating one of the activated power supplies in the power supplies; determining that the total power consumption of the one or more storage devices is less than the first percentage threshold of the load of the activated power supplies in the power supplies; and in response to the determination, deactivating another of the activated power supplies in the power supplies.

[0019] In some embodiments, activating deactivated ones of the power supplies one by one includes: activating one of the deactivated ones of the power supplies; determining that the total power consumption of the one or more storage devices is equal to or greater than the second percentage threshold of the load of the activated ones of the power supplies and, in response to the determination, enabling another deactivated one of the power supplies.

[0020] According to some embodiments of the present invention, a method for managing a storage system is provided, the storage system comprising one or more storage devices and multiple power supplies configured to supply power to the storage devices, the method comprising: determining, by a processor of the storage device, whether multiple power supplies among the multiple power supplies are activated; in response to determining that multiple power supplies are activated: determining, by the processor, the total power consumption of the one or more storage devices; in response to determining that the total power consumption is less than a first percentage threshold of the load of activated power supplies among the power supplies, deactivating, by the processor, the activated power supplies among the power supplies one by one until the total power consumption is equal to or greater than the first percentage threshold of the load of each of the activated power supplies among the power supplies; and in response to determining that the total power consumption is equal to or greater than a second percentage threshold of the load of each of the activated power supplies among the power supplies, activating, by the processor, the deactivated power supplies among the power supplies one by one until the total power consumption is less than the second percentage threshold of the load of each of the activated power supplies among the power supplies.

[0021] According to some embodiments of the present invention, a storage system is provided, comprising: a plurality of storage devices, each of the plurality of storage devices being configured to measure the power consumption of the storage device; a processor communicating with the plurality of storage devices; and a memory having instructions stored thereon, which, when executed by the processor, causes the processor to execute: determining whether one or more first storage devices among the plurality of storage devices are idle or in an idle state; in response to determining that the one or more first storage devices are in an idle state, instructing the one or more first storage devices to operate at a lower power upper limit; determining whether one or more second storage devices among the plurality of storage devices are consuming power below a threshold power level; and in response to determining that the one or more second storage devices are consuming power below the threshold power level, instructing the one or more second storage devices to operate at or below the threshold power level.

[0022] In some embodiments, determining whether one or more first storage devices are in an idle state: obtaining the power consumption of each of the multiple storage devices by retrieving a corresponding power log from the storage device; comparing the power consumption of each storage device with an idle power level; and determining whether the one or more first storage devices have a power consumption at or below the idle power level.

[0023] In some embodiments, the power log stores actual power consumption of the corresponding storage device measured by the corresponding power meter.

[0024] In some embodiments, instructing the one or more first storage devices to operate at a lower power cap includes instructing the one or more first storage devices to change a power state to a power state having a lower maximum power rating.

[0025] In some embodiments, determining whether the one or more second storage devices among the multiple storage devices are consuming power below a threshold power level includes: obtaining the power consumption of each storage device among the multiple storage devices by retrieving a corresponding power log from the storage device; comparing the power consumption of each storage device with the threshold power level; and determining whether the one or more first storage devices have power consumption below the threshold power level.

[0026] In some embodiments, instructing the one or more second storage devices to operate at or below the threshold power level includes instructing the one or more second storage devices to change a power state to a power state having a maximum rated power corresponding to the threshold power level.

[0027] In some embodiments, the instructions further cause the processor to perform: determining whether one or more storage slots are not occupied by any storage device; and in response to determining whether the one or more storage slots are not occupied by any storage device: identifying one or more power meters associated with the one or more storage slots; and instructing the identified one or more power meters to operate at a lower power cap.

[0028] In some embodiments, instructing the identified one or more power meters to operate at the lower upper power limit includes instructing the one or more power meters to operate at a lowest power state.

[0029] In some embodiments, instructing the identified one or more power meters to operate at the lower power cap includes instructing the one or more power meters to deactivate.

[0030] According to some embodiments of the present disclosure, a storage system is provided, comprising: at least one storage device, a storage device of the at least one storage device comprising a power meter configured to measure the power consumption of the storage device; a processor, communicating with the at least one storage device; and a memory, having instructions stored thereon, which, when executed by the processor, cause the processor to execute: determining that one or more first storage devices of the at least one storage device are idle or in an idle state, and operate in a first power state; and at least partially based on determining that the one or more first storage devices are in an idle state, instructing the one or more first storage devices to operate in a second power state, the second power state being associated with a lower power range, having the same exit delay as the first power state, and having a higher entry delay than the first power state, wherein the one or more first storage devices comprise non-volatile memory.

[0031] According to some embodiments of the present disclosure, a storage system is provided, comprising: at least one storage device, the storage device of the at least one storage device comprising a power meter, the power meter being configured to measure one or more metrics indicating the health of the at least one storage device; and a processor, communicating with the at least one storage device and being configured to execute: receiving from the device an instruction for operating one or more first storage devices of the at least one storage device in a first power state, the first power state being associated with a first power range, wherein, in the first power state, the one or more first storage devices operate in an idle state during a first time period; instructing the one or more first storage devices to operate in the first power state; receiving from the device an instruction for operating the one or more first storage devices in a second power state, the second power state being associated with a second power range, wherein, in the second power state, the one or more first storage devices operate in a state different from the idle state during a second time period; and instructing the one or more first storage devices to operate in the second power state, wherein the second power state exceeds the first power state, has the same exit delay as the first power state, and has a lower entry delay than the first power state, and wherein the second power range causes the one or more first storage devices to have higher power consumption than the first power range. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Further features and aspects will become apparent and will be best understood by reference to the following detailed description reviewed in conjunction with the accompanying drawings, in which:

[0033] Figure 1 is an internal block diagram of a storage device according to an embodiment of the present invention.

[0034] Figure 2 It is used to collect Figure 1 A flow chart of a method for storing power consumption measurement results of a power measurement unit in a storage device.

[0035] Figure 3 is a schematic diagram of a storage system incorporating multiple storage devices capable of providing power measurement.

[0036] Figure 4 The PCIe switch is used in Figure 3 A block diagram of an embodiment of a storage system.

[0037] Figure 5 is a diagram depicting an embodiment in which power measurements are communicated to a local service processor based on a query from the local service processor.

[0038] Figure 6 is a diagram depicting an embodiment in which power measurements are set by a local service processor.

[0039] Figure 7 An example of a power policy is shown that may be used by the local service processor 50 to control the power consumption of the storage device.

[0040] Figure 8 is a diagram depicting an embodiment in which power measurements are stored in a controller memory buffer until retrieved by a local service processor.

[0041] Fig. 9 is a diagram depicting an embodiment in which power measurements taken by a power measurement unit are directly accessible to a local service processor.

[0042] Fig.10 is an example of a power log according to an embodiment of the present invention.

[0043] Fig.11 How the storage system is used Fig.10 An illustrative method for managing power reporting for multiple storage devices in a rack in a power log.

[0044] Fig.12 is a block diagram illustrating a storage system using a storage bank and a power distribution unit according to some exemplary embodiments of the present invention.

[0045] FIG. 13A to FIG. 13D Some exemplary embodiments according to the present invention present a histogram of the power consumption of a storage system as generated by a local service processor.

[0046] Fig.14 is a flow chart illustrating a process of managing power supply of a storage system according to some exemplary embodiments of the present invention.

[0047] Fig.15is a flow chart showing a process of managing storage devices of a storage system according to some exemplary embodiments of the present invention. DETAILED DESCRIPTION

[0048] Hereinafter, example embodiments will be described in more detail with reference to the accompanying drawings, and in all the accompanying drawings, the same reference numerals refer to the same elements. However, the present invention can be implemented in various different forms, and should not be construed as being limited to the embodiments shown herein. On the contrary, these embodiments are provided as examples so that this disclosure will be thorough and complete, and will fully convey the aspects and features of the present invention to those skilled in the art. Accordingly, unnecessary processes, elements, and technologies for those skilled in the art to fully understand the aspects and features of the present invention may not be described. Unless otherwise specified, the same reference numerals represent the same elements in all the accompanying drawings and textual descriptions, and therefore will not be repeated to describe them. In the accompanying drawings, the relative sizes of elements, layers, and regions may be exaggerated for clarity.

[0049] It will be understood that when an element or layer is referred to as being "on," "connected to," or "coupled to" another element or layer, it can be directly on, connected to, or coupled to the other element or layer, or there may be one or more intervening elements or layers. Additionally, it will be understood that when an element or layer is referred to as being "between" two elements or layers, there may be only one element or layer between the two elements or layers, or there may also be one or more intervening elements or layers.

[0050] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the present invention. As used herein, the singular forms "a", "an" and "an" are intended to include the plural forms as well, unless the context clearly indicates that the plural forms are not included. It will be further understood that when used in this specification, the terms "comprises", "comprising", "includes" and "including" specify the presence of the features, integers, steps, operations, elements and / or components described, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. As used herein, the term "and / or" includes any and all combinations of one or more of the listed associated items. When located before a list of elements, expressions such as "at least one" modify the entire list of elements and modify the individual elements in the list.

[0051] As used herein, the terms "substantially," "approximately," and the like are used as approximate terms and not as terms of degree, and are intended to account for the inherent deviations of measured or calculated values ​​that will be appreciated by those skilled in the art. Further, "may" is used when describing embodiments of the present invention to refer to "one or more embodiments of the present invention." As used herein, the terms "use," "using," and "used" may be considered synonymous with the terms "utilize," "using," and "utilized," respectively. Likewise, the term "exemplary" is intended to indicate an example or display.

[0052] Any suitable hardware, firmware (e.g., application specific integrated circuits), software, or a combination of hardware, firmware, and software may be used to implement the electronic or electrical devices and / or any other related devices or components according to the embodiments of the present invention described herein. For example, the components of these devices may be formed on an integrated circuit (IC) chip or on separate IC chips. Further, the components of these devices may be implemented on a flexible printed circuit film, a tape carrier package (TCP), a printed circuit board (PCB), or formed on a substrate. Further, the components of these devices may be processes or threads operating on one or more processors in one or more computing devices, thereby executing computer program instructions and interacting with other system components to perform the functions described herein. The computer program instructions are stored in a memory, which may be implemented in a computing device using a standard memory device such as a random access memory (RAM). The computer program instructions may also be stored in other non-transient computer-readable media such as a CD-ROM, a flash drive, etc. Similarly, it should be understood by those skilled in the art that the functions of the various computing devices may be combined or integrated into a single computing device, or the functions of a specific computing device may be separated across one or more other computing devices without departing from the spirit and scope of the exemplary embodiments of the present invention.

[0053] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. It will be further understood that terms (such as those commonly defined in dictionaries) should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and / or this specification, and should not be interpreted in an idealized or overly formal sense, unless clearly defined herein.

[0054] Embodiments of the present invention include storage devices, such as SSDs (e.g., NVMe or NVMe-oF SSDs), that are capable of reporting their actual power consumption to a local service processor, such as a baseline board management controller (BMC). This enables the local service processor to provide a power profile and power consumption of the storage device. In some embodiments, the storage device may report to the local service processor or BMC via a system management bus (SMBus) or a peripheral component interconnect high speed (PCIe), and may report via one of a variety of protocols, such as via a management component transport protocol (MCTP) or via an NVMe management interface protocol for NVMe SSD storage devices. In some embodiments, the storage system may be an NVMe-oF-based system. Further embodiments include a storage system comprising a plurality of storage devices, each of which is capable of reporting its actual power consumption to a local service processor. In such a system, the local service processor may provide a power profile and analysis of the storage system and individual storage devices in the system.

[0055] Figure 1 An internal block diagram of a storage device 10 according to an embodiment of the present invention is depicted. Although the figure depicts features related to the illustrated embodiment of the present invention, the storage device 10 may include additional components. In some embodiments, the storage device 10 may be an SSD, an Ethernet SSD (eSSD), an NVMe SSD, an NVMe-oF SSD, a SAS, or a SATA SSD.

[0056] The storage device 10 includes internal components, including a controller 11, a memory 12, a flash die 13, a power metering unit (PMU) 14, and a connector 15. The controller 11 (also called a processor) implements firmware to retrieve and store data in the memory 12 and the flash die 13, and communicate with a host computer. In some embodiments, the controller 11 can be an SSD controller, an ASIC SSD controller, or an NVMe-oF / edge SSD controller. The memory 12 can be a random access memory such as DRAM or MRAM, and the flash die 13 can be a NAND flash memory device, although the present invention is not limited thereto. The controller 11 can be connected to the memory 12 via a memory channel 22, and can be connected to the flash die 13 via a flash channel 23. The controller 11 can communicate with the host computer through a host interface 20, which connects the controller 11 to the host computer through a connector 15. In some embodiments, the host interface 20 can be a PCIe connection, an Ethernet connection, or other appropriate connection. The connector 15 can be a U.2 / M.2 connector or other suitable (multiple) connectors. The PMU 14 allows the storage device 10 to support power management capabilities by measuring the actual power consumption of the storage device 10 .

[0057] The storage device 10 is powered by a power rail or pin 30 through the connector 15. In the example where the connector 15 is a PCIe connector, the pins 30 may be 12V and 30V pins. In the example where the connector 15 is a U.2 connector, the pins 30 may be 5V and 12V pins (NVMe SSDs may use only 12V pins, while SAS or SATA SSDs may use both rails). The power rail 30 provides power to the components of the storage device 10. For example, the power rail may provide power to the components of the storage device 10 through the PMU 14 and the intermediate voltage rails. Figure 1 An embodiment thereof is shown in FIG. 1 , in which a power rail 30 provides power to a PMU 14, which then distributes the power to other components of the storage device 10. For example, the PMU 14 drives power to the flash die 13 via a flash voltage rail 33. The PMU 14 may similarly drive power to the memory 12 via a memory voltage rail 32. Power may be supplied to the controller 11 by the PMU 14 via a plurality of voltage rails, such as a core voltage rail 34, an I / O voltage rail 35, and one or more other voltage rails 36. Additional voltage rails, such as an additional voltage rail 37, may be included to connect other components that may be included in the storage device 10. When the storage device 10 is an NVMe SSD, the various voltage rails 30, 33, 34, 35, 36, 37 used in the storage device 10 may be in the range from 12V down to 0.6V, including, for example, 12V and / or 3.3V rails. Although, in Figure 1 In the embodiment shown in , the voltage regulator is built into the PMU 14 (or integrated with the PMU 14 ), but the embodiments of the present invention are not limited thereto, and the voltage regulator may be outside the PMU 14 .

[0058] In addition to powering the storage device 10, the power rail 20 is provided by the PMU 14 inside the storage device 10, thereby generating power consumption measurements ("power measurements") of various voltage rails used by components of the storage device 10 (e.g., components such as the controller 11, flash die 13, memory 12, and other components that may be included in the storage device 10). In some embodiments, the PMU 14 may be programmed to support obtaining / setting power states via power information from a host computer or BMC.

[0059] The PMU 14 may measure the amount of current drawn on each voltage rail (e.g., voltage rails 32, 33, 34, 35, 36, and 37) that it is driving. The PMU may output power measurements, including the average, minimum, and maximum voltage usage of the voltage rails 32, 33, 34, 35, 36, and 37 of the storage device 10. In some embodiments, the PMU 14 may meter each voltage rail 32, 33, 34, 35, 36, and 37 individually, where the sum of all voltage rails 32, 33, 34, 35, 36, and 37 used by the storage device 10 is the total power consumed by the storage device 10. The power measurements metered at the PMU 14 may be read by the controller 11 using the PMU / controller interface 41. In some embodiments, the PMU / controller interface 41 may be an I2C / SMBus. The controller 11 may then provide these power measurements to the local service processor 50 (see FIG. 1 ) via either the host interface 20 or a separate controller / host interface 42. Figure 3 ), such as BMC. If a separate controller / host interface or sideband bus 42 is used, this interface can be I2C / SMBus. If the controller / host interface 42 is a PCIe connection, the controller 11 can provide power measurements to the local service processor 50 via NVMe-MI or MCTP protocols, such as Figure 4 The PMU 14 may report / output power measurements at a periodicity specified by the local service processor 50 or may be tracked passively via an internal counter accessible to the local service processor 50 .

[0060] Figure 2 1 is a flow chart of a method for collecting power consumption measurements from a PMU 14 of a storage device 10. Figure 2 As shown in , the power measurements may be read at predetermined intervals. For example, the power measurements may be read from the PMU 14 of the storage device 10 at a user-configurable frequency (e.g., 1 second, 5 seconds, more than 5 seconds, or every few minutes). In other embodiments, the storage device 10 may read the power measurements only when needed (see, e.g., Figure 8 and Fig. 9 ), for example, when a specific job is completed. The frequency with which the power measurement results are read is referred to as a time unit in the following.

[0061] For each time unit, the controller 11 prepares (S1) to receive power measurements for each voltage rail 30, 33, 34, 35, 36, 37 from the PMU 14. The controller 11 queries (S2) the PMU 14 to determine whether the power measurements from all rails have been completed. If not, a read request (S3) is sent to the DC-DC regulator corresponding to the voltage rail at the PMU 14 for which the power measurement has not yet been received (the PMU 14 may include multiple DC-DC regulators, each corresponding to a unique voltage rail). This read request may be sent via the PMU / controller interface 41 via the I2C protocol. When the power measurement is received from the PMU 14, the power measurement is annotated with a timestamp (S4) and a host ID (S5). The received power measurement is then saved (S6) to a power log. The power log may include (multiple) internal registers or may be included as part of the embedded non-volatile memory of the PMU.

[0062] Once the received power measurements are saved, the PMU 14 is polled again (S7) until all power measurements have been received from each voltage rail 30, 33, 34, 35, 36, 37. Once all power measurements are completed and the annotated power measurements are saved in the power log, these power measurements are retained (S8) in the power log through resets and power cycles.

[0063] In addition to the above annotations, the power log page may also include any or all of the following: Namespace ID, NMV Set, Read I / O, Write I / O, SQ ID, Stream ID, and other appropriate parameters. The controller 11 also implements an actual power (AP) register accessible by the local service processor 50. This allows the various parameters associated with the storage device and the power measurement results to be mapped at a fine granularity.

[0064] In some embodiments, the power log may be a special owner or vendor defined log page. The power log may be read by the local service processor 50 using existing standard protocols through either the host interface 20 or a separate controller / host interface or the sideband bus 42 (whichever is used). For example, the power log may be read by the BMC using the NVMe-MI protocol through the controller / host interface 42 (which may be SMBus or PCIe).

[0065] The above method provides a dynamic real-time output of actual power consumption measurement results without affecting the I / O of the storage device 10. With the power measurement information, the local service processor can implement a power budget and allocate power to the storage device 10 based on its actual power usage. For example, the local service processor can implement a power budget similar to the existing industry standard for the allocated power budget register. Similarly, the storage device 10 can report real-time power consumption to the system management software (DCP or Redfish of Samsung).

[0066] Figure 3 1 is a block diagram of a storage system 100 incorporating a plurality of storage devices 10. The storage system 100 includes a local service processor 50 attached to the plurality of storage devices 10. Each storage device 10 has a PMU 14 such that Figure 1 and Figure 2 The power consumption is measured as described. In the illustrated embodiment, the storage device 10 provides power measurements to the local service processor 50 via the controller / host interface 42. In some embodiments, the controller / host interface 42 may be an I2C / SMBus or PCIe bus. The power measurements may be communicated to the local service processor 50 using an NVMe protocol, such as the NVMe-MI protocol, the MCTP protocol for PCI-e, or the I2C bus protocol. If the storage device 10 is connected via an SMBus / I2C connection, the local service processor 50 may access the power log using these existing standard protocols even during a power failure.

[0067] Figure 4 In which the PCIe switch 60 is used, Figure 3 1 is a block diagram of an embodiment of a storage system 100. In this embodiment, the storage device 10 is connected to the local service processor 50 via a PCIe switch 60. The power measurement results can be transmitted to the local service processor 50 via the PCIe switch 60 using a suitable protocol such as NVMe-MI and / or MCTP.

[0068] exist Figure 3 and Figure 4 In the embodiment of the present invention, the local service processor 50 and the plurality of storage devices 10 can be housed in the same rack, thereby allowing the local service processor 50 to process the power requirements of the plurality of storage devices 10 according to the rack power management requirements; however, the present invention is not limited thereto. For example, the power measurement results can also be processed at the individual storage device level.

[0069] In an embodiment where the NVMe protocol is used to pass the power measurement results to the local service processor 50, the NVMe specification may define a power measurement and processing mechanism. Based on this mechanism, the storage device 10 (e.g., NVMe SSD) may support power management (or power measurement) or be controlled by the local service processor 50 ( Figure 5 ) or by the local service processor 50 ( Figure 6 )set up.

[0070] Figure 5 is a diagram depicting an embodiment in which power measurement results are communicated to the local service processor 50 based on a query from the local service processor 50. In this embodiment, the local service processor 50 queries the power measurement information by sending a GetFeature command (S10) (e.g., Feature ID=0x2) to the firmware of the controller 11 of the storage device 10 from which the local service processor 50 is seeking power measurement information. The firmware of the controller then retrieves (S11) the power measurement information from the PMU 14. The firmware of the controller 11 receives the information and sends (S12) the information to the local service processor 50 via direct memory access (DMA). The firmware of the controller then sends (S13) a completion notification to the local service processor 50 to indicate that the query is complete. This embodiment allows real-time retrieval of power measurement results from the storage device 10.

[0071] Figure 6 is a diagram depicting an embodiment in which power measurement is set by a local service processor 50. In this embodiment, the local service processor 50 sets power measurement information (hereinafter referred to as a power measurement budget) by sending a SetFeature command (S20) (e.g., Feature ID=0x2) to the firmware of the controller 11 of the storage device 10 for which the local service processor 50 intends to set a power measurement budget. The firmware of the controller then requests (S21) a power measurement budget from the local service processor 50 using DMA. The firmware of the controller 11 receives the information and sets (S22) the power measurement budget of the PMU 14. In response, the firmware of the controller processes a new power state transaction. In order to process the new power transaction, the firmware of the controller queries the current power state work in the PMU 14 to ensure that all tasks that rely on the current power state are fully and successfully completed. The firmware then changes the current power state from the current power state to the next power state required by the power measurement budget. The firmware of the controller begins to process the new task that relies on the power state using the allocated power measurement budget. The controller's firmware then sends (S23) a completion notification to the local service processor 50 to indicate that the new power state has been set.

[0072] By enabling this SetFeature function, the local service processor 50 can control and suppress the power consumption of a specific storage device 10 to meet the allocated power budget of the local service processor 50. The controller 11 can execute the power budget allocation programmed by the local service processor 50. If the actual power consumption exceeds the set threshold, the controller 11 can suppress I / O performance for this parameter in order to minimize power consumption and stay within the allocated power budget. When the allocated power budget is exceeded, the controller 11 can self-adjust, for example, by automatically reducing the internal power state. The controller 11 can then report to the local service processor 50 so that the local service processor 50 can reallocate available power to certain other devices that may require additional power. The controller 11 can also collect statistics on this performance suppression at a fine granularity.

[0073] Figure 7 An example of a power policy is shown that can be used by the local service processor 50 to control the power consumption of the storage device 10. The local service processor 50 can manage the power policy by monitoring each storage device 10 in the storage system and instructing each storage device 10 to maintain its corresponding allocated power budget. For example, if the storage device 10 changes from normal operation 61 to operating at more than 90% of its allocated power budget, as shown at 62, the controller 11 can throttle the I / O performance by, for example, introducing a small percentage of additional delay (e.g., 10% or 20% idle or overhead). However, if the current state is greater than 100% of its allocated power budget, as shown at 63, the controller 11 can introduce a much larger delay (e.g., 50% or more) or can introduce delays to NAND cycles, etc., in order to throttle the storage device 10 to meet its allocated operation. If the storage device 10 continues to exceed its allocated budget despite the introduction of delays, the local service processor 50 can execute a shutdown instruction 64 to shut down the storage device 10 or the controller 11 can shut itself down.

[0074] In further embodiments, the local service processor 50 may also monitor and detect increases in thermal load (temperature increases) and / or operate the resources during peak utilization periods (e.g., hot weather periods) or during brown-out periods to ensure that each storage device 10 performs as expected.

[0075] The above features enable the storage device 10 to autonomously optimize power vs. performance vs. assigned power budget / state.

[0076] Figure 8is a diagram depicting a further embodiment in which the power measurements are stored in a controller memory buffer until retrieved by the local service processor 50. In this embodiment, the controller 11 may store the power measurements locally in its own memory 12 until requested by the local service processor 50. For example, in an embodiment where the storage device 10 is an NVMe SSD, the controller 11 may store the power measurement information in a controller memory buffer of the memory 12. The NVMe specification defines a controller memory buffer (CMB), which is a portion of the storage device's memory but is assigned and logically owned by the host / local service processor.

[0077] The firmware of the controller 11 can retrieve the power measurement information from the PMU 14 and store it in the controller memory buffer of the memory 12. The controller memory buffer can be updated at any specified time unit. The local service processor 50 can then query the power measurement information by reading the power measurement results directly from the controller memory buffer of the memory 12. The power measurement results can be read from the controller memory buffer through the controller / host interface 42. If the controller / host interface 42 is PCIe, the power measurement information can pass through PCIe to process memRd / memWr directly based on the BAR configuration so as to read from the control memory buffer. In other embodiments, the power measurement information can directly access the control memory buffer through a sideband such as SMBus or I2C.

[0078] Alternative Figure 8 Advantageously, the storage device 10 may be configured such that the PMU 14 is directly accessible by the local service processor 50 so that the local service processor can access the power measurement information when desired / needed and in real time.

[0079] Fig. 9 1 is a diagram depicting an embodiment in which power measurements taken by PMU 14 are directly accessible to local service processor 50. In this embodiment, storage device 10 may be configured with an auxiliary bus, such as I2C or AXI, to allow direct access to PMU 14 by local service processor 50. This allows local service processor 50 to process power measurement information by directly accessing PMU 14 and allows real-time retrieval of power measurements.

[0080] Fig.10is an example of a power log 70 according to an embodiment of the present invention. As shown in this embodiment, the storage device 10 may have, for example, up to 32 power states (PowerState) 71, which are recorded in the power log 70. Each power state 71 has predefined performance information, the maximum power (MP) 72 that can be used in this power state 71, and the actual power (AP) 73 actually used in this power state. AP 73 is a measured time period based on a time unit (e.g., 1 minute) and workload / QoS. In the current embodiment, each row in the power log 70 represents a power state that has been defined in the NVMe specification 1.3. For example, a total of 32 power states are defined in the NVMe specification. In some embodiments, a vendor-specific definition may be used for each power state 71.

[0081] The power log 70 may include in its table entries various power states 71, and for each power state corresponding MP 72, AP 73, and additional information identifying power measurements and the relationship between maximum power / power state, actual power, and QoS. The QoS information may include, for example, current entry latency (ENTLAT), current exit latency (EXTLAT), RRT (relative read throughput), RWT (relative write throughput), and other suitable variables.

[0082] Fig.10 Power state_3 is shown, where the defined maximum power = 20 W. However, the storage device 10 in this power state currently consumes actual power = 19 W. The current QoS is shown in other columns, such as RRT = 2, RWT = 2, ENTLAT = 20 us, and EXTLAT = 30 us. If the applications 80 running on the storage system 200 expect the best QoS (e.g., the best RRT and RWT), these applications 80 can instruct the local service processor 50 to give more power to the storage device 10 by transferring from power state_3 to power state_0.

[0083] The current power state 71 is retrieved by the local service processor 50 by getting a feature (GetFeature) (feature ID=0x2), such as Figure 5 The desired power state (ie, power measurement budget) may be set by the local service processor 50 by setting a feature (SetFeature) (Feature ID = 0x2), as described in relation to Figure 6Other power related information can be managed by the local service processor 50 through VUCmd (Vendor Unique Cmd) or accessed directly through the local service processor 50. For example, if a user wants to get power measurement information that is not defined in the NVMe specification, VUCmd can be used to allow the host to retrieve such non-standard power information, similar to the log page.

[0084] Fig.11 is an illustrative method of how a storage system 200 manages power reporting for multiple storage devices 10 in its rack. According to this method, each PMU 14 of each storage device 10 measures the current AP 73 and stores the information in a power log 70, which is queried and / or retrieved by the local service processor 50 (S50). The local service processor 50 then updates / uploads (S51) the power log 70 from the local service processor 50 to the storage system 200. Each application 80 in the storage system 200 can analyze (S52) the power logs 70 of the storage devices 10 in the rack at the local service processor 50. The results of these analyses can determine how to allocate power to obtain better performance, for example, whether more power needs to be allocated to a specific power state 71 or whether power needs to be reallocated from one power state 71 to another power state to meet QoS requirements. For example, the local service processor 50 can request (S53) the storage device 10 (such as regarding Fig.10 ) transfers the maximum power state (in this example) from power state 3 to power state 0. The local service processor 50 may then either assign a new MP 72 to the storage device 10 or may request (S54) a power distribution unit (PDU) 90 to assign a new MP 72 budget to the storage device 10, i.e., redistribute the power allocation. If a PDU 90 is used, the PDU 90 will assign (S55) a new MP 72 to the storage device 10. The PDU 90 may be a separate component located within the rack and may be responsible for distributing the MPs to each storage device 10. The local service processor 50 then updates (S56) the power log 70 with these changes.

[0085] As discussed above, once the local service processor 50 has access to and can read the power measurements, the local service processor 50 can use this information to create graphs or histograms for trending and to run diagnostics.

[0086] Embodiments of the present invention also enable the local service processor to provide individual actual power profiles of each storage device in the system to software developers, cloud service providers, users, and others by allowing them to understand the actual power consumption consumed by their workloads on each storage device. This provides software developers / users with the ability to optimize performance based on actual energy costs, and also allows cloud service providers to provide more accurate billing to storage system users based on actual power consumption. Embodiments of the present invention can also provide better supervision and tracking of storage devices that violate the allocated power budget.

[0087] Embodiments of the present invention can be used in a variety of fields. For example, embodiments of the present invention provide components with key information that can be used for analysis purposes in artificial intelligence software (e.g., Samsung's DCP). Embodiments also provide information that may be useful for systems based on ADRC (Active Disturbance Rejection) efficient thermal control.

[0088] Although exemplary embodiments of the present invention have been described, it is to be understood that the present invention should not be limited to these exemplary embodiments, but various changes and modifications may be made by those skilled in the art within the spirit and scope of the present invention as hereinafter claimed in the appended claims and their equivalents.

[0089] Fig.12 is a block diagram illustrating a storage system 300 utilizing a storage bank 302 and a power distribution unit (PDU) 90 according to some exemplary embodiments of the present invention.

[0090] In some embodiments, a storage library (e.g., an Ethernet SSD rack or a Just-a-bunch-of-flashes JBOF) 302 includes a plurality of storage devices 10, and the PDU 90 includes a plurality of power supply units (PSUs or power supplies) 304 to power the storage devices 10 of the storage library 302 under the direction of a local service processor (or BMC) 50. In some embodiments, the PSUs 304 are interchangeable, i.e., each may have the same form factor and the same power capacity (e.g., having the same output wattage); however, embodiments of the present invention are not limited thereto, and one or more of the PSUs 304 may have a different power capacity than the other PSUs 304. In one example, the plurality of PSUs 304 may have an N+1 configuration, wherein N (an integer greater than or equal to 1) PSUs are sufficient to serve the power needs of the storage library 302, and additional PSUs 304 are provided as redundancy, which may be activated in the event that any one of the PSUs experiences a failure.

[0091] like Fig.12 As shown in , in some embodiments, the PSUs 304 may be coupled together using a switch network (e.g., a FET network) 305, rather than being directly connected to the power bus 306, in order to protect the power bus 306 from electrical shorts and transients when other PSUs 304 are connected. The switch network may include a plurality of switches (e.g., transistors) connected to the plurality of PSUs 304 at one end and to the power bus 306 at the other end. According to some embodiments, the switches are independently controlled by a local service provider (BMC) 50, so that any one of the PSUs 304 may be connected to or disconnected from the power bus 306 based on a control signal from the local service provider 50.

[0092] According to some embodiments, each storage device 10 is configured to report its actual power consumption to the local service processor 50 via, for example, SMBus or PCI-e and by means of, for example, NVMe-MI or MCTP protocols. The actual power consumption is measured by a PMU (i.e., a power meter) 14, which may be inside (e.g., integrated inside) the storage device 10 (e.g., Fig.12 ) or external to but coupled to the storage device 10. The power consumption reporting enables the local service processor 50 to provide a power profile and analyze the storage repository 302, which in turn can be used for diagnosis and to provide value-added services. This also allows each storage device 10 to more flexibly manage its own power usage via the local service processor 50 as directed by the administrator 308.

[0093] FIG. 13A to FIG. 13D Some exemplary embodiments according to the present invention present a histogram of the power consumption of the storage system as generated by the local service processor 50 .

[0094] According to some embodiments, the local service processor 50 periodically reads power measurements from the storage devices 10. According to some examples, in doing so, the local service processor 50 may read the power log 70 page using the NVMe-MI protocol on SMBus or PCIe. The local service processor 50 may then process the read power data to generate power usage trends, such as the overall power usage of the storage repository 302 over time (e.g., hourly, during the day, night, weekdays, weekends, etc.), the power consumption of each storage device 10 over time, the relative power consumption of the storage devices 10 within the storage repository 302, and the like. In addition, the local service processor 50 may generate a number of derivative / additional graphs to understand power consumption behavior relative to time, users, activities, and the like. The local service processor 50 may also use this data for diagnostic purposes, power supply, future needs, cooling, and planning, and the like.

[0095] As an example, Fig.13A The power consumption of a single memory device 10 over time is shown. Fig.13A , the Y-axis represents power consumption in watts, and the X-axis represents time in hours.

[0096] In some embodiments, the local service processor 50 manages host access policies and receives raw power data and host IDs for activated storage devices. Thus, according to some embodiments, the local service processor 50 recognizes / understands which host or application is accessing each storage device 10 at any given time, and can combine this information with power usage metrics to provide a snapshot of the power consumption of each host or application. This information can provide each application with a deeper insight into storage power requirements and can be used to more accurately calculate storage costs for each host or application.

[0097] As an example, Fig. 13B The power consumption of different hosts or applications is shown. Fig. 13B , the Y-axis represents the average power consumption in watts over a period of time (eg, hourly, daily, etc.), and the X-axis represents the host ID or application ID.

[0098] According to some embodiments, the local service processor 50 can use power usage metrics for diagnostic purposes. In some embodiments, when abnormal power consumption is observed for the storage device 10, the local service processor 50 can alert the storage administrator 308. Abnormal power consumption may be the result of a fault within the storage device 10, or may be caused by abnormal activity of the host or application that is accessing the storage device 10. For example, the fault may be the result of a flash die or flash channel failure, which can initiate a RAID-like recovery mechanism to consume excess power; or the result of a higher bit rate error in the medium or volatile memory, which may cause the error correction algorithm to not converge and spend more time and energy on the process. The local service processor 50 can query the storage device health and status log (e.g., SMART log) and proprietary diagnostic logs to evaluate abnormal behavior. Based on the policy set by the administrator 308, some of the abnormal behaviors can be alerted to the administrator 308 to take further action.

[0099] For example, Fig. 13C The present invention shows a potential fault detected in the storage device 10 when the power consumption per hour suddenly rises from a normal level (e.g., 3 to 10 W / hour) to a maximum value (e.g., about 25 W). Fig. 13C, the Y axis represents the average power consumption in watts, and the X axis represents the time in hours. Thus, in some embodiments, the criterion for fault detection may be that the derivative of the power consumption is greater than a set threshold. However, embodiments of the present invention are not limited thereto, and the actual power consumption may be measured relative to the storage device performance to determine whether a fault has occurred. In some examples, the fault detection criterion / strategy may be set by the administrator 308.

[0100] further, Fig.13D An example is shown in which a potential failure is detected in a storage device 10 (e.g., the storage device in slot #8). In this example, the storage device 10 may be expected to consume a maximum power of approximately 25 W at 1 MIOPS (one million input / output operations per second) performance. However, if the average power consumption of the storage device in slot #8 reaches a maximum power of approximately 25 W, but the average performance is much less than 1 MIOPS, the local service processor may mark the storage device in slot #8 as potentially faulty or at least a good candidate for further fault analysis.

[0101] Accordingly, aspects of the present invention provide building blocks with key information for analysis by other artificial intelligence SW. In addition, it also provides useful information for use by ADRC (active disturbance rejection control) efficient thermal control based systems.

[0102] Fig.14 is a flow chart illustrating a process 400 for managing the operation of a PDU 90 according to some exemplary embodiments of the present invention.

[0103] According to some embodiments, the local service provider 50 manages (e.g., optimizes) the operation of the PDU 90 by dynamically monitoring the operation of the PSUs 304 in the PDU 90 and ensuring that the active PSUs 304 operate within their high power efficiency range. In doing so, the local service provider 50 determines (S100) whether the PDU 90 includes multiple active PSUs 304. The active PSUs 304 may be connected to the power bus 306 through a switch network (i.e., turning on the corresponding switches), and the deactivated PSUs 304 may be disconnected from the power bus 306 (e.g., by turning off the corresponding switches). In some embodiments, the local service provider 50 determines the status of each PSU 304 in the PDU 90 through a bus (e.g., SMBus / PMBus), and is thus able to determine the number of PSUs 304 at the PDU 90. In some examples, the local service provider 50 reads the PSU status register of each PSU 304 present in the PDU 90 to determine its status (i.e., activated / enabled or deactivated / disabled). If there is only one active PSU 304, the local service provider 50 determines (S114) whether this active PSU 304 is the only existing PSU 304 and is in HA mode (see below for more on this). Otherwise, the local service provider 50 determines (S102) whether the total power consumption of the storage library 302 is less than a first percentage threshold (e.g., 40%, or a value between 30% and 50%) of the load of each of the active PSUs 304. In some embodiments, the local service provider 50 does this by obtaining the actual power consumption of each storage device 10 (as measured by the corresponding PMU 14) and adding up the actual power consumption. In some examples, local service provider 50 may obtain actual power consumption of each storage device 10 by querying / retrieving power log 70 from storage device 10 or PMU 14 corresponding to storage device 10 (which may be internal or external to storage device 10 ).

[0104] If the total power consumption is less than the first percentage threshold of the load of each of the active PSUs 304, the active PSUs 304 may be operated in a low power efficiency mode, which may be undesirable. Thus, the local service provider 50 disables the active PSUs 304 (S104), waits (S106) for a period of time (e.g., a few seconds or minutes), and rechecks (S102) whether the total power consumption of the storage bank 302 is still less than the first percentage threshold of the load of each of the active PSUs 304. If so, the loop continues, and the local service provider 50 continues to disable the active PSUs 304 one by one until the total power consumption is equal to or greater than the first percentage threshold of the load of each of the active PSUs 304.

[0105] At this point, the local service provider 50 determines (S108) whether the total power consumption of the storage bank 302 is greater than a second percentage threshold (e.g., approximately 90%, or a value between 85% and 95%) of the load of each of the activated PSUs 304. If so, the activated PSUs 304 may operate in a high power state, which, if prolonged, may be detrimental to the life of the PSUs 304. Thus, the local service provider 50 enables (i.e., activates) the disabled (i.e., deactivated) PSUs 304 (S110), waits (S112) for a period of time (e.g., a few seconds or a few minutes), and rechecks (S108) whether the total power consumption of the storage bank 302 is still equal to or greater than the second percentage threshold of the load of each of the activated PSUs 304. If so, the loop continues, and the local service provider 50 continues to enable the activated PSUs 304 one by one until the total power consumption is less than the second percentage threshold of the load of each of the activated PSUs 304.

[0106] At this time, the local service provider 50 determines (S114) whether there is only one PSU 304 in the PDU 90 when the storage system 300 is in a high availability (HA) mode (which indicates a multipath IO mode and N+1 redundant PSUs). Generally, in the HA mode, the storage system 300 is in a multipath IO mode, and there are N+1 redundant PSUs to ensure that there is no single point of failure. In this way, when there is only one PSU 304 in the PDU 90 while the storage system 300 is in the HA mode, the local service provider 50 sends a warning (e.g., a serious warning) message to the administrator 308 (S116) so that another redundant PSU 304 is installed in the PDU 90. Otherwise, the system operates normally and no warning message is sent to the administrator 308.

[0107] Fig.15 is a flow chart illustrating a process 500 of managing storage devices 10 of a storage system 300 according to some exemplary embodiments of the present invention.

[0108] According to some embodiments, the local service provider 50 manages (eg, optimizes) the storage device 10 by dynamically adjusting (eg, reducing) its maximum power range or power upper limit (cap) based on the current workload of the storage repository 302 .

[0109] In some embodiments, the local service provider 50 identifies (S118) which storage devices 10 of the storage library 302 are in an idle state or consume near-idle power. Here, the idle state may refer to the following operating state: the storage device 10 does not have any active or outstanding host commands, such as read or write, in its command queue for a period of time. That is, the host command queue of the storage device controller is empty for a period of time, which is programmable (for example, by the administrator 308). Near-idle power can be any power consumption below a set threshold, which is programmable (for example, by the administrator 308). In some embodiments, the local service provider 50 obtains the actual power consumption of each storage device 10 by querying / retrieving the power log 70 from the storage device 10, and the actual power consumption is measured by the corresponding PMU 14. The local service provider 50 then compares the actual power consumption with the idle power level. If the power consumption of the storage device 10 is at or below the idle power level, the storage device is identified as being in an idle state. The local service provider 50 then instructs (S120) the identified storage devices 10 to operate at a lower power cap. For example, the local service provider 50 may instruct each of the identified storage devices 10 to change the power state to a power state with a lower maximum rated power (e.g., from power state 2 to power state 5). This may be done based on a power policy implemented by the local service provider 50 (and defined, for example, by the administrator 308), which associates each power state with a range of actual power consumption.

[0110] According to some embodiments, the local service provider 50 identifies (S122) which storage devices 10 consume power at a level below a threshold power level. In some examples, the threshold may be set at 75% of the maximum power (which may be 25W or 75W, etc.), depending on the type of PSU and / or the power connector used.

[0111] In some embodiments, the local service provider 50 obtains the actual power consumption of each storage device 10 by querying / retrieving the power log 70 from the storage device 10, which is measured by the corresponding PMU 14. The local service provider 50 then compares the actual power consumption with the threshold power level to determine whether the consumed power of the storage device 10 is below the threshold power level. The local service provider 50 then dynamically instructs the identified storage device 10 to operate at a power cap corresponding to a first level (e.g., 75% or 80% of maximum power), as opposed to the default power cap of 100% maximum power. Since the power efficiency of a PSU decreases when it reaches its maximum load capacity, lowering the power cap of the storage device 10 may reduce the overall power usage of the storage library 302, thereby allowing the PSU to operate at a lower power level and in a higher (e.g., peak) power efficiency range. This may be particularly popular in large data centers where overall power usage and cooling are of great concern.

[0112] In some examples, the local service provider 50 can dynamically instruct each of the identified storage devices 10 to operate at a lower power cap by instructing them to change their power state to a power state where the maximum power corresponds to (e.g., is at or less than) a threshold power level (e.g., the power state can be changed from power state 0 to power state 1).

[0113] In some embodiments, the local service provider 50 identifies (S126) which storage device slots are empty (i.e., not occupied by or connected to any storage device 10). In some examples, each storage device 10 may have a presence pin on the slot connector 15, which is used by the service provider 50 to determine whether the slot is empty or occupied by a storage device 10. If any of the empty slots has a corresponding PMU 14 external to its corresponding storage device 10 (i.e., not integrated with and external to it) (e.g., may be at a power distribution board or at a mid-plane of a storage rack), the local service provider 50 instructs (S128) these PMUs 14 to operate at a lower power cap (e.g., operate at a minimum power state (power state 31)) or disable / deactivate altogether. This will allow the storage library 302 to eliminate or reduce unnecessary power usage.

[0114] Although operations S118-S120, S122-S124, and S126-S128 are Fig.15For example, operations S118-S120 may be performed after any one or both of operations S122-S124 and S126-S128, and operations S126-S128 may be performed before any one or both of operations S118-S120 and S122-S124.

[0115] The operations performed by the local service provider 50 (e.g., processes 400 and 500) may be described as software routines executed by one or more processors in the local service provider 50 based on computer program instructions stored in a memory. However, it should be understood by those skilled in the art that the routines may be performed by hardware, firmware (e.g., by an ASIC), or a combination of software, firmware, and / or hardware. Furthermore, the order of the steps in the process is not fixed, but may be changed to any desired order as understood by those skilled in the art.

Claims

1. A storage system, comprising: at least one storage device, the storage device of the at least one storage device comprising a power meter configured to measure power consumption of the storage device; a processor in communication with the at least one storage device; and A memory having instructions stored thereon, which, when executed by the processor, cause the processor to execute: determining that one or more first storage devices of the at least one storage device are idle or in an idle state and are operating in a first power state; and Based at least in part on determining that the one or more first storage devices are in an idle state, instructing the one or more first storage devices to operate in a second power state, the second power state being associated with a lower power range, having the same exit latency as the first power state, and having a higher entry latency than the first power state, Wherein, the one or more first storage devices include a non-volatile memory.

2. The storage system according to claim 1, wherein: The one or more first storage devices are non-volatile memory high-speed NVMe solid-state drives SSDs; The processor is configured to receive power measurements from the one or more first storage devices using an NVMe protocol; Determining that the one or more first storage devices are in an idle state includes: obtaining power consumption of a storage device of the one or more first storage devices by retrieving a power log from the storage device; comparing the power consumption of the storage device to an idle power level; and determining that the storage device has a power consumption at or below the idle power level, The power log stores actual power consumption of the storage device measured by a corresponding power meter.

3. The storage system of claim 1 , wherein instructing the one or more first storage devices to operate in the second power state comprises: The one or more first storage devices are instructed to change a power state to a power state having a lower maximum power rating.

4. The storage system of claim 1 , wherein the instructions further cause the processor to execute: determining whether one or more second storage devices of the at least one storage device are consuming power below a threshold power level; and Based at least in part on determining that the one or more second storage devices are consuming power below the threshold power level, The one or more second storage devices are instructed to operate at or below the threshold power level.

5. The storage system according to claim 4, wherein: Determining whether the one or more second storage devices in the at least one storage device are consuming power below a threshold power level includes: obtaining power consumption of a storage device of the one or more second storage devices by retrieving a power log from the storage device; comparing the power consumption of the storage device to the threshold power level; and A determination is made as to whether the storage device has a power consumption below the threshold power level.

6. The storage system of claim 4, wherein instructing the one or more second storage devices to operate at or below the threshold power level comprises: The one or more second storage devices are instructed to change a power state to a power state having a maximum power rating corresponding to the threshold power level.

7. The storage system of claim 1, wherein the instructions further cause the processor to execute: determining whether one or more storage slots are not occupied by any storage device; and Based at least in part on determining that the one or more storage slots are not occupied by any storage device: identifying one or more power meters associated with the one or more storage slots; and The one or more power meters are instructed to remain activated and operate at the lower power cap.

8. A storage system comprising: at least one storage device, the storage device of the at least one storage device comprising a power meter configured to measure one or more metrics indicative of a health of the at least one storage device; and a processor, in communication with the at least one storage device, and configured to execute: receiving, from a device, an instruction to operate one or more first storage devices of the at least one storage device in a first power state, the first power state being associated with a first power range, wherein in the first power state, the one or more first storage devices operate in an idle state during a first time period; instructing the one or more first storage devices to operate in the first power state; receiving, from the device, an instruction to operate the one or more first storage devices in a second power state, the second power state being associated with a second power range, wherein in the second power state, the one or more first storage devices operate in a state different from the idle state during a second time period; and instructing the one or more first storage devices to operate in the second power state, wherein the second power state exceeds the first power state, has the same exit latency as the first power state, and has a lower entry latency than the first power state, and The second power range causes the one or more first storage devices to have higher power consumption than the first power range.

9. The storage system of claim 8, wherein the processor is configured to perform: determining one or more parameters of the one or more first storage devices, the one or more parameters being indicative of a health of the one or more first storage devices; and A health measure of the one or more first storage devices is determined based on the one or more parameters.

10. The storage system according to claim 9, wherein: The one or more first storage devices are non-volatile memory high-speed NVMe solid-state drives SSDs; and The processor is configured to receive the health measurement using the NVMe protocol.

Images