Extending the life of non-volatile memory by limiting the number of writes

Abstract

A method for limiting write cycles to a storage device includes tracking a first count of a first set of writes to memory performed by a first subsystem and determining that the first count satisfies a write threshold of the first subsystem. The method includes determining that a blocking criterion is met by the first subsystem based on the first count exceeding the write threshold of the first subsystem, and determining that the blocking criterion is not met by a second subsystem of the computing device. The method includes blocking the first set of writes from being synchronized to the storage device based on the blocking criterion being met by the first subsystem, and synchronizing memory contents of the second subsystem to the storage device based on the blocking criterion not being met by the second subsystem.

Description

[Technical field]

[0001] Technical Field Various implementations of the present invention relate to non-volatile memory, and more particularly, to limiting write cycles to extend the life of non-volatile memory. [Background technology]

[0002] background Flash storage is a form of non-volatile computer memory that can be electrically written and erased. Flash storage has various advantages over other forms of non-volatile memory. For example, flash storage is more economical, more reliable, and has lower power consumption requirements. It is currently widely used in smartphones, memory sticks, digital audio players, scientific instruments, and more. Summary of the Invention

[0003] overview In one implementation, a method of limiting write cycles to a storage device includes tracking a first count of a first set of writes to memory performed by a first subsystem of a computing device and determining that the first count satisfies a write threshold of the first subsystem. The method includes determining that a blocking criterion is met by the first subsystem based on the first count exceeding the write threshold of the first subsystem, and further determining that the blocking criterion is not met by a second subsystem of the computing device. The method includes blocking the first set of writes from being synchronized to the storage device based on the blocking criterion being met by the first subsystem, and synchronizing memory contents of the second subsystem to the storage device based on the blocking criterion not being met by the second subsystem.

[0004] In another embodiment, a computer program product for limiting write cycles to a storage device includes a computer readable storage medium having program instructions embodied thereon. The program instructions are executable by a processor to cause the processor to perform a method. The method includes tracking a first count of a first set of writes to memory performed by a first subsystem of a computing device, and determining that the first count satisfies a write threshold of the first subsystem. The method includes determining that a blocking criterion is met by the first subsystem based on the first count exceeding the write threshold of the first subsystem, and further determining that the blocking criterion is not met by a second subsystem of the computing device. The method includes blocking the first set of writes from being synchronized to the storage device based on the blocking criterion being met by the first subsystem, and synchronizing memory contents of the second subsystem to the storage device based on the blocking criterion not being met by the second subsystem.

[0005] In yet another embodiment, a computing device includes a processor and a memory. The processor is configured to execute computer-readable instructions, and the memory is configured to store computer-readable instructions that, when executed by the processor, cause the processor to perform operations. The operations include tracking a first count of a first set of writes to memory performed by a first subsystem of the computing device, and further determining that the first count satisfies a write threshold of the first subsystem. The operations include determining that a blocking criterion is met by the first subsystem based on the first count exceeding the write threshold of the first subsystem, and determining that the blocking criterion is not met by a second subsystem of the computing device. The operations include blocking the first set of writes from being synchronized to the storage device based on the blocking criterion being met by the first subsystem, and synchronizing memory contents of the second subsystem to the storage device based on the blocking criterion not being met by the second subsystem.

[0006] These exemplary aspects and features are mentioned not to limit or define the presently described subject matter, but to provide examples to aid in understanding the concepts described herein. Other aspects, advantages, and features of the presently described subject matter will become apparent after consideration of the entire application. [Brief description of the drawings]

[0007] These and other features, aspects, and advantages of the present disclosure will become better understood from the following detailed description when read in conjunction with the accompanying drawings.

[0008] [Figure 1] FIG. 1 is a conceptual diagram of a storage protection system for protecting the lifespan of a storage device in accordance with certain embodiments of the invention described herein. [Diagram 2]FIG. 2 is an architectural diagram of a computing device, specifically a utility meter, on which the storage protection system can execute, in accordance with certain embodiments of the present invention. [Diagram 3] FIG. 3 is a flow diagram of a method for managing writes to a storage device to protect the lifespan of the storage device, according to certain embodiments of the invention. [Figure 4] FIG. 4 is a flow diagram of a method for managing a blacklist of offending subsystems to protect the lifespan of a storage device according to certain embodiments of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Detailed Description

[0010] Devices utilizing flash storage technology, such as NAND and embedded multimedia controllers (eMMC), have a limited life span that is a function of the cumulative number of write and erase operations (i.e., write cycles) that have been performed. As write cycles are performed, flash storage ages and data written to the flash storage becomes corrupted. Specifically, the more writes are performed, the shorter the memory retention time (the time that data written to memory is valid). Eventually, if a block of flash storage experiences a large number of write cycles, that block may become unusable due to its extremely short retention period. This aging phenomenon due to write cycles is irreversible. Various applications on a device typically share the non-volatile memory on the device. Therefore, if one application malfunctions or maliciously performs a large number of write cycles, the result may be failure of the non-volatile memory, which may affect other applications on the device. Thus, the deterioration of retention caused by a large number of write cycles is a hardware failure that can be caused by software operation.

[0011] The effect of write cycles on flash storage retention makes its use impossible to incorporate into devices intended for long term use. Devices with short life cycles, such as flash drives and smartphones, often use flash storage, while products intended for years of life typically use some other storage technology, such as magnetic storage devices. As a result, the benefits of flash storage are not incorporated into such devices, which tend to make these devices more costly to manufacture, less reliable, or have higher power requirements.

[0012] In the implementations described herein, the storage protection system tracks the number of attempted writes, which may include both completed writes and blocked writes as described herein. If the attempted writes of a subsystem meet (e.g., equal or exceed) a write threshold deemed to represent an excess, the storage protection system may take action to remedy this excess. For example, the storage protection system may suspend or stop the subsystem, or the storage protection system may reboot the entire device. Thus, the implementations described herein limit the write cycles of a subsystem to the associated write threshold. Additionally, in some implementations, the storage protection system maintains a whitelist and a blacklist. Subsystems on the whitelist may be exempt from having their write cycles limited, while subsystems on the blacklist may be subject to more severe penalties due to having a history of meeting their respective write thresholds. In this manner, the life of the storage device may be protected by limiting the write cycles that occur.

[0013] The various write thresholds of the subsystems of the device may be selected based on the desired lifespan of the device. By requiring the subsystems to adhere to such write thresholds or risk improvement, the storage protection system may thus limit the write cycles that occur on the storage device. Since the lifespan of a storage device is based on the number of write cycles, the storage protection system may thus ensure that the lifespan is as expected. As a result, device manufacturers may utilize flash storage without concern that the lifespan of such devices will be adversely affected by unpredictable write cycles, and since the implementations described herein can ensure that devices operate within expected parameters with respect to write cycles, the manufacturers may more accurately provide the lifespan of such devices when creating device specifications.

[0014] Figure 1 is a conceptual diagram of a storage protection system 100 for protecting the lifespan of a storage device 110, in accordance with certain embodiments of the invention described herein. As shown in Figure 1, the storage protection system 100 may be integrated into a desktop computer, a notebook computer, a utility meter, or other computing device 120 that utilizes flash storage.

[0015] The computing device 120 may include a processing unit 130, a memory 140, and a storage device 110, also referred to herein as storage 110. The processing unit 130 may be configured to execute program code stored in the memory 140 or the storage 110. For example, the processing unit 130 may be a microprocessor, or the processing unit 130 may be a microcontroller integrated with the memory 140 or the storage 110, or both. The memory 140 may be a form of volatile memory, such as a random access memory (RAM), configured to store data and to store program code to be executed by the processing unit 130. The storage 110 may be a non-volatile memory configured to store data and program code. For example, the storage 110 is a form of flash storage, such as NAND or eMMC.

[0016] In some implementations, the storage protection system 100 includes a subsystem monitor 150 and a blacklist manager 160, each of which may execute on and communicate with the computing device 120. The subsystem monitor 150 may utilize a set of write thresholds, each write threshold being associated with a corresponding subsystem 170 of the computing device 120. Specifically, each write threshold may indicate the number of write cycles permitted to be performed by the corresponding subsystem 170 within a time interval (e.g., an hour). Based on the write attempts performed by the subsystem 170 during the interval compared to the respective write thresholds, the subsystem monitor 150 may block the write attempts, blacklist the subsystem 170, or take other remedial action. The blacklist manager 160 may manage the subsystems 170 on the blacklist 180, may take remedial action against those subsystems 170, or may release those subsystems 170 from the blacklist 180 as necessary.

[0017] Subsystem 170 may be, for example, an application package installed on computing device 120, an operating system, or a set of one or more processes that operate toward a common task. Subsystem 170 may include one or more related processes through which writes may be performed that are associated with subsystem 170. In some implementations, a designer (e.g., a manufacturer or service provider) of computing device 120 defines subsystems 170; for example, the designer defines which processes together form subsystem 170. The designer may further determine a write threshold that corresponds to each subsystem 170. In some implementations, a subsystem 170 that commits a violation (e.g., exceeds its respective write threshold) risks remedial action, such as being stopped, killed, or denied write access.

[0018] To select write thresholds for the various subsystems 170 of the computing device 120, the designer may determine a desired life span for the computing system. Based on the endurance of the storage 110 (i.e., the number of write cycles before a memory block becomes unusable) and the desired life span, the designer may determine a total number of write cycles per interval (e.g., per day) to achieve such that the computing device 120 remains usable for the desired life span. The designer may assign a number of write cycles per interval to the various subsystems 170 initially installed in the computing device 120 such that each such assignment amounts to or is the basis for a write threshold. It is not necessary that each subsystem 170 assigns the same number of write cycles per interval. Rather, the assignment may be based on expected usage, such that, for example, a subsystem that is expected to perform many writes during standard usage is assigned a larger portion of the write cycles per interval compared to a subsystem that is expected to perform fewer writes during standard usage.

[0019] In some implementations, the various write thresholds may be stored on the computing device 120, e.g., in a dedicated partition of the storage 110. Furthermore, in some implementations, the write thresholds are dynamic. For example, when a new subsystem 170 is added to the computing device 120, or when a critical process not subject to a write threshold performs more writes than expected, the various write thresholds of the computing device 120 may be adjusted downward to accommodate the additional writes that have occurred or are expected to occur. For example, when a new subsystem 170 is added to the computing device, e.g., when a third-party application is installed, the write thresholds of the other subsystems 170 may be adjusted downward to accommodate the addition of the new write thresholds for the new subsystem 170. Specifically, in some implementations, the new write thresholds may be equal to or otherwise based on the amount of writes subtracted from the write thresholds of the other subsystems 170. When the write thresholds are adjusted, the stored values ​​of the write thresholds may be adjusted to the new values. Through the use of dynamic write thresholds, the lifespan of the storage 110 can be protected even as the subsystems 170 change in the computing device 120.

[0020] In some implementations, storage protection system 100 utilizes a whitelist 190, a blacklist 180, or both. Whitelist 190 may include preferred subsystems 170, as described further below, and blacklist 180 may include non-preferred subsystems 170. In some implementations, blacklist 180 is implemented as a set of references to blacklisted subsystems 170, and whitelist 190 is implemented as a set of references to whitelisted subsystems 170, as shown in FIG.

[0021] Whitelist 190 may include subsystems 170 that are permitted to operate freely with no write threshold or that have a write threshold set so high that it is practically non-existent. In some implementations, subsystems 170 are added to whitelist 190 at design time and addition to whitelist 190 is permanent. Additionally, in some implementations, whitelist 190 may be manually modified by an administrator of computing device 120. Whitelisted subsystems 170 may include, for example, processes of storage protection system 100 itself, critical processes, trusted processes, processes expected to generate write activity in bursts, or processes utilized for debugging.

[0022] Blacklist 180 may include subsystems that have violated their write thresholds one or more times. As a result of violations and being on blacklist 180, subsystem 170 may face penalties or more stringent requirements from storage protection system 100. For example, a blacklisted subsystem 170 may be blocked from writing to storage 110 or may be subject to other remedial action as described below.

[0023] The blacklist 180 and the whitelist 190 may be implemented in a variety of ways. For example, for each of the blacklist 180 and the whitelist 190, the storage protection system 100 may maintain a data structure representing the list. Additionally or alternatively, each subsystem 170 may be associated with a status indicating whether the subsystem 170 is blacklisted, whitelisted, or neither. For example, the storage protection system 100 may maintain a table or other data structure that maps each subsystem 170 to its associated status, or each subsystem 170 may be represented by a data structure that includes or is associated with a variable that represents the respective status. In such an implementation, determining whether the subsystem 170 is on the blacklist 180, the whitelist 190, or neither may include determining a status associated with the subsystem 170.

[0024] In some implementations, subsystem 170 may be considered to be on blacklist 180 if its status is blacklisted. Similarly, subsystem 170 may be considered to have a blacklisted status if subsystem 170 is on blacklist 180. Furthermore, subsystem 170 may be considered to have been added to blacklist 180 if its status is changed from blacklisted, and subsystem 170 may be considered to have been removed from blacklist 180 if its status is changed from blacklisted to some other status (e.g., non-blacklisted). Similarly, subsystem 170 may be considered to have been added to blacklist 180 if its status is changed to some other status (e.g., non-blacklisted) if subsystem 170 is removed from blacklist 180. Similarly, subsystem 170 may be considered to have been on whitelist 190 if its status has a whitelisted status, and subsystem 170 may be considered to have a whitelisted status if subsystem 170 is on whitelist 190.

[0025] As discussed above, various remedial actions may be taken against subsystems 170 on blacklist 180. However, in some implementations, before such remedial actions can be taken, storage protection system 100 must identify which subsystem 170 is violating. In some implementations, subsystem monitor 150 is configured to monitor attempted writes, block write access to storage 110 for the violating subsystems 170, and blacklist the violating subsystems 170.

[0026] Subsystem monitor 150 may be implemented as hardware, software, or a combination of both. For example, subsystem monitor 150 may be implemented as program code executed by processing unit 130, or subsystem monitor 150 may be implemented as a dedicated hardware device. In some implementations, subsystem monitor 150 has a trusted status and is therefore able to monitor writes to memory 140, block writes to storage 110 (e.g., by blocking direct writes to storage 110 or by blocking synchronization of memory contents to storage 110), or allow writes to storage 110 (e.g., by allowing direct writes to storage or by allowing synchronization of memory contents to storage 110).

[0027] In one implementation, the subsystem monitor 150 includes or has access to a driver for the memory 140. Thus, when the subsystem 170 writes to the memory 140, the subsystem monitor 150 tracks the writes, enables the writes to be saved as memory contents in the memory 140, and determines whether those memory contents have been synchronized to the storage 110. Typically, a process writes data to the memory 140, and that data is synchronized (i.e., written) to the storage 110 in the normal operation of the computing device 120. However, in some implementations, the subsystem monitor 150 suspends the synchronization between the memory 140 and the storage 110, and thus controls whether the write to the storage actually occurs. The subsystem monitor 150 can initiate synchronization of the memory 140 to the storage 110 when certain conditions are met, such as when the subsystem 170 does not meet its write threshold, as described in this disclosure. In another implementation, each subsystem 170 utilizes an application programming interface (API) to write data, and a library implementing the API calls the subsystem monitor 150 when a write operation is performed. As described above, the subsystem monitor 150 allows writes to modify memory 140 but may prevent direct writes to storage 110 and may prevent synchronization of memory 140 to storage 110. The subsystem monitor 150 may initiate synchronization of memory 140 to storage 110 or may allow direct writes to storage 110 when a write condition is met, such as when subsystem 170 does not meet its write threshold.

[0028] Periodically, the subsystem monitor 150 may determine write statistics describing how various subsystems 170 of the computing device 120 have written to the memory 140. Additionally or alternatively, in some implementations, the subsystem monitor 150 continuously updates the write statistics, such as after each write attempt. Based on these write statistics, the subsystem monitor 150 may determine which subsystem 170 has committed a violation. The subsystem monitor 150 may block such subsystem 170 from writing to the storage 110 so that memory contents written by that subsystem 170 to the memory 140 are not synchronized to the storage 110. Additionally or alternatively, the subsystem monitor 150 may add such subsystem 170 to the blacklist 180. Furthermore, in some implementations, the subsystem monitor 150 may report the write statistics to a centralized location, such as a head-end system, where the write statistics of the various computing devices 120 may be aggregated together or compared, such as for debugging or reporting purposes.

[0029] Subsystem monitor 150 may be configured to initiate various remedial actions when subsystem 170 commits a violation (e.g., meets its write threshold). Remedial actions may include, for example, adding subsystem 170 to blacklist 180 and preventing write access to storage 110 while subsystem 170 is on blacklist 180, logging the violation, allowing the write threshold to be exceeded on a one-time or limited-time basis, temporarily halting processes associated with subsystem 170, terminating processes associated with subsystem 170, resetting or restarting computing device 120 as a whole, or terminating various subsystems 170 in computing device 120 except for critical processes (i.e., entering safe mode). Each of these remedial actions has its own advantages. At each violation of subsystem 170, subsystem monitor 150 may initiate one or more remedial actions. In some implementations, each subsystem 170 is associated with a predefined set of remedial actions, and when subsystem 170 commits a violation, subsystem monitor 150 may initiate each remedial action in the predefined set in response to the violation.

[0030] A remedial action to prevent write access to storage 110 may directly prevent subsystem 170 from exceeding its write threshold, or from exceeding its write threshold to a predefined extent. As a result of preventing writes to storage 110 when the write threshold is met, storage protection system 100 can protect the lifespan of storage 110 by controlling the total write cycles that occur.

[0031] The remedial action of logging violations allows the storage protection system 100 or a remote system, such as a head-end system, to track the history of writes so that the protection system 100, the head-end system, or some other system or entity can identify whether the computing device 120 is behaving abnormally or can identify an abnormal subsystem 170.

[0032] The remedy of allowing the write threshold to be met on a one-time or limited-time basis may allow subsystems 170 to have some leeway with respect to their respective write thresholds. For example, for a subsystem 170 that experiences bursts of writes on a one-time or infrequent basis, such bursts may not impact the lifespan of storage 110. In this case, however, the occurrence of meeting the write threshold may be recorded to prevent future writes from subsystem 170 for a period of time (e.g., while subsystem 170 is on blacklist 180).

[0033] A remedial action of temporarily halting the subsystem (e.g., stopping a process associated with subsystem 170) delays subsystem 170 from completing writes until subsystem 170 is resumed. Thus, the write rate of subsystem 170 may be effectively reduced based on a function of the suspend period (i.e., the time between suspending and resuming subsystem 170). In some embodiments, the suspend period is selected to allow the write rate of subsystem 170 to be reduced to an acceptable level below the write threshold. Furthermore, suspending and resuming subsystem 170 may allow minimal disruption to logic within subsystem 170. For subsystems 170 that are timing insensitive, special handling of suspend and resume operations may not be necessary and functionality may be allowed to continue normally, except at a slower rate.

[0034] A remedial action of terminating the subsystem (e.g., terminating a process associated with subsystem 170) may stop subsystem 170 from attempting writes for at least the amount of time it takes for subsystem 170 to restart. Additionally, restarting subsystem 170 may correct the malfunction that was causing subsystem 170 to perform excessive write attempts.

[0035] The remedial action of terminating various subsystems 170 within the computing device 120 except for critical processes, thus transitioning the computing device 120 to a safe mode, may stop such subsystems 170 from performing excessive writes. In some cases, the subsystems 170 interact with each other, and thus the excessive entitlements of one subsystem 170 may be caused by an interaction with another subsystem 170. Entering the safe mode may allow the continuation of critical processes, such as network communications and, if the computing device 120 is a utility meter, metering the consumption of resources. Thus, the computing device 120 may continue to perform its critical functions and may be able to communicate with external resources, such as a head-end system, for debugging, software updates, or consumption reporting purposes.

[0036] A remedial action of resetting or rebooting computing device 120 as a whole has the same advantages as entering safe mode, but is a potentially more drastic solution since critical processes are temporarily suspended during the reboot or reset. This more drastic action may be useful, particularly if the subsystem 170 whose violation prompted the remedial action is a critical process or is a subsystem 170 that interacts with a critical process.

[0037] In some implementations, blacklist manager 160 may manage subsystems 170 on blacklist 180. For example, each subsystem 170 may be associated with at least one timer, such as a release timer, an improvement timer, or both. Each such timer may be initialized (e.g., reset) when the associated subsystem 170 is added to blacklist 180. Blacklist manager 160 may manage timers associated with blacklisted subsystems 170 and may perform actions on blacklisted subsystems 170 upon expiration of such timers.

[0038] When associated with a blacklisted subsystem 170, the release timer may count out a release time, after which the blacklist manager 160 may release (i.e., remove) the subsystem 170 from the blacklist 180. Thus, the blacklist manager 160 may utilize a release timer to ensure that the associated subsystem 170 is only temporarily blacklisted for a violation. In some implementations, if a currently blacklisted subsystem 170 commits another violation, the blacklist manager 160 may reset the release timer such that release of the subsystem does not occur until the release time has elapsed after the most recent violation by the subsystem 170. In this manner, the release time may function as a threshold time after which the subsystem 170 is removed from the blacklist 180.

[0039] In the case of a blacklisted subsystem 170, a remediation timer counts out a remediation time after which the blacklist manager 160 may execute at least a portion of the remediation action, such as initiating a remediation action or terminating a remediation action previously initiated by the subsystem monitor 150. In one example, the subsystem monitor 150 suspends the subsystem 170 upon adding the subsystem 170 to the blacklist 180. In that case, the remediation timer may be used to count out a remediation time equal to a desired suspend period, upon the expiration of which the blacklist manager 160 may resume the subsystem 170. In another example, when the subsystem 170 is blacklisted, the blacklist manager 160 may abort the subsystem 170 (e.g., by aborting some or all processes of the subsystem 170) after the remediation time expires. In this manner, the storage protection system 100 may enable the subsystem 170 to prepare for its own shutdown during the remediation time, such as by performing pre-shutdown tasks or by shutting itself down. In another example, blacklist manager 160 may reboot computing device 120 as a whole or reset the operating system on computing device 120 after the remediation time has elapsed. Thus, use of a remediation timer may allow various subsystems 170 of computing device 120 to prepare for shutdown. In this manner, the remediation time may act as a threshold time after which any remedial action may be implemented or terminated.

[0040] The timers may be implemented in a variety of ways. For example, the timers may be implemented as countdown timers. For example, when a subsystem 170 is placed on the blacklist 180 or commits a violation while already on the blacklist 180, an associated release timer, implemented as a countdown timer, may be initialized with a value representing a release time and may count downward as time passes. When the release timer reaches zero, the release timer and associated release time are considered to have expired, and the blacklist manager 160 may remove the subsystem 170 from the blacklist 180 in response to such expiration. As another example, when a subsystem 170 is placed on the blacklist 180, an associated remediation timer, implemented as a countdown timer, may be initialized with a value representing a remediation time and may count downward as time passes. When the remediation timer reaches zero, the remediation timer and the associated remediation time may be considered expired, and in response to the expiration, the blacklist manager 160 may perform a remediation action (e.g., terminate the subsystem 170 or restart the computing device 120) or terminate the remediation action (e.g., resume the subsystem 170 after an interruption).

[0041] However, in some implementations, the timers may count upward. In one example of this, when a subsystem 170 is placed on the blacklist 180, or commits a violation while already on the blacklist 180, an associated release timer implemented as an upward counting timer may be initialized with a value of zero and may count upward over time toward a release time. When the release timer reaches the release time, the release timer and the associated release time are deemed expired, and the blacklist manager 160 may remove the subsystem 170 from the blacklist 180 in response to such expiration. As another example, when a subsystem 170 is placed on the blacklist 180, an associated remediation timer implemented as an upward counting timer may be initialized with a value of zero and may count upward over time toward the remediation time. When the remediation timer reaches the remediation time, the remediation timer and the remediation time are deemed expired, and in response to the expiration, the blacklist manager 160 may implement or conclude a remedial action.

[0042] Each timer may count time based on established units of time associated with that timer. For example, a timer may represent time in seconds and count downs or ups in seconds. As another example, a timer may represent time in intervals, in which case the timer may count downs or ups as the interval progresses. It will be appreciated that various units of time measurement may be used by the timers.

[0043] FIG. 2 is an architecture diagram of a computing device 120, specifically a utility meter 200, on which the storage protection system 100 can execute, according to certain embodiments of the present invention. For example, but not limited to, the utility meter 200 may be a water meter, a gas meter, or other type of meter that measures consumption of a resource 210 on a premises 220. To this end, the utility meter 200 may include a metering engine 205 that detects a signal indicative of usage of the resource 210 and determines usage of the resource 210 on the premises 220 based on the signal. As with other types of computing devices 120, the utility meter 200 may include a processing unit 130, a memory 140, and a storage 110, which may be connected to each other via a system bus 230. For example, but not limited to, the memory 140 may be a RAM and the storage 110 may be a NAND memory or a flash storage such as an eMMC. Although the utility meter 200 is shown in FIG. 2, it will be understood that the computing device 120 is not limited to the utility meter 200. Rather, for example, the computing device 120 may be a collector, a gateway, or another computing device other than the utility meter 200 .

[0044] 3 is a flow diagram of a method 300 for managing writes to storage device 110 to protect the lifespan of storage device 110, according to certain embodiments of the present invention. Method 300 may be performed by computing device 120. For example, method 300 may be implemented as program code integrated into subsystem monitor 150, executable by processing unit 130 on computing device 120. In some implementations, an instance of method 300, or the like, may be executed each time a write is attempted by subsystem 170. Additionally or alternatively, although not shown in FIG. 3, method 300, or the like, may be executed periodically (e.g., every few seconds).

[0045] Prior to the first execution of method 300 (e.g., the first time method 300 runs after a reboot of computing device 120), a respective write threshold may be established for each subsystem 170 installed on computing device 120. For example, the various write thresholds may be encoded in storage 110 and accessible by subsystem monitor 150. If the write thresholds are dynamic, the write thresholds may be updated as needed, and each future execution of method 300 may use the updated write thresholds. In some embodiments, the time is divided into intervals, such as several seconds each, and the write threshold is applied to each interval individually, such that the corresponding subsystem 170 assumes risk of improvement if it meets the write threshold in a given interval.

[0046] In some implementations, storage protection system 100 executes entirely in memory 140, and program code implementing storage protection system 100 may be verified prior to execution of method 300. In such implementations, computing device 120 may load program code implementing this method of managing writes 300 into memory 140 prior to a first execution of method 300.

[0047] Additionally, prior to an initial execution of method 300, subsystem monitor 150 may establish blacklist 180, which may be a list of subsystems 170 that have behaved badly in the past (e.g., by exceeding their respective write thresholds) and therefore have been subject to penalties. Although blacklist 180 may be empty when an initial execution of method 300 begins, blacklist 180 may be stored (e.g., in memory 140) and utilized across various executions of method 300. Each subsystem 170 on blacklist 180 may be associated with at least one respective timer, such as a release timer or an improvement timer.

[0048] 3, at block 305 of method 300, subsystem monitor 150 detects a write attempt (i.e., an attempt to write to storage 110). For example, a write to memory 140 may be considered a write attempt because the written memory contents are typically subsequently synchronized to storage 110. Each attempt to write to a single block of memory may be considered as a write attempt, whereas attempts to write to multiple blocks of memory may be considered as multiple write attempts, specifically, one write attempt per block.

[0049] In block 310, subsystem monitor 150 identifies the subsystem 170 that performed the write attempt. The technique used to determine which subsystem 170 performed the write attempt may be based in part on how subsystem 170 is defined. For example, if subsystem 170 is defined to include a set of files, subsystem monitor 150 may identify the subsystem 170 that performed the write attempt based on the file being written. As another example, if subsystem 170 is defined to include a set of processes, subsystem monitor 150 may identify the subsystem 170 that performed the write attempt based on the process performing the write attempt.

[0050] In block 310, the subsystem monitor 150 updates statistics describing the write attempt for the subsystem 170 that performed the write attempt. For example, in some implementations, the subsystem monitor 150 maintains a count of the write attempts made by each subsystem 170 during an active interval (i.e., an interval that has started and not yet ended). In some implementations, other statistics may be maintained, such as, for example, the total number of write attempts or write cycles (i.e., memory blocks that were actually synchronized to the storage 110) of the subsystem 170 over time, the rate of write attempts, or the like, or a comparison of write attempts in an active interval to a preceding interval (i.e., an interval that occurred prior to the active interval). These statistics may be logged for use as described herein, and in some implementations for debugging, reporting, or other purposes.

[0051] In some implementations, the computing devices 120 communicate the statistics to a head-end system, which is a centralized location for managing the various computing devices 120. For example, the head-end system may aggregate statistics received from the various computing devices 120, and additionally or alternatively, the head-end system can identify whether the computing device 120 is behaving in an abnormal manner based on the statistics compared to statistics of other computing devices 120 that report to the head-end system.

[0052] At decision block 315, the subsystem monitor 150 determines whether the subsystem 170 is on the blacklist 180. If the subsystem 170 is on the blacklist 180 (i.e., the subsystem 170 was previously placed on the blacklist 180 and has not yet been removed from the blacklist 180), then at block 320, the subsystem monitor 150 blocks the write attempt. In this case, the subsystem monitor 150 may block all write attempts of the subsystem 170 that occur during the active interval. More specifically, for example, to block the write attempt, the subsystem monitor 150 does not allow the kernel of the computing device 120 to synchronize memory contents written by the subsystem 170 during the write attempt to the storage 110. In some implementations, the subsystem monitor 150 notifies the subsystem 170 (e.g., notifies a process within the subsystem 170) that the write attempt was blocked. In that case, the method 300 then ends and may resume upon the next write attempt by the subsystem 170.

[0053] If the subsystem 170 is not on the blacklist 180, then in decision block 325 the subsystem monitor 150 determines whether the subsystem 170 is on the whitelist 190. If the subsystem is on the whitelist 190, then in block 330 the subsystem monitor 150 allows the subsystem 170 to write to the storage 110. Specifically, in one implementation, the subsystem monitor 150 does not interfere with the write process, allowing the kernel to synchronize the memory contents written by the subsystem 170 to the storage 110. In another implementation, the subsystem monitor 150 requests that the kernel synchronize the memory contents written by the subsystem 170 on the write attempt. The method 300 then ends and may resume upon the next write attempt by the subsystem 170.

[0054] If subsystem 170 is not on blacklist 180 or whitelist 190, method 300 proceeds to decision block 335. At decision block 335, subsystem monitor 150 determines whether the write attempt, if completed (e.g., synchronized to storage 110), would cause subsystem 170 to meet (e.g., equal or exceed) its write threshold for the active interval. If the write attempt would not cause subsystem 170 to meet its write threshold, method 300 proceeds to block 340, where subsystem monitor 150 allows subsystem 170 to complete the write attempt to storage 110. Method 300 then ends and may resume at the next write attempt by subsystem 170.

[0055] However, if the subsystem 170 is not on either the blacklist 180 or the whitelist 190 and the write attempt causes the subsystem 170 to meet its write threshold for the active interval, the method 300 proceeds to block 345. In block 345, if this is the first time that the subsystem 170 has met its write threshold since initialization (e.g., since the computing device 120 was last rebooted), the subsystem monitor 150 may log this as the first time that the subsystem 170 has met its write threshold. The resulting log may be used for debugging, reporting, or other purposes. In some implementations, the log may be sent to a centralized location, such as a head-end system, for further use.

[0056] Although not shown in Figure 3, in some implementations, the violation may be allowed if this is the first time that the subsystem's write threshold is met. If so, subsystem monitor 150 may allow the write attempt, thus causing the memory contents written by subsystem 170 to be synchronized to storage 110. However, in the example method 300 of Figure 3, the determination of whether this is the first time that the subsystem's write threshold is met is used only for logging purposes, and the write attempt may still be blocked, as described below.

[0057] In some implementations, regardless of whether the subsystem 170 has previously met its write threshold, in decision block 350 the subsystem monitor 150 determines whether the subsystem 170 will meet a second write threshold for the active interval if the write attempt is completed. Like the write threshold, the second write threshold is associated with the subsystem 170 and may therefore vary from one subsystem 170 to another. The second write threshold may be higher than the write threshold of the subsystem 170, or more specifically, may be based on the write threshold. For example, the second write threshold may be a percentage of the write threshold, where the percentage is greater than the percentage. In one example, the second write threshold is 125% of the write threshold. Additionally or alternatively, the second write threshold applies to an extended interval that includes the active interval but is longer. For example, if the write threshold applies to an interval having a length of one hour, the second write threshold may be applied to an extended interval of one day. In this case, a second write threshold can be used to tolerate occasional bursts of excessive writes, as long as the writes remain limited for a longer period of time.

[0058] If the write attempt would not cause subsystem 170 to satisfy its second write threshold for the applicable period of time (e.g., active interval or active extension interval), then at block 355 subsystem monitor 150 allows the write attempt to proceed so that memory contents written by subsystem 170 in the write attempt may be synchronized to storage 110. However, if the write attempt would cause subsystem 170 to satisfy its second write threshold for the applicable period of time, method 300 proceeds to decision block 360.

[0059] Some implementations utilize a second write threshold as described above to allow subsystem 170 to slightly exceed its write threshold without risk of being blacklisted or having write attempts blocked. However, it will be understood that other implementations need not utilize this second threshold. In that case, decision block 350 may be skipped and the method may proceed from block 345 to decision block 360.

[0060] More generally, storage protection system 100 may utilize a set of blocking criteria to determine whether to blacklist subsystem 170, block subsystem 170 from writing to storage 110, or both. In one implementation, the blocking criteria are met only if subsystem 170 meets its write threshold, and in response to the blocking criteria being met, subsystem monitor 150 blacklists subsystem 170 and blocks the write attempt. In another implementation, the blocking criteria are met only if subsystem 170 meets its second write threshold, and in response to the blocking criteria being met, subsystem monitor 150 blacklists subsystem 170 and blocks the write attempt. In yet another implementation, the blocking criterion is met only if the subsystem 170 meets its write threshold a predetermined number of times within a time frame (e.g., every week or since the computing device 120 was last rebooted) or meets its second write threshold a predetermined number of times within a time frame, and in response to such blocking criterion being met, the subsystem monitor blacklists the subsystem 170 and blocks write attempts.

[0061] In decision block 360, the subsystem monitor 150 determines whether the violation threshold has been met by a current violation (i.e., by a write attempt that caused the subsystem 170 to exceed the write threshold). In some implementations, the violation threshold is or represents the number of violations tolerated by a subsystem 170 that is not on the blacklist 180 before the subsystem 170 is added to the blacklist 180. The violation threshold may apply to an established time frame, such as for the last day or more, the last week or more, for a set number of intervals or more, or since the last reboot of the computing device 120. In that case, the subsystem monitor 150 determines whether the subsystem 170 has met the violation threshold within that time frame. If the violation threshold is not met (e.g., equal to or exceeded) as determined in decision block 360, the subsystem monitor 150 allows the write attempt in block 365. Otherwise, if the subsystem 170 has met the violation threshold, the method 300 proceeds to block 370.

[0062] In some implementations, no violation threshold is used. The absence of a violation threshold may be logically equivalent to using a value of 1 as the violation threshold. In that case, decision block 360 may be skipped and method 300 may proceed from decision block 350 to block 370.

[0063] In block 370, the subsystem monitor 150 adds the subsystem 170 to the blacklist 180 and initializes each timer associated with the subsystem 170. As mentioned above, such timers may include one or both of a release timer and an improvement timer.

[0064] In decision block 375, the subsystem monitor 150 determines one or more remedial actions, in addition to blocking the write attempt, to handle the violation of the subsystem 170. This determination may be based on previously established settings associated with the subsystem 170 in the storage protection system 100. For example, the settings may indicate to suspend (i.e., pause) the subsystem 170, terminate (i.e., stop its various processes), or take some other remedial action when the subsystem 170 is blacklisted. In some implementations, each subsystem 170 may be associated with a set of remedial actions, which may be an empty set or may include one or more remedial actions that are triggered when the subsystem 170 is added to the blacklist 180. To determine which remedial action to execute, the subsystem monitor 150 may check the settings of the storage protection system 100 to determine which remedial actions are associated with the subsystem 170. In some implementations, the subsystem monitor 150 executes each remedial action indicated in the settings. If no such remedial action is associated with the subsystem 170 other than simply blocking write attempts, the method 300 may skip to block 390 .

[0065] In the example of FIG. 3, other than blocking the write attempt, only two remedial actions are available: pausing subsystem 170 and terminating subsystem 170. If subsystem monitor 150 determines to terminate subsystem 170, then in block 380, subsystem monitor 150 terminates subsystem 170. More specifically, if subsystem 170 includes multiple processes, some implementations of subsystem monitor 150 may terminate only the process that made the write attempt, while other implementations may terminate multiple processes or all processes in subsystem 170. Because such processes may be dependent on each other, it may be beneficial to terminate all processes of subsystem 170. Method 300 may then skip to block 390. However, if subsystem monitor 150 determines to suspend subsystem 170, then in block 385, subsystem monitor 150 suspends subsystem 170. More specifically, if subsystem 170 includes multiple processes, some implementations may suspend only the process that made the write attempt, while other implementations may suspend multiple processes or all processes in subsystem 170. Because such processes may be dependent on each other, it may be beneficial to suspend all processes of subsystem 170. When suspending subsystem 170, subsystem 170 may be associated with a remediation timer that counts a remediation time, upon the expiration of which subsystem 170 will be resumed. Method 300 may then proceed to block 390.

[0066] In block 390, the subsystem monitor 150 blocks the write attempt. In other words, the subsystem monitor 150 may block memory contents written by the subsystem 170 during the active interval from being synchronized to the storage 110. For example, the subsystem monitor 150 may request that the kernel not synchronize the memory contents, or in implementations where the kernel performs this synchronization only on request, the subsystem monitor 150 may simply choose not to request that the kernel synchronize the memory contents. In some implementations, the subsystem monitor 150 may notify the subsystem 170 of any remedial action taken (e.g., stopped, blocked write attempt). The method 300 then ends and may resume at the next write attempt by the subsystem 170.

[0067] FIG. 4 is a flow diagram of a method 400 for managing a blacklist 180 of a violating subsystem 170 to protect the lifespan of a storage device 110, according to certain embodiments of the invention. As described above, a subsystem 170 may be added to the blacklist 180, such as for meeting (e.g., equaling or exceeding) a respective write threshold. The method 400 of FIG. 4, or a similar one, may be used to manage that blacklist 180, including, for example, removing the subsystem 170 from the blacklist 180 when certain conditions are met. In some implementations, the method 400 for managing a blacklist 180 as shown in FIG. 4 may be performed in parallel with the method 300 for managing writes as shown in FIG. 3. For example, in some implementations, the method 400 of FIG. 4, or a similar method, may be performed once per second or on some other periodic schedule to manage the blacklist 180 populated by the method 300 for managing writes, and the method 300 for managing writes may be performed when a write attempt occurs.

[0068] The method 400 of Figure 4 may iterate through the subsystems 170 on the blacklist 180. As shown in Figure 4, in decision block 405, the blacklist manager 160 determines whether any more subsystems 170 on the blacklist 180 remain to be considered in the current execution of the method 400. If no more subsystems 170 remain to be considered, the method 400 may end at block 410. For example, no more subsystems 170 remain to be considered if the blacklist 180 is empty or if one or more iterations have already been performed such that all subsystems 170 on the blacklist 180 have already been considered.

[0069] However, if one or more subsystems 170 remain to be considered, then in block 415 the blacklist manager 160 may select a subsystem 170 to consider in this iteration.

[0070] As discussed above, each subsystem 170 may be associated with at least one timer, including a release timer, a remediation timer, or both. At decision block 420, it may be determined whether the selected subsystem 170 has a release timer. If the subsystem 170 has a release timer, the method 400 proceeds to decision block 425; otherwise, if the subsystem 170 does not have a release timer, the method 400 skips to decision block 440.

[0071] If subsystem 170 has a release timer, blacklist manager 160 determines whether the release timer has expired in decision block 425. In some implementations, the release timer is reset each time subsystem 170 commits a violation, and thus the release timer expires only after the associated release time has elapsed since the last violation.

[0072] If the release timer has expired, then in block 430, the blacklist manager 160 removes the subsystem 170 from the blacklist 180 and the method 400 returns to block 405 to determine whether any more subsystems 170 remain to be considered in the current execution of the method 400.

[0073] However, if the release timer has not expired, then in block 435, blacklist manager 160 decrements the release timer by the time that has elapsed between the current execution of method 400 and the last execution of method 400. For example, if the release timer maintains time in seconds and method 400 is executed once per second, the release timer may be decremented by one to represent the time that has elapsed. Method 400 then proceeds to decision block 440 to consider a refinement timer, if applicable.

[0074] At decision block 440, the blacklist manager 160 determines whether the subsystem 170 is associated with a remediation timer. If the subsystem 170 is not associated with a remediation timer, the method may return to block 405 to determine whether any additional subsystems 170 remain for consideration. However, if the subsystem 170 is associated with a remediation timer, at decision block 445, the blacklist manager 160 determines whether the remediation timer has expired.

[0075] If the remediation timer has not expired, then in block 455, the blacklist manager 160 may decrement the remediation timer by the time that elapsed between the current execution of the method 400 and the last execution of the method 400. The method 400 may then return to block 405 to determine whether any subsystems 170 remain for consideration.

[0076] However, if the remediation timer expires, then in block 450, the blacklist manager 160 performs a timed action associated with the remediation timer. For example, if the timed action is to return from suspend, the blacklist manager 160 may resume the process of the subsystem 170 that was suspended. If the timed action is to restart the subsystem 170 after it has terminated, the blacklist manager 160 may restart the process of the subsystem 170 that has terminated. In the case of a restart or restart of the subsystem 170, the blacklist manager 160 may remove the subsystem 170 from the blacklist 180, or, if the subsystem 170 is associated with a release timer, may reset the release timer to allow the subsystem 170 an opportunity to be removed from the blacklist 180. If the timed action is a restart of the entire computing device 120, the blacklist manager 160 may initiate a restart of the computing device 120.

[0077] In some implementations, rebooting the computing device 120 clears the blacklist 180. For example, the storage protection system 100 may run on memory 140, which may be volatile, and thus rebooting the computing device 120 may clear the memory 140 and thus the blacklist 180. Alternatively, if the blacklist 180 is stored in the storage 110, the blacklist manager 160 may, in some implementations, clear the blacklist 180 before rebooting the computing device 120.

[0078] The method 400 then returns to block 405 to determine whether any more subsystems 170 remain for consideration.

[0079] In this manner, implementations described herein protect storage device 110 by limiting the write cycles of storage device 110 per subsystem 170. By managing the writes of various subsystems 170 individually, storage protection system 100 can identify subsystems 170 that are performing too many writes and limit the writes performed by such subsystems. Targeting subsystems 170 individually may, in some cases, preserve the life of storage device 110 over time while allowing the rest of computing device 120 to continue to perform normally.

[0080] Numerous specific details are described herein to provide a thorough understanding of the claimed subject matter. However, those skilled in the art will understand that the claimed subject matter may be practiced without these specific details. In other instances, methods, apparatus, or systems that would be known by one skilled in the art have not been described in detail so as not to obscure the claimed subject matter.

[0081] The features discussed herein are not limited to any particular hardware architecture or configuration. A computing device may include any suitable arrangement of components that provide results conditional on one or more inputs. Suitable computing devices include general-purpose microprocessor-based computer systems that access stored software (i.e., computer-readable instructions stored on the computer system's memory) that programs or configures the computing system from a general-purpose computing device to a specialized computing device that implements one or more aspects of the present subject matter. Any suitable programming, scripting, or other type of language or combination of languages ​​may be used to implement the teachings contained herein in the software used in programming or configuring a computing device.

[0082] Aspects of the methods disclosed herein may be performed in operation of such a computing device. The order of the blocks presented in the above examples may be changed, e.g., the blocks may be reordered, combined, and / or divided into sub-blocks. Certain blocks or processes may be performed in parallel.

[0083] The use of "adapted to" or "configured to" herein is intended as open and inclusive language that does not preclude apparatus adapted or configured to perform additional tasks or steps. Additionally, the use of "based on" is intended to be open and inclusive in the sense that when a process, step, calculation, or other operation is "based on" one or more recited conditions or values, it may in fact be based on additional conditions or values ​​beyond those recited. The headings, lists, and numbering contained herein are for ease of description only and are not intended to be limiting.

[0084] While the present subject matter has been described in detail with respect to certain embodiments thereof, it will be appreciated that those skilled in the art, upon gaining the foregoing understanding, may readily make modifications, variations, and equivalents to such embodiments. It will therefore be understood that the present disclosure has been presented for purposes of illustration and not limitation, and is not intended to exclude the inclusion of modifications, variations, and / or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art.

Claims

1. 1. A method executed by a processor of a computing device for limiting write cycles to a storage device, comprising: tracking a first count of a first set of writes to a first memory content performed by a first subsystem of the computing device; determining that the first count satisfies a write threshold for the first subsystem; determining that a blocking criterion is satisfied by the first subsystem based on the first count satisfying the write threshold of the first subsystem; determining that a blocking criterion is not met by a second subsystem of the computing device; selectively synchronizing second memory contents written by the second subsystem to the storage device while preventing synchronization of the first memory contents written by the first subsystem to the storage device based on the blocking criteria not being satisfied by the second subsystem and being satisfied by the first subsystem; A method comprising:

2. The method of claim 1 , wherein determining that the blocking criterion is met for the first subsystem further comprises determining that the first subsystem previously exceeded the write threshold.

3. 2. The method of claim 1, wherein determining that the blocking criterion is met for the first subsystem further comprises determining that the first count satisfies a second write threshold that is higher than the write threshold.

4. Furthermore, changing a status of the first subsystem to blacklisted based on the first count satisfying the write threshold; and blocking additional writes performed by the first subsystem from being synchronized to the storage device while the first subsystem has a blacklisted status; The method of claim 1 , comprising:

5. Furthermore, initializing a release timer associated with the first subsystem based on the first subsystem having the blacklisted status; and changing a status of the first subsystem from blacklisted in response to expiration of the release timer; The method of claim 4 , comprising:

6. Furthermore, initializing a remediation timer associated with the first subsystem based on the first subsystem having the blacklisted status; and performing at least a portion of the remediation action in response to expiration of the remediation timer; The method of claim 4 , comprising:

7. performing at least a portion of the remediation actions in response to expiration of the remediation timer includes rebooting the computing device in response to expiration of the remediation timer; The method according to claim 6.

8. performing at least a portion of the remediation action in response to expiration of the remediation timer includes terminating the first subsystem in response to expiration of the remediation timer. The method according to claim 6.

9. Furthermore, suspending the first subsystem based on the first subsystem having a blacklisted status. Including, performing at least a portion of the remediation action in response to expiration of the remediation timer includes restarting the first subsystem in response to expiration of the remediation timer. The method according to claim 6.

10. The method of claim 1 , wherein determining that the blocking criteria are not met for the second subsystem comprises determining that the second subsystem is on a whitelist.

11. The first subsystem is defined as a set of one or more processes. The method of claim 1.

12. The first subsystem is defined as a set of one or more files; The method of claim 1.

13. 1. A computer program having program instructions for limiting write cycles to a storage device, the program instructions executable by a processor to cause the processor to perform a method, The method comprises: tracking a first count of a first set of writes to a first memory content performed by a first subsystem of the computing device; determining that the first count satisfies a write threshold for the first subsystem; determining that a blocking criterion is satisfied by the first subsystem based on the first count satisfying the write threshold of the first subsystem; determining that a blocking criterion is not met by a second subsystem of the computing device; selectively synchronizing second memory contents written by the second subsystem to the storage device while preventing synchronization of the first memory contents written by the second subsystem to the storage device based on the blocking criteria not being satisfied by the second subsystem and being satisfied by the first subsystem; A computer program comprising:

14. The method further comprises: changing a status of the first subsystem to blacklist based on the first count satisfying the write threshold; and blocking additional writes performed by the first subsystem from being synchronized to the storage device while the first subsystem has the blacklisted status; Including, 14. A computer program product as claimed in claim 13.

15. The method further comprises: initializing a release timer associated with the first subsystem based on the first subsystem having a blacklisted status; and changing a status of the first subsystem from blacklisted in response to expiration of the release timer; 15. The computer program of claim 14, comprising:

16. The method further comprises: initializing a remediation timer associated with the first subsystem based on the first subsystem having a blacklisted status; and performing at least a portion of the remediation action in response to expiration of the remediation timer; 15. The computer program of claim 14, comprising:

17. 1. A computing device comprising: A processor configured to execute computer readable instructions; and a memory configured to store computer-readable instructions that, when executed by a processor, cause the processor to perform operations; Including, The operation includes: tracking a first count of a first set of writes to a first memory content performed by a first subsystem of the computing device; determining that the first count satisfies a write threshold for the first subsystem; determining that a blocking criterion is satisfied by the first subsystem based on the first count satisfying the write threshold of the first subsystem; determining that a blocking criterion is not met by a second subsystem of the computing device; selectively synchronizing second memory content written by the second subsystem to the storage device while preventing synchronization of the first memory content written by the first subsystem to the storage device based on the blocking criteria not being satisfied by the second subsystem and being satisfied by the first subsystem; a computing device comprising:

18. The operation further comprises: changing a status of the first subsystem to blacklisted based on the first count satisfying the write threshold; and blocking additional writes performed by the first subsystem from being synchronized to the storage device while the first subsystem has the blacklisted status; 20. The computing device of claim 17, comprising:

19. The operation further comprises: initializing a release timer associated with the first subsystem based on the first subsystem having the blacklisted status; and changing a status of the first subsystem from blacklisted in response to expiration of the release timer; 20. The computing device of claim 18, comprising:

20. The operation further comprises: initializing a remediation timer associated with the first subsystem based on the first subsystem having a blacklisted status; and performing at least a portion of the remediation action in response to expiration of the remediation timer; 20. The computing device of claim 18, comprising:

Images