Data preservation using memory aperture flushing sequence

Abstract

Combined operational steps and device characteristics help preserve data from integrity threats. Data is divided into critical data and non-critical data based on criteria such as customer requirements, workload criticality, or virtual machine criticality. For example, data can be generated in a compute node to be stored in a storage node. Critical data is stored at physical addresses in a battery-backed memory aperture, where critical data will be flushed before non-critical data due to a flush order imposed by or on the battery-backed memory (e.g., a bottom-up NVDIMM flush order). Redundant copies of data (particularly non-critical data) can also be retained in case it is not flushed in a timely manner. The battery-backed memory aperture is sized and positioned according to the characteristics of its battery, and can be relocated or resized as conditions change. Flush defragmentation is performed to optimize the use of the aperture, particularly within the portion that holds critical data.

Description

Background Art

[0001] System architects and others who design, implement, modify, or optimize the system architecture of a computing system recognize that data storage choices often involve trade - offs. For example, some devices capable of storing digital data, such as some dynamic random - access memory devices, support relatively fast data storage operations but cannot reliably retain data in the event of a power outage. These devices are commonly referred to as "volatile" storage devices. There are many volatile storage devices, which have different technical characteristics, such as capacity, cost, expected working life, operating speed, difficulty of replacement, electrical requirements, and compatibility with hardware or software standards or best practices. Other data storage devices, such as electromechanical hard disk drives and solid - state drives, can reliably retain data values even after a power outage. These devices are commonly referred to as "non - volatile" storage devices. There are also many non - volatile storage devices, which also have different technical characteristics.

[0002] Accordingly, system architects and others who must choose between available storage mechanisms and processes are faced with an extremely large number of storage device choices, with many interdependent trade - offs between technology choices and system performance, cost, convenience, reliability, and other characteristics. To reduce complexity and enhance predictability, it may be helpful to focus on specific performance assumptions, goals, or insights in order to narrow the range of storage architecture choices to be seriously considered and prioritize them. Summary of the Invention

[0003] Some embodiments described in this document provide improved data preservation tools and techniques, especially in a networked multi - node environment, such as a cloud running virtual machines. In some embodiments, by performing memory allocation based on the battery - backed memory aperture flush order and related information about battery characteristics, the risk of losing critical data is reduced. Such storage allocation can also reduce data redundancy requirements without compromising data integrity.

[0004] In some embodiments, a computing system has a data preservation subsystem that includes battery - backed memory having an aperture. The aperture has a flush order, which is the order in which data is copied from the aperture to a non - volatile storage device associated with the aperture in response to a data integrity threat such as a power outage or a restart. The flush order defines the first - to - flush end and the last - to - flush end of the aperture.

[0005] In this example, a data preservation circuit communicates operably with a battery-backed memory. The data preservation circuit, which may include a processor and firmware, is configured to perform data preservation steps. In this example, these data preservation steps may include (a) receiving a request to store dataset A-data in an aperture, where the A-data includes data designated as critical data, (b) identifying a portion A-memory of the unallocated memory of the aperture, where the A-memory is large enough to hold the A-data and the address of the A-memory is closer to the earliest flushed end of the aperture than any other address of any other unallocated memory of the aperture, and (c) marking the A-memory as allocated and placing a copy of the A-data in the A-memory. These steps (a)-(c) may be repeated multiple times for different respective datasets containing critical data.

[0006] In this example, the data preservation steps may include (d) receiving a request to store dataset Z-data in the aperture, where the Z-data does not include any data designated as critical data, (e) identifying a portion Z-memory of the unallocated memory of the aperture, where the Z-memory is large enough to hold the Z-data and the address of the Z-memory is closer to the last flushed end of the aperture than any other address of any other unallocated memory of the aperture, and (f) marking the Z-memory as allocated and placing a copy of the Z-data in the Z-memory. These steps (d)-(f) may be repeated for different respective datasets not containing critical data and may be omitted when only critical data is stored.

[0007] The labels (a)-(f) are used herein only as identifiers and do not necessarily specify an order of operations. For example, Z-data may be stored in the aperture before any A-data is stored in the aperture.

[0008] In this context, the example data preservation subsystem provides a higher likelihood of successfully flushing and thereby preserving critical data by placing critical data before non-critical data in battery-backed memory in a flush order as compared to storing data in battery-backed memory regardless of data criticality. Given the capabilities of the data preservation subsystem described herein, alternative forms of data preservation, such as maintaining a duplicate or other redundant copy of critical data outside of the threatened computing system, may be reduced or eliminated.

[0009] In operation, some of the data storage embodiments described herein receive multiple requests, each request seeking to store a corresponding data set in an aperture of a battery-backed memory. The aperture has a flush order, which is the order in which data is copied from the aperture to a non-volatile storage device associated with the aperture in response to a data integrity threat. The flush order defines the first-flush end and the last-flush end of the aperture. In this example, each corresponding data set includes data designated as critical data. For at least two of the requests, the embodiment identifies corresponding portions of unallocated memory in the aperture, each portion of unallocated memory being large enough to hold the corresponding data set. The addresses of the identified corresponding portions of unallocated memory are closer to the first-flush end of the aperture than any other addresses of any other unallocated memory in the aperture. For at least one of the requests, the embodiment marks the identified corresponding portions of unallocated memory as allocated and places a copy of the corresponding data set therein.

[0010] Continuing the example, at some point after storing at least one piece of critical data, the embodiment detects a data integrity threat, e.g., detects an upcoming reboot or an impending loss of external power followed by a switch to battery power. In response to the threat detection, the embodiment flushes all of the critical data copied into the aperture, thereby copying the critical data from the aperture to the non-volatile storage device. The flush preserves all of the critical data copied into the aperture and thus preserves the critical data (and possibly also some or all of the non-critical data) without relying on having redundant copies of data outside the aperture and its non-volatile backup memory range.

[0011] Other technical activities and features related to the teachings herein will also become apparent to those skilled in the art. The examples given are merely illustrative. The summary of the invention is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Instead, the summary of the invention is provided to introduce - in a simplified form - some technical concepts that will be further described in the detailed description below. The invention is defined by the claims as properly understood, and if the summary of the invention conflicts with the claims, the claims shall prevail. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] A more detailed description will be given with reference to the accompanying drawings. These drawings only illustrate selected aspects and thus do not fully determine the coverage or scope.

[0013] Figure 1 is a block diagram generally showing a computer system and also a block diagram generally showing a configured storage medium;

[0014] Figure 2is a block diagram showing an environment including computing nodes and at least one storage node;

[0015] Figure 3 is a block diagram showing certain aspects of certain computing environments;

[0016] Figure 4 is a block diagram showing aspects of a system configured with data preservation functionality;

[0017] Figure 5 is a block diagram showing certain examples of battery characteristics;

[0018] Figure 6 is a block diagram showing certain examples of data criticality criteria;

[0019] Figure 7 is a block diagram showing certain examples of certain computing systems;

[0020] Figure 8 is an architectural diagram of a computing node hosting a number of virtual machines and networked with a number of storage nodes having a battery - backed memory aperture;

[0021] Figure 9 is an architectural diagram of an operating system running a number of virtual machines and using a storage - device - supported memory aperture;

[0022] Figure 10 is a flowchart showing steps in certain data preservation methods; and

[0023] Figure 11 is a further flowchart showing steps in certain data preservation methods. Detailed Description

[0024] Overview

[0025] The present invention may go beyond its origins, but understanding the origins of the present invention can help people more fully understand the present invention. In the current context, the motivation for some of the teachings described herein is the technical challenge of storing big data for machine learning. Internet - of - Things devices can output data quickly and in large quantities, and machine - learning tools and techniques can sometimes process this vast amount of data in a useful way. Some machine - learning databases contain millions of records in an active database, so fast read speeds are very important. Despite the large amount of data involved, data integrity is also important, both in terms of not expecting unauthorized and deliberate changes to individual data and in terms of not expecting data changes due to a restart or an unexpected power outage.

[0026] Other factors also play a role in this motivation scenario. Specifically, cost considerations prevent or discourage people from simply putting all data in NVDIMMs or similar battery-backed fast random access memories. Even if all data could be put into NVDIMMs, it would not address the risk to data integrity posed by battery failures. For example, battery backups are not as reliable as storage devices on hard disks, DVDs, tapes, or some similar non-volatile media. Battery life varies over time. Copying data across multiple NVDIMM devices can be considered to increase security, but cost is also an important factor. Additionally, the acceptable risk level for data integrity is not always constant. For example, different customers may have different service level agreements (SLAs) or other service goals or guarantees.

[0027] In short, some of the data preservation tools and techniques described herein are motivated in part by the technical challenges present in the effort to preserve large amounts of machine learning data. However, those skilled in the art will recognize that the teachings provided herein are also beneficially applicable to many other technical scenarios.

[0028] Some of the embodiments described herein implement a combination of operational steps and device characteristics to preserve data from integrity threats. Based on criteria such as customer requirements, workload criticality, or virtual machine criticality, data is classified into critical data and non-critical data. Critical data is stored at physical addresses in battery-backed memories where the critical data will be flushed before non-critical data. Some embodiments recognize and address the risk that the power of the battery may not be sufficient to flush both critical data and non-critical data to safety, for example, by retaining redundant copies of non-critical data in case it is not flushed in time. The size of the battery-backed memory aperture is set according to the characteristics of the battery and can be adjusted as conditions change. Defragmentation is performed to optimize the use of the aperture, particularly within portions that hold critical data. Other aspects of data preservation according to the invention are also discussed herein.

[0029] Some of the embodiments described herein can be viewed by some in a broader context. For example, concepts such as allocation, data, order, power, preservation, and request can be considered relevant to specific embodiments. However, the availability of a broad context does not mean that this document seeks exclusive rights to abstract concepts; it does not. Instead, this disclosure focuses on providing suitable specific embodiments whose technical effects fully or partially solve specific technical problems, such as how to balance NVDIMM costs with service level agreement data availability requirements. Other configurations of storage media, systems, and processes involving allocation, data, order, power, preservation, or request are not within the scope hereof. Thus, ambiguity, mere abstraction, lack of technical features, and the attendant problems of proof are also avoided in a proper understanding of this disclosure.

[0030] More generally, one of ordinary skill in the art will recognize that not every part of the present disclosure or any particular detail thereof is necessary to meet legal standards such as enablement, written description, or best mode. Moreover, the embodiments are not limited to the specific incentive examples, requests, responses, operating systems or other kernels, software development environments, interface standards, software processes, development tools, identifiers, files, data structures, symbols, control flows, pseudocode, naming conventions, node architectures, or other implementation choices described herein. Any apparent conflict with any other patent disclosure, even from the owner of the present invention, will not play any role in construing the claims set forth in this patent disclosure.

[0031] Technical Features

[0032] The technical features of the embodiments described herein will be apparent to one of ordinary skill in the art and will also be apparent in several ways to the interested reader. Some embodiments address technical activities such as memory allocation, flushing data from volatile memory to non-volatile memory, adjusting settings based on battery characteristics or sizing a memory aperture, and defragmenting memory, which are activities deeply rooted in computing technology. Some of the technical mechanisms discussed include, for example, NVDIMM or other battery-backed memory, Unified Extensible Firmware Interface, compute nodes, storage nodes, memory mapping, flushing order, and virtual machines. Some of the technical effects discussed include, for example, an increased likelihood of preserving critical data from integrity threats, reducing reliance on data redundancy to provide data preservation, and effectively sizing a battery-backed memory aperture considering criteria such as battery characteristics and the amount of memory reserved for non-critical data. Thus, pure mental processes are clearly excluded. Some embodiments improve the functionality of computing systems and services by preserving data from integrity threats while balancing criteria such as NVDIMM cost, customer service level requirements, and the availability of node storage for data redundancy. Based on the description provided, other advantages of the technical features based on the teachings will also be apparent to one of ordinary skill in the art.

[0033] Acronyms, Abbreviations, Names, and Symbols

[0034] Some acronyms, abbreviations, names, and symbols are defined below. Others are defined elsewhere in this document or need not be defined herein for one of ordinary skill in the art to understand.

[0035] ACPI: Advanced Configuration and Power Interface

[0036] ALU: Arithmetic Logic Unit

[0037] API: Application Programming Interface

[0038] BIOS: Basic Input / Output System

[0039] BMC: Baseboard Management Controller

[0040] CD: Compact Disc

[0041] CPU: Central Processing Unit

[0042] EFI: Extensible Firmware Interface

[0043] DRAM: Dynamic Random Access Memory

[0044] DVD: Digital Versatile Disc or Digital Video Disc

[0045] FPGA: Field Programmable Gate Array

[0046] FPU: Floating Point Unit

[0047] GPU: Graphics Processing Unit

[0048] GUI: Graphical User Interface

[0049] HDD: Hard Disk Drive (e.g., solid state, electromechanical, optical)

[0050] IaaS or IAAS: Infrastructure as a Service

[0051] ID: Identification or Identity

[0052] IoT: Internet of Things

[0053] LAN: Local Area Network

[0054] NVDIMM: Non-Volatile Dual In-line Memory Module

[0055] NVMe: Non-Volatile Memory Express

[0056] OS: Operating System

[0057] PaaS or PAAS: Platform as a Service

[0058] RAM: Random Access Memory

[0059] ROM: Read-Only Memory

[0060] SATA: Serial ATA (computer bus interface)

[0061] SLA: Service Level Agreement

[0062] SMM: System Management Mode

[0063] TCP / IP: Transmission Control Protocol / Internet Protocol

[0064] UEFI: Unified Extensible Firmware Interface

[0065] VM: Virtual Machine

[0066] WAN: Wide Area Network

[0067] Some additional terms

[0068] Reference is made herein to exemplary embodiments such as those shown in the accompanying drawings, and specific language is used herein to describe them. However, changes and further modifications to the features shown herein, as well as additional technical applications of the abstract principles shown in the specific embodiments herein, should be considered within the scope of the claims, as would occur to one skilled in the relevant art(s) having the present disclosure.

[0069] The meaning of terms is clarified in this disclosure, so these clarifications should be carefully noted when reading the claims. Specific examples are given, but (multiple) persons skilled in the relevant art(s) will understand that other examples may also fall within the meaning of the terms used and are within the scope of one or more claims. The meaning of terms here is not necessarily the same as their general usage (especially in non-technical usage), usage in a particular industry, or in a particular dictionary or dictionary collection. Figure numerals may be used with various phrases to help show the breadth of the terms. The omission of a figure numeral from a given text does not necessarily mean that the text does not discuss the contents of the figure. The inventor claims and exercises the right to specific and selected lexicographical compilations. Referenced terms are explicitly defined, but terms may also be implicitly defined without the use of quotation marks. Terms may be explicitly or implicitly defined in the detailed description here and / or elsewhere in the application documents.

[0070] As used herein, a "computer system" (also referred to as a "computing system") may include, for example, one or more servers, motherboards, processing nodes, laptops, tablet computers, personal computers (portable or non-portable), personal digital assistants, smart phones, smart watches, smart bands, cellular or mobile phones, other mobile devices having at least one processor and memory, video game systems, augmented reality systems, holographic projection systems, televisions, wearable computing systems, and / or other devices that provide one or more processors controlled at least in part by instructions. The instructions may be in the form of firmware or other software in memory and / or dedicated circuits.

[0071] A "multi-threaded" computer system is a computer system that supports multiple execution threads. The term "thread" should be understood to include code that is capable of or subject to scheduling and may be subject to synchronization. Outside of this disclosure, a thread may also be known by another name, e.g., such as "task", "process", or "coroutine". However, a distinction is made herein between a thread and a process because a thread defines an execution path within a process. Additionally, the threads of a process share a given address space, while different processes have different respective address spaces. The threads of a process may run in parallel, sequentially, or in a combination of parallel execution and sequential execution (e.g., time slicing).

[0072] A "processor" is a thread processing unit, such as a core in a simultaneous multi-threading implementation. A processor includes hardware. A given chip may house one or more processors. A processor may be general-purpose or customized for a specific purpose, such as vector processing, graphics processing, signal processing, floating-point arithmetic processing, encryption, I / O processing, machine learning, etc.

[0073] A "kernel" includes an operating system, a hypervisor, a virtual machine, BIOS or UEFI code, and similar hardware interface software.

[0074] "Code" refers to processor instructions, data (including constants, variables, and data structures), or both instructions and data. "Code" and "software" are used interchangeably herein. Executable code, interpreted code, and firmware are some examples of code.

[0075] As used extensively herein, a "program" includes applications, kernels, drivers, interrupt handlers, firmware, state machines, libraries, and other code written and / or automatically generated by a programmer (also referred to as a developer).

[0076] A "service" is a consumable program that provides resources or access to resources for multiple programs in a cloud computing environment or other network or computing system environment.

[0077] A "cloud" refers to pooled resources for computing, storage, and networking that can be flexibly used for metered on-demand services. A cloud can be private, public, community, or hybrid, and cloud services can be provided in the form of infrastructure as a service (IaaS), platform as a service (PaaS), software as a service (SaaS), or other services. Unless otherwise specified, any discussion of reading from or writing to a file includes reading / writing to a local file or reading / writing over a network, which can be a cloud network or other network, or both (local and network reading / writing).

[0078] "IoT" or "Internet of Things" refers to any networked collection of addressable embedded computing nodes. These nodes are examples of computer systems as defined herein, but they also have at least two of the following characteristics: (a) no local human-readable display; (b) no local keyboard; (c) the primary input source is a sensor that tracks non-verbal data sources; (d) no local rotating disk storage device - RAM chips or ROM chips provide the only local memory; (e) no CD or DVD drive; (f) embedded in household appliances or household fixtures; (g) embedded in implantable or wearable medical devices; (h) embedded in vehicles; (i) embedded in process automation control systems; or (j) designed with a focus on one of the following: environmental monitoring, civil infrastructure monitoring, industrial equipment monitoring, energy usage monitoring, human or animal health monitoring, physical security, or physical transportation system monitoring. IoT storage devices can be targeted for unauthorized access, whether via the cloud, via another network, or via a direct local access attempt.

[0079] In some cases, for example, in the context of the confidentiality-integrity-availability triad of cybersecurity discussions, a distinction can be made between the meaning of "data availability" and the meaning of "data integrity". But as used herein, "data integrity" means both the ability to access data and the ability to access data with the correct values. Thus, power loss of a storage device, value overwriting, irreversible encryption or encoding or compression, tampering, misplacement, scrambling, inaccessibility, and data value uncertainty are all examples of data integrity threats.

[0080] Unless otherwise explicitly stated, as used herein, "scrubbing" refers to flush scrubbing ("scrubbing" is an abbreviation of "scrubber"). Flush scrubbing is an operation that reduces the amount of unallocated memory near the first-flushed end of an aperture, or increases the distance of unallocated memory from the first-flushed end of the aperture, or both. For example, flush scrubbing can involve copying data, or moving data, or inserting data into previously unallocated memory.

[0081] "Flush scrubbing" differs from "consolidation scrubbing" in that they have different objectives. The objective of flush scrubbing is to preferentially flush critical data rather than non-critical data or garbage or irrelevant data in unallocated memory. In contrast, the objective of consolidation scrubbing is to reduce the time required to retrieve data from a storage device for use, particularly the time required to retrieve data from a rotating disk or tape. Consolidation scrubbing attempts to consolidate all data belonging to a particular owner (e.g., all data in a particular file) together. These are not equivalent operations.

[0082] For example, let A or B respectively represent the critical data owned by A or B, let the underscore _ represent the unallocated space in the memory, and use curly braces to represent the memory aperture boundary. Then changing {ABA_} to {AAB_} is consolidation defragmentation, but not flush defragmentation. In addition, changing {A_B_} to {AB__} is flush defragmentation (if the first flushed end is on the left), but not consolidation defragmentation.

[0083] As used herein, unless otherwise specified, "comprising" permits additional elements (i.e., comprising means including).

[0084] "Optimized" means improved, not necessarily perfect. For example, a program or algorithm that has been optimized can be further improved.

[0085] "Process" is sometimes used herein as a term in the field of computing science and, in that technical sense, includes users of computing resources. For example, it can also include or be referred to as coroutines, threads, tasks, interrupt handlers, application processes, kernel processes, procedures, or object methods. In fact, a "process" is a computing entity identified by system utilities such as Task Manager, ps or similar utilities in other operating system environments (trademarks of Microsoft Corporation and Linus Torvalds, respectively). "Process" is also used herein as a term in the field of patent law, for example, when describing process claims as opposed to system claims or claims for articles of manufacture (configured storage media). Similarly, "method" is sometimes used herein as a technical term in the field of computing science (a "routine") and also as a term in the field of patent law ("process"). "Process" and "method" in the sense of patent law are used interchangeably herein. One skilled in the art will understand what the intention is in a particular case and will also understand that a given claimed process or method (in the sense of patent law) can sometimes be implemented using one or more processes or methods (in the sense of computing science).

[0086] In contrast to the case without automation, "automatic" means by using automation (e.g., general-purpose computing hardware configured by software for the particular operations and technical effects discussed herein). Specifically, steps that are "automatically" performed are not performed manually on paper or in the human mind, although they can be initiated by a person or interactively guided by a person. A machine is utilized to perform the automatic steps in order to obtain one or more technical effects that would not be achieved without such provided technical interaction.

[0087] Those skilled in the art understand that the technical effect is the assumed purpose of the technical embodiment. For example, in an embodiment, calculations are involved, and the fact that some calculations can also be performed without technical components (e.g., by paper and pen, or even as a mental step) does not eliminate the existence of the technical effect or change the specific and technical nature of the embodiment. It can be understood that data preservation operations such as identifying unallocated memory, copying data to memory, marking the memory as allocated, flushing data from volatile memory to non-volatile memory, and many other operations discussed herein are digital in nature. The human brain cannot directly interface with a CPU or other processor, or with RAM or other digital memory, to read and write the necessary data to perform the data preservation steps taught herein. Given this disclosure, those skilled in the art will well understand this, but sometimes it may be necessary to inform or remind others of this point.

[0088] "Computationally" also refers to the computing device being used (at least a processor plus memory), excluding obtaining results solely through human thought or solely through human behavior. For example, doing arithmetic with paper and pen is not arithmetic calculation as understood herein. The computational result is faster, broader, deeper, more accurate, more consistent, more comprehensive, and / or provides a technical effect beyond the scope of human performance in other ways. A "computational step" is a step performed by computation. Neither "automatically" nor "computationally" necessarily means "immediately". Here, "computation" and "automatic" are used interchangeably.

[0089] "Proactive" means without a direct request from the user. In fact, the user may not even be aware that the proactive step of the embodiment is possible until the result of the step has been presented to the user. Unless otherwise stated, any computational and / or automatic step described herein can also be done proactively.

[0090] Throughout the document, the optional plural forms "(s)", "(several)", or "(plural)" are used to indicate the existence of one or more of the indicated features. For example, "(s) processor" means "one or more processors" or equivalently "at least one processor".

[0091] For the purposes of U.S. law and practice, in the claims or elsewhere, the word "step" as used herein is not intended to invoke the claim interpretation of means-plus-function, step-plus-function, or 35 U.S.C. § 112, ¶ 6 / § 112(f). Any presumption in this regard is hereby expressly rebutted.

[0092] For purposes of U.S. law and practice, these claims are not intended to invoke a means-plus-function interpretation unless they use the term "means for" a component. Claim language that is intended to be interpreted as means-plus-function language (if any) will expressly state that intention by use of the phrase "means for" a component. When means-plus-function interpretation applies, whether by use of "means for" and / or by a court's legitimate interpretation of claim language, the components stated in the specification for a given noun or given verb shall be understood to be linked to the claim language and are linked together herein by any of the following: appearing within the same box in a block diagram of the drawings, being denoted by the same or similar names, being denoted by the same reference numerals, a functional relationship described in any of the drawings, a functional relationship noted in the present disclosure. For example, if a claim limitation recites "zac widget" and that claim limitation is subject to means-plus-function interpretation, then all structures identified anywhere in any drawing box, paragraph, or example that mentions "zac widget" in the specification, at least, or bundled by any reference numeral assigned to the zac widget, or disclosed as having a functional relationship to the structure or operation of the zac widget, will be considered to be part of the structure identified in the application for the zac widget and will contribute to defining the set of equivalent forms of the zac widget structure.

[0093] Those skilled in the art will recognize that the present invention discloses and discusses various data values and data structures and will recognize that these items reside in a memory (RAM, disk, etc.) and thereby configure the memory. Those skilled in the art will also recognize that the present invention discloses and discusses various algorithmic steps to be embodied in executable code in a given implementation and that such code also resides in the memory and that it effectively configures any general-purpose processor that executes it, thereby transforming it from a general-purpose processor into a special-purpose processor that is functionally dedicated hardware.

[0094] Accordingly, those skilled in the art will not make the mistake of treating (a) the memory recited in the claims and (b) the data structures or data values or code recited in the claims as non-overlapping items. The data structures and data values and code are understood to reside in the memory even if the claims do not expressly recite the residence of every data structure or data value or code segment mentioned. Accordingly, no express recitation of such residence is required. However, they are not prohibited, and one or two alternative recitations may be presented for emphasis without thereby excluding all other data values and data structures and code from residence. Similarly, the code functions recited in the claims are understood to configure the processor regardless of whether the claims expressly recite that configuration quality.

[0095] Throughout this document, unless otherwise explicitly stated, any reference to a step in a process is assumed to be one that can be directly performed by a party in interest and / or indirectly performed by that party through an intervention mechanism and / or intervention entity and still be within the scope of that step. That is, unless direct performance is an explicitly stated requirement, the party in interest is not required to directly perform the step. For example, steps involving actions of a party in interest regarding a destination or other subject matter, such as allocate, copy, defragment, specify, detect, determine, eliminate, execute, fail, flush, identify, hold, mark, move, place, save, provide, receive, reduce, retain, reside, resize, restore, archive, send, specify, store (and allocate, allocated, copy, copied, etc.), may involve intervention actions by some other party, such as forward, copy, upload, download, encode, decode, compress, decompress, encrypt, decrypt, authenticate, invoke, etc., including any actions recited in this document, but are still understood to be directly performed by the party in interest.

[0096] Whenever data or instructions are mentioned, it should be understood that these items configure a computer-readable memory and / or a computer-readable storage medium so as to transform it into a particular article, rather than simply existing, for example, on paper, in a human mind, or merely as a signal propagating on a wire. For the purposes of patent protection in the United States, in accordance with the interpretation by the United States Patent and Trademark Office (USPTO) in the In Re Nuijten case, a memory or other computer-readable storage medium is not a propagating signal, a carrier wave, or mere energy that is outside the scope of patentable subject matter. In the United States, no claim covers a signal itself or mere energy, and any claim interpretation to the contrary asserted in view of this disclosure is prima facie unreasonable. Unless otherwise explicitly stated in claims authorized outside the United States, the claims do not cover a signal itself or mere energy.

[0097] Furthermore, despite any apparent contrary indication elsewhere in this document, the distinct difference between (a) on the one hand, a computer-readable storage medium and a computer-readable memory, and (b) on the other hand, a transmission medium (also known as a signal medium) should be understood. A transmission medium is a computer-readable medium that propagates a signal or a carrier wave. In contrast, a computer-readable storage medium and a computer-readable memory are not computer-readable media that propagate a signal or a carrier wave. Unless otherwise explicitly stated in the claims, "computer-readable medium" refers to a computer-readable storage medium, not the propagating signal itself or mere energy.

[0098] "Embodiments" here are examples. The term "embodiment" cannot be interchanged with "the present invention". Embodiments can freely share or borrow aspects to create other embodiments (as long as the result is operable), even if the combinations of the resulting aspects are not explicitly described herein. For those skilled in the art, it is unnecessary to require explicit and separate description of each allowable combination, and it is contrary to the policy of recognizing that the patent specification is written for those skilled in the art. Both formal combinatorial calculations and informal common intuition regarding the number of possible combinations resulting from even a small number of combinable features will indicate that there are a large number of aspect combinations for the aspects described here. Therefore, requiring explicit recording of each combination will run counter to the policies of requiring the patent specification to be concise and the reader to have a certain understanding of the relevant technical field.

[0099] List of Reference Numerals

[0100] For convenience, and to support the figures, the following list is provided as part of the body of the specification, where aspects of the present invention are described by reference to multiple items. Items not listed here may still be part of a given embodiment. For the sake of readability of the text, certain (but not all) occurrences of items cited in the text are accompanied by the given reference numerals. The same reference numeral may be used for different examples or different instances of a given article. The list of reference numerals is as follows:

[0101] 100 Operating environment, also referred to as a computing environment

[0102] 102 Computer system, also referred to as a computing system or a computational system

[0103] 104 User

[0104] 106 Peripheral device

[0105] 108 Network, generally including, for example, LAN, WAN, software-defined networks, clouds, and other wired or wireless networks

[0106] 110 Processor

[0107] 112 Computer-readable storage medium, such as RAM, hard disk

[0108] 114 Removably configurable computer-readable storage medium

[0109] 116 Instructions executable by a processor; may be on a removable storage medium or in other memory (volatile or non-volatile or both)

[0110] 118 Data

[0111] 120 (Multiple) kernels, such as (multiple) operating systems, BIOS, UEFI, device drivers

[0112] 122 Tools, such as antivirus software, firewalls, packet sniffer software, intrusion detection systems, intrusion prevention systems, debuggers, profilers, compilers, interpreters, decompilers, assemblers, disassemblers, source code editors, auto-completion software, simulators, fuzzers, repository access tools, version control tools, optimizers, collaboration tools, software development tools and tool suites, hardware development tools and tool suites, diagnostic tools, etc.

[0113] 124 Applications, such as word processors, web browsers, spreadsheets, games, email tools

[0114] 126 Display screens, also known as "monitors"

[0115] 128 Computing hardware not otherwise associated with reference numerals 106, 108, 110, 112, 114; in Figure 8 which, reference numeral 128 also refers to the processor 110 and memory 112 hardware 200 computing functionality, such as in a computing node

[0116] 202 Computing nodes

[0117] 204 Data sources, such as programs that output or otherwise generate data

[0118] 206 Virtual machines, such as computing constructs that provide hardware virtualization and include an operating system; may include, for example, working memory resources, CPU resources, IO resources, and non-volatile memory

[0119] 208 System code, such as operating systems, hypervisors, BIOS, UEFI, firmware 210 Storage functionality, such as in a storage node

[0120] 212 Storage nodes

[0121] 214 Non-volatile storage devices, such as NVDIMMs, disks, flash memory

[0122] 216 Batteries

[0123] 218 Battery-backed memories

[0124] 220 Memory apertures; may refer to a portion of the physical address space associated with a digital data storage location in a physical device, or may refer to the digital data storage location itself

[0125] 222 Data storage requests

[0126] 224 Responses to data storage requests

[0127] 226 Data copies

[0128] 300 Computing environment aspects

[0129] 302 Cloud; may also be referred to as "cloud computing environment"

[0130] 304 Data storage circuit; includes electronic devices and any firmware that controls these electronic devices or their use for the data storage circuit; does not necessarily include the non-volatile memory to which data is flushed

[0131] 306 Data storage subsystem; includes the non-volatile storage device to which data is flushed

[0132] 308 Hypervisor

[0133] 310 UEFI; may refer to the Unified Extensible Firmware Interface, or to firmware having such an interface; for example, may include or provide functions of SEC (Security), PEI (Pre-EFI Initialization), DXE (Driver Execution Environment), BDS (Boot Device Selection), SMM (System Management Mode)

[0134] 312 Flush order

[0135] 314 Data integrity threats

[0136] 316 Storage relationship between two or more nodes

[0137] 318 Data criticality; may refer to whether data has been designated as critical, or to whether data meets criteria that I consider to be designated as critical, even if not so designated yet; may be a boolean value or another indication, for example, a value in a range of two or more values indicating how data flush priority is determined

[0138] 402 Volatile memory, e.g., DRAM itself

[0139] 404 NVDIMM

[0140] 406 Battery characteristics

[0141] 408 Defragmenter (memory defragmentation) code

[0142] 410 Defragmented portion of the memory, i.e., the portion to be defragmented

[0143] 412 Dataset including critical data (may also have some non-critical data)

[0144] 414 Memory allocated (or being allocated) to store critical data 412

[0145] 416 Dataset not including any critical data

[0146] The memory 418 that is allocated (or being allocated) to store the non-critical data 416

[0147] 420 Firmware

[0148] 422 Allocator software, i.e., the software that identifies the unallocated space in the aperture, places the critical data therein at the earliest flushed end of the aperture, and marks the space as allocated; the allocator can also defragment the aperture; the allocator can also identify the space for non-critical data and place the non-critical data at the last flushed end of the aperture

[0149] 424 Physical address in the memory 112; can refer to the address (e.g., 0x0000) itself or the storage location at the address (e.g., storage cell)

[0150] 502 Battery capacity, e.g., in milliampere-hours

[0151] 504 Reliability value, e.g., mean time between failures or remaining expected life, or an enumerated value (e.g., 2 in a specification where 10 is the best and 0 is the worst) or a reliability level (e.g., medium reliability).

[0152] 506 Battery life, e.g., hours elapsed since installation, or number of charge cycles experienced, or number of times used during flushing

[0153] 600 Example of data criticality criteria

[0154] 602 Data criticality criteria

[0155] 604 Priority assigned to the virtual machine that generates the data

[0156] 606 Customer, e.g., cloud tenant

[0157] 608 Status assigned to the customer; can be explicit, e.g., in the SLA

[0158] 610 Workload, e.g., processing tasks assigned to a specific computing node

[0159] 612 Priority assigned to the workload that includes data or generates data from its execution

[0160] 702 Physical machine, e.g., processor, circuit, chip, battery, display, power supply, enclosure, etc., including any embedded firmware or software stored in the non-volatile memory of the physical machine; the computing system 102 includes one or more physical machines 702

[0161] 704 Node in the cloud or other network; the computing node 202 and the storage node 212 are examples of nodes 704

[0162] 706 A container, e.g., a computing construct that provides user space virtualization and does not itself include an operating system but still relies on an operating system to execute

[0163] 800 A computing system having one or more computing nodes networked with one or more storage nodes and equipped with the data preservation functionality taught herein

[0164] 802 I / O (input / output) hardware and software, e.g., ports, connectors, sockets, network interface cards; may also be referred to as "IO"

[0165] 804 Top of memory

[0166] 806 The first flushed end of an aperture

[0167] 808 The last flushed end of an aperture

[0168] 810 Unallocated space within an aperture

[0169] 812 A memory portion within the same storage device but outside the aperture

[0170] 814 The logical mapping between virtual address space and physical address space

[0171] 900 A computing system of a system having an operating system running virtual machines or containers and equipped with the data preservation functionality taught herein

[0172] 902 A non-volatile storage device 112 dedicated to receiving data flushed thereto

[0173] 904 A memory region mapped to a dedicated storage device 902; the battery-backed memory aperture 220 is an example of region 904

[0174] 906 Operating system

[0175] 908 A preservation operation of copying data from volatile memory to a dedicated non-volatile memory for backup volatile memory; the preservation may be done intentionally without any impending threat to the integrity of the data being preserved, or it may be a flush done in response to an impending or current threat

[0176] 910 A recovery operation of copying data from a dedicated non-volatile storage device to volatile memory backed by the dedicated non-volatile storage device

[0177] 912 Execution of a program; may refer to the act of executing a program or an instance of a program operation

[0178] 1000 Flowchart; 1000 also refers to a data preservation method as shown in or consistent with the flowchart Figure 10 As shown in the flowchart or consistent with it

[0179] 1002 receives a request to store critical data, e.g., as a result of a malloc() call, constructor call, file or block or blob save, or other operations seeking to store data in an unallocated storage location (bytes, pages, etc.).

[0180] 1004 identifies unallocated portions of memory, e.g., using system memory management software, free lists, bit vectors showing free blocks, or similar mechanisms

[0181] 1006 marks a portion of memory as allocated, e.g., by setting bits in a data structure that tracks memory allocations

[0182] 1008 places a copy of the data in memory, e.g., by performing functions used by memcpy(), block copy, etc.

[0183] 1010 detects threats to data integrity, e.g., by receiving a power command on an ACPI-compliant system, or by receiving a signal from a voltage level monitoring circuit

[0184] 1012 flushes data from a volatile storage device to a dedicated non-volatile backup storage device, e.g., by copying data from the DRAM portion of an NVDIMM to the flash portion of the NVDIMM

[0185] 1100 Flowchart; 1100 also refers to a data saving method shown by or consistent with the Figure 11 flowchart (which combines the steps of Figure 9 and the steps of Figure 10 )

[0186] 1102 Save data, i.e., maintain the integrity of at least one copy of the data

[0187] 1104 Receive a data storage request

[0188] 1106 Determine or specify the size or location of aperture 220

[0189] 1108 The size of aperture 220, e.g., in bytes or blocks

[0190] 1110 The location of aperture 220, e.g., the offsets of the two ends of the aperture relative to the lowest physical address of the storage device containing the aperture

[0191] 1112 Send a data storage request

[0192] 1114 Resize the aperture

[0193] 1115 Move the aperture

[0194] 1116 Reserve at least a specified portion of the aperture to save only non-critical data

[0195] Reserved portion of the 1118 aperture; may be specified by an absolute or relative memory address, or as a percentage of the total aperture size

[0196] 1120 Defragment at least a portion of the aperture by copying data or moving data, or inserting data into previously unallocated memory, thereby reducing the amount of unallocated memory or increasing the distance of the unallocated memory from the earliest flushed end of the aperture, or both; this may be referred to as "flush defragmentation" to distinguish it from other uses of "defragmentation" that involve combining memory portions belonging to, for example, the same user or the same process or the same file

[0197] 1122 Failure to flush a portion of the data from the aperture; this results in at least partial loss of the data, unless there is another copy of the data available somewhere outside the aperture and outside the non-volatile storage device into which the other portions of the data are flushed

[0198] 1124 Reside in a physical machine, e.g., stored in memory (volatile or non-volatile) that is part of the physical machine

[0199] 1126 Save a copy of the data somewhere outside the aperture and also outside the non-volatile memory to which the aperture will be flushed

[0200] 1128 Move data

[0201] 1130 Make space, i.e., change a given portion of the memory from allocated to free

[0202] 1132 Designate data as critical; when data is considered as a unit, designating any part of the unit as critical also designates the unit as critical

[0203] 1134 Reduce the amount of unallocated space between allocated portions

[0204] 1136 Eliminate the unallocated space between allocated portions

[0205] 1138 Provide a better likelihood of saving critical data against threats

[0206] 1140 Likelihood of saving critical data against threats; may be a measured probability or a reasonable assessment

[0207] 1142 Execute firmware; an example of executing 912

[0208] 1144 Any step discussed in this disclosure that is not given some other reference numeral

[0209] Operating environment

[0210] Reference Figure 1 In an embodiment, the operating environment 100 includes at least one computer system 102. The computer system 102 may or may not be a multi-processor computer system. The operating environment may include one or more machines in a given computer system, which may be clustered, client-server networked, and / or peer-networked within a cloud. A single machine is a computer system, and a group of cooperating machines is also a computer system. A given computer system 102 may be configured for end users, e.g., to utilize applications, for administrators, as a server, as a distributed processing node, and / or otherwise.

[0211] A human user 104 can interact with the computer system 102 via typing text, touch, voice, movement, computer vision, gestures, and / or other forms of I / O by using a display, keyboard, and other peripheral devices 106. The screen 126 may be a removable peripheral device 106 or may be part of the system 102. The user interface may support the interaction between the embodiment and one or more human users. The user interface may include a command-line interface, a graphical user interface (GUI), a natural user interface (NUI), a voice command interface, and / or other user interface (UI) presentations, which may be presented as different options or may be integrated.

[0212] System administrators, network administrators, cloud administrators, security analysts and other security personnel, operators, developers, testers, engineers, auditors, customers, and end users are all specific types of users 104. An automated agent, script, playback software, device, etc. that represents the actions of one or more persons may also be a user 104, e.g., to facilitate testing of the system 102. Storage devices and / or networking devices may be considered peripheral devices in some embodiments and part of the system 102 in other embodiments, depending on their removability from the processor 110. For example, Figure 1 Other computer systems not shown may use one or more connections to the network 108 via a network interface device to interact technically with the computer system 102 or with another system embodiment.

[0213] Each computer system 102 includes at least one processor 110. The computer system 102, like other suitable systems, also includes one or more computer-readable storage media 112. The storage media 112 can be of different physical types. The storage media 112 can be volatile memory, non-volatile memory, fixed-location media, removable media, magnetic media, optical media, solid-state media, and / or other types of physically durable storage media (as opposed to merely propagated signals or merely energy). Specifically, a configured storage media 114 such as a portable (i.e., external) hard drive, CD, DVD, memory stick, or other removable non-volatile storage media can, when inserted or otherwise installed, functionally become a technical part of the computer system such that its contents are accessible for interaction and use by the processor 110. The removable configured storage media 114 is an example of the computer-readable storage media 112. Some other examples of the computer-readable storage media 112 include built-in RAM, ROM, hard drives, and other memory storage devices that are not easily removable by the user 104. For compliance with current U.S. patent requirements, under any claim pending or issued in the United States, a computer-readable medium, a computer-readable storage medium, or a computer-readable memory is not a signal itself nor merely energy.

[0214] The storage media 114 is configured with binary instructions executable by the processor 110; for example, "executable" as used herein in this broad sense includes machine code, interpretable code, bytecode, and / or code that runs on a virtual machine. The storage media 114 is also configured with data 118 that is created, modified, referenced, and / or otherwise used for a technical effect by executing the instructions 116. The instructions 116 and the data 118 configure the memory or other storage media 114 in which they reside; when that memory or other computer-readable storage media is a functional part of a given computer system, the instructions 116 and the data 118 also configure that computer system. In some embodiments, a portion of the data 118 represents real-world items such as product characteristics, inventory, physical measurements, settings, images, readings, targets, volumes, etc. This data is also transformed by backup, recovery, commit, abort, reformatting, and / or other technical operations.

[0215] Although embodiments may be described as software instructions executed by one or more processors in a computing device (e.g., a general-purpose computer, server, or cluster), such a description does not imply an exhaustive list of all possible embodiments. Those skilled in the art will understand that the same or similar functionality can generally also be implemented directly in hardware logic, in whole or in part, to provide the same or similar technical effect. Alternatively, or in addition to software implementation, the technical functions described herein can be performed at least in part by one or more hardware logic components, which may include firmware or be controlled by firmware, or both. For example, without excluding other implementations, embodiments can include hardware logic components 110, 128, such as field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-chip components (SOCs), complex programmable logic devices (CPLDs), and the like. For example, the components of an embodiment can be grouped into interactive functional modules based on their inputs, outputs, and / or their technical effects.

[0216] In addition to processor 110 (e.g., CPU, ALU, FPU, and / or GPU), memory / storage medium 112, and display 126, the operating environment can also include other hardware 128, such as batteries, buses, power supplies, wired and wireless network interface cards. The terms "screen" and "display" are used interchangeably in the text. Display 126 can include one or more touchscreens, a screen responsive to input from a pen or tablet, or a screen that operates only for output. In some embodiments, peripheral devices 106, such as human user I / O devices (screens, keyboards, mice, tablets, microphones, speakers, motion sensors, etc.), will communicate operably with one or more processors 110 and memory.

[0217] In some embodiments, the system includes multiple computers connected by a wired and / or wireless network 108. The network interface device 128 can provide access to network 108 using network components such as packet-switching network interface cards, wireless transceivers, or telephone network interfaces that may be present in a given computer system. Virtualization of network interface devices and other network components (such as switches or routers or firewalls) can also exist in, for example, software-defined networks or sandboxes or other secure cloud computing environments. A given embodiment can also transfer technical data and / or technical instructions by direct memory access, removable non-volatile storage media, or other information storage-retrieval and / or transmission methods.

[0218] Those skilled in the art will recognize that the foregoing aspects and other aspects presented under "operating environment" herein can form part of a given embodiment. The headings of this document are not intended to strictly categorize specific features as sets of embodiments and non-embodiments.

[0219] One or more items are shown in outline form in the figures, or are listed in parentheses, to emphasize that they are not necessarily part of the operating environment shown or of all embodiments, but may interoperate with items in the operating environment discussed herein or in some embodiments. This does not mean that items that are not in outline or parentheses form are essential in any figure or any embodiment. In particular, the provision of Figure 1 is for convenience; the inclusion of an item in Figure 1 does not mean that the item or the described use of the item was known prior to the present invention.

[0220] More information about the system

[0221] Refer to Figures 1 to 11 , some embodiments use or provide a system 800 or 900 with enhanced functionality. This enhanced functionality helps to promote data preservation by implementing a combination of operating steps and device characteristics to protect data from integrity threats 314. Data 118 is divided into critical data 412 and non-critical data 416, and the critical data is stored in one or more battery-backed memories 218 apertures 220 at physical address 424, where the critical data will be flushed 1012 before the non-critical data when a threat 1010 is detected. The battery-backed memory aperture 220 is sized according to the characteristics 406 of the battery and can be resized as conditions change. A defragmenter 408 optimizes the use of the aperture to store critical data 412.

[0222] As shown in the example architecture of Figure 2 , data 118 can be generated by a source 204, such as a virtual machine 206 or a container 706 running 912 on a computing node 202. These data sources call or rely on system code 208 to send the generated data 118 to a storage node 212. In this Figure 2 example, the storage node 212 has a fast volatile memory 402, at least a portion of which is a battery-backed memory 218. At least a portion of the battery-backed memory 218 is designated as an aperture 220 to help preserve critical data, as taught herein. The storage node 212 and the computing node 202 communicate via a request 222 (e.g., here is the data to be stored, please send back the data stored under identifier Z) and a response 224 (e.g., successfully stored, out of space in the aperture, here is the requested data, or other status or error code). For example, the request 222 and the response 224 can be transmitted over a TCP / IP network 108, an Ethernet 108, a storage area network 108, or a combination thereof.

[0223] Typically, the compute node - storage node relationship is not necessarily one - to - one. A compute node can process data sent or received from one storage node, or communicate with multiple storage nodes; a storage node can store data on behalf of one or more compute nodes. For example, Figure 8 illustrates one compute node 202 using four storage nodes 212.

[0224] As Figure 2 and Figure 8 shown, in some embodiments, critical data 412 is not replicated across storage nodes, while non - critical data 416 is replicated across storage nodes. Thus, critical data can be preserved from threats mainly or only by flushing it to non - volatile storage devices, while non - critical data can be preserved from threats mainly or only by replicating it across nodes.

[0225] In Figure 8 the critical data 412 in aperture 220 and the corresponding allocation regions 414 belong to virtual machines A, C, and D, and are represented by rounded corners in the figure, while the non - critical data 416 in aperture 220 and the corresponding allocation regions 418 belong to virtual machines B and X, and are represented by right angles in the figure. Data replicas 226 of data belonging to virtual machines B and X are shown, where storage nodes S1, S2, S3 have replicas of non - critical data 416 of virtual machine B, and storage nodes S1, S2, S4 have replicas of non - critical data 416 of virtual machine X.

[0226] Figure 8 and Figure 9 illustrate some examples of system code 208, namely kernel 120 (e.g., hypervisor 308, operating system 906) and firmware (e.g., UEFI firmware 310). The compute node and storage node are the computing system 102 and thus include hardware. Although for clarity, hardware other than memory 112 is not explicitly labeled in Figure 2 it, some examples of hardware are labeled in Figure 8 such as, for example, IO 802 and CPU 110, as well as memory 112, 218. Memory 218 includes aperture 220, memory outside the aperture 812, allocated memory 418 and 414 within the aperture, and unallocated memory 810 within the aperture.

[0227] The physical address 424 of the battery-backed memory 218 or another mapped volatile region 904 that includes an aperture can be mapped to a virtual address through the mapping 814. The mapping 814 implements a degree of indirection that allows the same save 908 and restore 910 firmware to be used with apertures that can have different physical start addresses 806 and physical end addresses 808 from each other. The non-volatile memory 112 to which the aperture 220 data is flushed can include flash memory or other non-volatile memory in the NVDIMM 404 or other dedicated non-volatile memory 902, or both.

[0228] Figure 3 Aspects 300 of some computing environments 100 are shown, and these aspects are discussed appropriately at various points in the present disclosure.

[0229] Figure 4 An example data save subsystem 306 is shown in which the innovative data save functionality of the present invention can be added to a computing system 102. The memory of the subsystem 306 includes an aperture 220 in the battery-backed memory 218 (e.g., NVDIMM 404). As Figure 4 and Figure 5 shown, the battery can have characteristics 406 that are tracked or otherwise indicated, such as one or more of capacity 502, reliability 504, and lifespan 506. These battery characteristics 406 can overlap or affect each other; for example, reliability may decrease as lifespan increases. The subsystem 306 can also include volatile memory 402 that is not supported by the battery 216.

[0230] In operation, the aperture 220 will have one or more regions 414 that contain critical data 412, and the aperture 220 can have one or more regions 418 that contain non-critical data 416. For example, as Figure 8 shown by storage nodes S2, S3, S4, at least at a given point in time, some apertures 220 can contain only non-critical data, but the focus of interest herein is on apertures 220 that do contain critical data. Specifically, the present disclosure teaches how to place critical data in the memory 112 (through initial allocation, or through defragmentation, or both) to increase the likelihood 1140 of successfully flushing the critical data and thus saving 1102 the critical data in response to a data threat.

[0231] Figure 6Some examples 600 of criteria 602 that can be used to distinguish critical data 412 from non - critical data 416 are shown. For example, if the data is generated by a virtual machine 206 or a container 706 or another digital artifact, and the artifact has a high priority 604, the data can be designated as critical data. If the data 118 is generated by or on behalf of a particular customer 606 that has paid or otherwise obtained an indication of the status 608 of critical data, the data 118 can be designated 1132 as critical data. Similarly, if the data is generated by or on behalf of or processed by a particular workload 610 that has an indication of the priority or status 612 of critical data, the data can be designated 1132 as critical data. These example criteria can be combined in various ways, or modified along with other criteria 602, or both. For example, (a) data can be designated as critical data when the data is generated by a high - priority virtual machine of a high - status customer but is otherwise designated as non - critical; (b) data can be designated as critical data when the data is generated by any virtual machine of a high - status customer running on a designated security server blade 702 but is otherwise designated as non - critical; (c) data can be designated as critical data when the data is generated by any customer during a designated period but is otherwise designated as non - critical data, and so on.

[0232] Figure 7 Various examples of some computing systems 102 are shown, which are appropriately discussed at various points in the present disclosure.

[0233] Some embodiments provide or use a data preservation subsystem 306 in the computing system 102. The data preservation subsystem 306 can include a battery - backed memory 218 having an aperture 220. The aperture 220 has a flush order 312, which is the order in which data is copied from the aperture to non - volatile memory 902 associated with the aperture in response to a data integrity threat 314. The flush order defines the first - to - flush end 806 and the last - to - flush end 808 of the aperture.

[0234] In cases where the higher physical address 424 is above the lower physical address, phrases such as "top - down" or "bottom - up" can also be used to describe the flush order. Then a top - down flush will start from the higher address and continue to the lower address, while a bottom - up flush will start from the lower address and continue to the higher address. In accordance with the top - down flush order, the first - to - flush end 806 of the aperture will have a higher address than the last - to - flush end 808 of the aperture, and in accordance with the bottom - up flush order, the first - to - flush end 806 of the aperture will have a lower address than the last - to - flush end 808 of the aperture. In Figure 8In this case, a bottom-up (low address to high address) flushing order 312 is shown because the first flushing end 806 of the aperture has a lower physical address 424 than the last flushing end 808 of the aperture.

[0235] In some embodiments, the data preservation subsystem 306 includes data preservation circuitry 304 that is operatively communicable with the battery-backed memory 218. The data preservation circuitry 304 is configured to perform data preservation steps that may include: (a) receiving 1002 a request 222 to store dataset A-data 412 in the aperture, where A-data includes data designated as critical data, (b) identifying 1004 a portion A-memory 414 of the unallocated memory 810 of the aperture, where A-memory is large enough to hold A-data and has an address 424 closer to the first flushing end 806 of the aperture than any other address of any other unallocated memory of the aperture, (c) marking 1006 A-memory as allocated and placing 1008 a copy of A-data in A-memory, (d) receiving a request 222 to store dataset Z-data 416 in the aperture, where Z-data does not include any data designated as critical data, (e) identifying a portion Z-memory of the unallocated memory 810 of the aperture, where Z-memory is large enough to hold Z-data and has an address 424 closer to the last flushing end 808 of the aperture than any other address of any other unallocated memory of the aperture, and (f) marking Z-memory as allocated and placing a copy of Z-data in Z-memory. The labels (a) through (f) here are only used as identifiers and thus do not necessarily specify an order of operations themselves. The data preservation subsystem 306 provides a higher likelihood of successfully flushing 1012 critical data and thus preserving it by placing critical data before non-critical data in the battery-backed memory in the flushing order. The likelihood of critical data preservation is higher than in an architecture that stores data in the battery-backed memory without regard to the criticality or lack thereof of the data.

[0236] In some embodiments, the battery-backed memory aperture 220 resides in or is controlled by a network node 704 designated herein as X, and the request 222 to store A-data in the aperture is sent from a different network node designated herein as Y. For example, in Figure 8 , the aperture resides in storage nodes 212, 704S1, and the request 222 to store A-data of virtual machine A in the aperture 220 is sent from a network node different from S1 (i.e., from a compute node). In some embodiments, network node X includes a storage node in the cloud 302, and network node Y includes a compute node in the cloud. However, in other embodiments, the nodes may reside in a non-cloud network.

[0237] In some embodiments, network node X and network node Y are associated by a storage relationship 316 designated herein as R, where R is defined such that two nodes M and N are associated by R when M stores data on behalf of N. In some cases, R is not a one-to-one relationship for specific nodes X and Y. For example, a compute node may send critical data of virtual machine VM7 to storage node S7 and send critical data of virtual machine VM8 to storage node S8. Or storage node S12 may hold critical data from compute node C10 and may also hold critical data from a different compute node C11. Of course, many other examples of relationship 316 are possible.

[0238] In some embodiments, the battery-backed memory 218 includes NVDIMM memory. For example, NVDIMM 404 may include NVDIMM-F flash memory, NVDIMM-N byte-addressable memory, NVDIMM-P memory having dynamic RAM and NAND on the same device, NVDIMM-SW memory, or NVRAM non-volatile RAM memory. NVDIMM-SW may include memory (e.g., DD4 RAM), disk, or flash memory (e.g., SSD partition), battery, and signaling mechanism.

[0239] In some embodiments, the computing system 102 includes virtual machines 206, and dataset A-data 412 is sent from virtual machine VM-A to the data preservation subsystem 306, and dataset Z-data 416 is sent from virtual machine VM-Z to the data preservation subsystem 306. For example, in Figure 8 the example, the critical dataset 412 is sent from virtual machine A to the enhanced storage node S1 (and thus to its data preservation subsystem 306), and the non-critical dataset 416 is sent from virtual machine B to S1.

[0240] Through the extensive discussion of computing hardware herein, other system embodiments are also described herein, which are directly or derivatively system versions of the described processes or configured media.

[0241] Although specific architectural examples are shown in the figures, embodiments may differ from these examples. For example, in an embodiment, items shown in different figures may be included together, items shown in the figures may be omitted, functions shown in different items may be combined into fewer items or a single item, items may be renamed, or items may be connected differently from each other.

[0242] Examples are provided here to help illustrate aspects of the technology, but the examples given within this document do not describe all possible embodiments. The embodiments are not limited to the specific examples, component names, optimizations, algorithm selections, data, data types, configurations, implementations, arrangements, displays, features, methods, or scenarios provided herein. For example, a given embodiment may include additional or different technical features, mechanisms, sequences, data structures, or functions and may otherwise deviate from the examples provided herein.

[0243] Process (also referred to as method)

[0244] Figure 10 Method 1000 is shown, which is an example of a method that can be performed or assisted by an enhanced system with data preservation capabilities taught herein and is illustrated by one or more of the Figures 1 to 9 figures. Figure 10 The steps of receiving 1002, identifying 1004, marking 1006, placing 1008, detecting 1010, and flushing 1012 shown in are discussed throughout this disclosure, whether or not those reference numerals are explicitly recited. Specifically, Figure 2 、 Figure 8 and Figure 9 the discussion of involves Figure 10 the steps 1002 to 1012 shown in.

[0245] Figure 11 An access control method (which may also be referred to as a "process" in the legal sense of the term) suitable for use during the operation of a system with enhanced data preservation capabilities is also shown, including some improvements, supplements, or contextual actions for the Figure 10 steps shown in. Figure 11 It also incorporates Figure 9 or Figure 10 the steps shown in. Unless otherwise specified, the technical processes shown in the figures or otherwise disclosed will be automatically performed, for example, by the data preservation subsystem 306. The process can also be performed partially automatically and partially manually to the extent that actions involving a human administrator or other person are involved. For example, in some embodiments, a person can specify the minimum or maximum percentage of the aperture to be reserved 1116 for non-critical data. No process considered to be the invention herein is entirely manual. In a given embodiment, different parameters or data to be operated on can be utilized, and zero or more of the shown steps of the process can be repeated. The steps in the embodiments can also be in an order different from that of Figure 10 and Figure 11be completed in a different order from top to bottom as shown. The steps may be performed serially, in a partially overlapping manner, or fully in parallel. Specifically, the flowchart 1000 action items or the flowchart 1100 action items are traversed to indicate that the order of steps performed during the process can vary from one manifestation of the process to another. The flowchart traversal order can also vary from one process embodiment to another. Steps may also be omitted, combined, renamed, regrouped, performed on one or more machines, or otherwise deviate from the shown flow, as long as the process performed is operable and meets at least one claim.

[0246] Some embodiments use or provide a method for data preservation in a computing system. The method may include receiving 1002 a request to store dataset A - data in an aperture of a battery - backed memory. The aperture has a flush order 312, which is the order in which data is copied 908 from the aperture to non - volatile memory associated with the aperture in response to a data integrity threat 314. The flush order defines the first - flushed end and the last - flushed end of the aperture. A - data includes data designated as critical data.

[0247] The method may also include identifying 1004 a portion A - memory of the unallocated memory of the aperture, where A - memory is large enough to hold A - data, and the address of A - memory is closer to the first - flushed end of the aperture than any other address of any other unallocated memory of the aperture. The method may also include marking 1006 A - memory as allocated and placing 1008 a copy of A - data in A - memory.

[0248] The method may also include receiving 1104 a request to store dataset Z - data in the aperture, where Z - data does not include any data designated as critical data.

[0249] The method may also include identifying 1004 a portion Z - memory of the unallocated memory of the aperture, where Z - memory is large enough to hold Z - data, and the address of Z - memory is closer to the last - flushed end of the aperture than any other address of any other unallocated memory of the aperture. The method may also include marking 1006 Z - memory as allocated and placing 1008 a copy of Z - data in Z - memory.

[0250] By such a method, by placing critical data before non - critical data in the battery - backed memory in the flush order, compared to not placing critical data before non - critical data in the battery - backed memory in the flush order, the likelihood 1138 of successfully flushing 1012 the critical data and thus preserving 1102 the critical data is increased.

[0251] In some embodiments, the method may further include determining 1106 the size 1108 of the aperture based at least in part on battery characteristics of the battery-backed memory. For example, a battery with a smaller capacity, an older battery, and a less reliable battery may result in a smaller aperture. In some embodiments, the method may further include determining 1106 the location 1110 of the aperture based at least in part on battery characteristics of the battery-backed memory. For example, an aperture in a memory backed by a newer, larger capacity, or more reliable battery may be selected rather than an aperture in a memory backed by a smaller, older, or less reliable battery. In some embodiments, the method may include adjusting 1114 the aperture size or moving 1115 the aperture based at least in part on a change in battery characteristics of the battery-backed memory, e.g., by increasing the aperture size when replacing the battery with a newer, larger, or more reliable battery. As with other steps disclosed herein, these steps may be combined in a given embodiment.

[0252] In some embodiments, the method may further include defragmenting 1120 at least one defragmentable portion of the aperture that includes critical data. As discussed elsewhere herein, defragmenting 1120 refers to flush defragmentation; consolidation defragmentation may be a side effect but is not sufficient in nature to act as flush defragmentation 1120. Defragmenting 1120 may be proactive. Defragmenting 1120 may also be triggered when a tenant or process terminates, vacates, or relocates.

[0253] In some embodiments, the method may further include designating at least a portion of the A-data as critical data based on at least one of the following criteria 602: workload criticality, virtual machine priority, or customer status.

[0254] In some embodiments, the method may include detecting 1010 a data integrity threat, flushing 1012 all critical data from the aperture to non-volatile memory, and failing 1122 to flush at least some non-critical data from the aperture to non-volatile memory. For example, the power stored in the battery may not be sufficient to flush the entire aperture but may still be sufficient to flush all critical data. This example scenario illustrates the advantage of placing critical data such that, in the case of battery power, critical data is flushed before non-critical data.

[0255] In some embodiments, the method may include detecting 1010 a data integrity threat, flushing 1012 at least all critical data from the aperture to non-volatile memory, and then restoring 910 all flushed data from the non-volatile memory. The flushed critical data may be restored 910 to at least one of the following: volatile memory, the battery-backed memory from which the data was flushed, or a battery-backed memory different from the battery-backed memory from which the data was flushed.

[0256] In some embodiments, the method may include keeping at least one copy of the Z-data outside the aperture and outside the non-volatile storage device associated with the aperture. That is, when building a system for data preservation, the replication of non-critical data is compatible with the careful placement 1008 and flushing 1012 of critical data.

[0257] In some embodiments, a battery-backed memory aperture resides 1124 in a physical machine herein represented as M, and a request to store A-data in that aperture is sent 1112 from a different physical machine herein represented as N. For example, in Figure 2 or Figure 8 the configuration shown, the compute code and the storage node can be different physical machines. In an alternative configuration, they can be virtual devices running on the same underlying physical hardware.

[0258] In some embodiments, the method may include storing only the A-data in one storage node and storing copies of the Z-data in multiple storage nodes. That is, in some examples, critical data preservation relies only on placement 1008 and flushing 1012, while non-critical data preservation relies at least in part on replication 226.

[0259] Configured storage medium

[0260] Some embodiments include a configured computer-readable storage medium. The storage medium 112 may include a disk (magnetic, optical, or other), RAM, EEPROM, or other ROM, and / or other configurable memory, specifically including a computer-readable storage medium (which is more than just a propagated signal). The configured storage medium can in particular be a removable storage medium 114 such as a CD, DVD, or flash memory. A general-purpose memory, which can be removable or non-removable and can be volatile or non-volatile, can be configured to form a configured storage medium in the form of data 118 and instructions 116 read from a removable storage medium 114 and / or another source such as a network connection using items such as an allocator 422, a defragmenter 408, and a mapper 814. As disclosed herein, the configured storage medium 112 enables a computer system 102 to perform technical processing steps for data preservation. Thus, these figures help illustrate configured storage medium embodiments and process (also called, method) embodiments, as well as system and process embodiments. Specifically, Figure 9 , 10 any process steps shown in or otherwise taught herein can be used to help configure the storage medium to form a configured storage medium embodiment.

[0261] Some embodiments use or provide computer-readable storage media 112, 114 that are configured with data 118 and instructions 116 that, when executed by a processor 110, cause a computing system or a subsystem thereof to perform a method for data preservation. The method includes receiving 1002 a plurality of requests, each request seeking to store a corresponding data set in an aperture 220 of a battery-backed memory, the aperture having a flush order 312 that is an order to copy data from the aperture to a non-volatile storage device associated with the aperture in response to a data integrity threat, the flush order defining a first flush end 806 of the aperture and a last flush end 808 of the aperture, and each corresponding data set including data designated as critical data.

[0262] The method further includes, for at least two of the requests, identifying 1004 corresponding portions of unallocated memory of the aperture, each portion of unallocated memory being large enough to hold a corresponding data set, and the address of the identified corresponding portions of unallocated memory being closer to the first flush end of the aperture than any other address of any other unallocated memory of the aperture.

[0263] The method further includes, for at least one of the requests, marking 1006 the identified corresponding portions of unallocated memory as allocated and placing 1008 a copy of the corresponding data set therein.

[0264] The method further includes detecting 1010 a data integrity threat and flushing 1012 all critical data copied into the aperture to the non-volatile storage device. Thus, the method preserves 1102 all critical data copied into the aperture despite the data integrity threat.

[0265] In some embodiments, the method further includes specifying 1106 at least one of an aperture size 1108 or an aperture address 1110, and the specifying is based on at least one of the following: a battery capacity 502, a battery life 506, or a battery reliability value 504.

[0266] In some embodiments, flushing 1012 includes executing 1142 firmware 310 having a Unified Extensible Firmware Interface.

[0267] In some embodiments, the method further includes reserving 1116 at least ten percent 1118 of the aperture to hold data not designated as critical data. For example, reserving twenty percent or reserving thirty percent would each comply with reserving 1116 at least ten percent. In some embodiments, the portion 1118 reserved 1116 is at most ten percent, or at most twenty percent. Reserving 1116 can also be considered or implemented as partitioning the aperture or memory into sub-partitions.

[0268] In some embodiments, the method further includes moving 1128 data that is not designated as critical data to a storage location outside the aperture to make room for data that is designated as critical data. In some cases, the method includes defragmenting 1120 at least a portion of the aperture, thereby reducing 1134 or eliminating 1136 unallocated portions that were previously located between two allocated portions of the aperture.

[0269] Additional examples and observations

[0270] Those skilled in the art will recognize that not every part of the present disclosure or any particular detail thereof is necessarily required to meet legal standards such as enablement, written description, or best mode. Moreover, embodiments are not limited to the specific networks, protocols, tools, identifiers, fields, data structures, functions, secrets, or other proofs, or other implementation choices described herein. Any apparent conflict with any other patent disclosure, even from the owner of the present invention, will not play any role in interpreting the claims presented in this patent disclosure. Based on an understanding related to all parts of the present disclosure, some additional examples and observations are provided.

[0271] In some embodiments, the volatile memory 402 is backed up by a battery 216 and can be quickly flushed 1012 to create a copy of the data 118 in the non-volatile storage device 902 when the integrity of the data is threatened 314 (e.g., due to an impending power outage or reboot). The flush 1012 is subject to a flush order 312, so some data may not be flushed to a secure location. For example, the flush may be incomplete due to battery failure or depletion. Flushing is an example of a data lifecycle operation; some other examples are allocation, use, copy, release, move, defragment, and recovery.

[0272] The design and implementation of some embodiments have the goal of providing a battery that is large enough. Thus, the aperture size can depend on the amount of data 118 that the battery can protect as determined by the architect or developer. Thus, a larger battery, a newer battery, a more reliable battery can result in a larger aperture, and conversely, a smaller / older / more unreliable battery or unknown battery characteristics can result in a smaller aperture. Additionally, the battery may degrade over time, so the ability to save large amounts of data may be affected over time. The size of the aperture 220 can be adjusted accordingly.

[0273] Some embodiments implement a hybrid placement order, whereby critical data is grouped at or near the first flush end 806 of the aperture and non-critical data is grouped at or near the last flush end 808 of the aperture. Non-critical data or critical data (or both) can also be copied to other apertures or other memories outside of any battery-supported aperture.

[0274] In some cases, the presence of the data preservation techniques taught herein can be inferred from a document, from a memory map 814, from a memory dump combined with information about data criticality, from a probe of the machine or communication between them, from Joint Test Action Group (JTAG) debugging, from NVDIMM-SW save / restore testing, from a firmware or other code listing, from other evidence, or from a combination thereof.

[0275] Some embodiments provide or use an NVDIMM-SW save algorithm that can take the criticality 612 of a workload as a way to rank and sort NVDIMM-SW data sets. Some implementations describe a Storage Class Memory (SCM) partition 904 by a virtual machine ACPI address range, and mappings 222, 224 between virtual machines and to the SCM partition. This will direct NVDIMM-SW to save routing in the runtime components of the system firmware (e.g., UEFI / SMM for Intel / AMD CPUs and UEFI / TrustZone for ARM CPUs). Some embodiments protect 908 the content 118 in any graceful or ungraceful shutdown, allowing the virtual machine content 118 to be restored 910 on the next boot. In some cases, NVDIMM-SW helps ensure fast access to data while helping ensure that the data is flushed to non-volatile storage even in the event of a service interruption such as a node going offline.

[0276] In some embodiments, for example, events that can trigger a flush can include an orderly shutdown, a hard reboot, or a soft reboot.

[0277] An orderly shutdown gives the running software the opportunity to prevent data corruption by saving the current versions of directories, allocation tables, and other structures that describe the organization of data, as well as saving the data itself. For example, the shutdown command commands applications and other processes to close, which in turn gives these processes the opportunity to flush data to non-volatile memory, close any open files, and release the allocated memory back to the OS. The shutdown command also commands device drivers to flush IO data and current directory information to attached devices. On an ACPI-compliant system, the shutdown command can cause a power command to be issued, which causes NVDIMM to save 908 data from its volatile portion to its non-volatile portion.

[0278] A hard reboot starts with no power applied, except possibly the power button. Power is given to the system. The boot memory code (BIOS / EFI / UEFI) performs a POST. Then, the boot memory code loads the boot loader from the boot device into RAM. The boot loader loads the OS into RAM from non-volatile working memory or over a network connection. Anything in RAM at the start of a hard reboot may be overwritten.

[0279] A soft reset starts when the system is powered on. The power-on self-test (POST) is skipped. The boot memory code loads the boot loader from the boot device into RAM. The boot loader loads the OS into RAM from non-volatile working memory or over a network connection. Anything in RAM at the start of a soft reset may be overwritten.

[0280] In some implementations, the OS uses a system memory map to identify which memory regions 904 are reserved for UEFI runtime services after the OS boots and which regions are reclaimed for OS use. In some cases, the memory map is developed during all phases of UEFI pre-boot. It changes as memory is allocated for storing code and data. The type of memory allocation varies based on the type of code and data, such as boot service code, boot service data, runtime code, runtime data, ACPI NVS memory, SMM, etc.

[0281] In some embodiments, there is NVDIMM-SW, and a firmware-assisted component is provided, which simulates a non-volatile DIMM through the coupling of a DIMM and a block storage device (such as NVMe or SATA). The goal is to adopt a standard driver approach to seamlessly expose the NVDIMM-SW device to the operating system. In the standard operating mode, the pre-boot firmware exposes the available NVDIMM-SW devices to the operating system. All write operations from applications that utilize NVDIMM-SW are directed to the DIMM regions associated with NVDIMM-SW. SAVE# is initiated in both normal and abnormal shutdown scenarios. The SAVE# operation is completed before system shutdown and subsequent reboot. The RESTORE# operation is initiated by the pre-boot firmware before control is returned to the operating system. NVDIMM-SW can be constructed by coupling a DDR4 DIMM and an on-board M.2 NVMe module. The NVDIMM-SW paradigm repurposes traditional DIMMs to simulate byte-addressable memory. In some implementations, all DIMMs in the system are populated with the same size and memory type. NVDIMM-SW can support two operating modes, namely NVDIMM non-interleaved or NVDIMM interleaved. For NVDIMM non-interleaved, the DIMM on slot 1 will be selected as the non-volatile NVDIMM (SW) NVDIMM, depending on the non-volatile memory size selected via the setup option. The maximum non-volatile memory size depends on the power duration during an abnormal save scenario. Non-interleaved NVDIMM-SW is substantially the same as NVDIMM-N except for simulating all I2C (serial protocol)_DSM (device-specific method) data. In NVDIMM interleaved, all DIMMs within a slot are interleaved to support NUMA. Additionally, based on the selected non-volatile memory size, the top memory of each slot is carved out from the system memory map to serve as NVDIMM-SW. The goal of event handling is to attempt to save data from volatile memory to the non-volatile memory region. The firmware stack (UEFI, BMC) is also responsible for logging errors for both in-band and out-of-band listeners.

[0282] Some additional combinations and variations

[0283] Any one of these combinations of code, data structures, logic, components, communications, and / or their functional equivalents can also be combined with any of the above systems and their variations. The process can include any of the steps described herein in any operable subset or combination or sequence. Each variation can occur alone or in combination with any one or more other variations. Each variation can occur with any process, and each process can be combined with any one or more other processes. Each process or process combination including variations can be combined with any of the combinations and variations of the above-configured storage medium.

[0284] Conclusion

[0285] In short, the teachings provided herein can be applied to enhance data retention capabilities in computing systems. In some embodiments, the combined operating steps and device characteristics help protect data 118 from integrity threats 314. Based on criteria 602 such as customer 606 requirements 608, workload 610 criticality 612, or virtual machine 206 criticality 604, data 118 is divided into critical data 412 and non-critical data 416. For example, data 118 can be generated in computing node 202 and sent 222 for storage in storage node 212. Critical data 412 is stored at physical address 424 within the aperture 220 of battery-backed memory 218, where due to the flush order 312 imposed by the circuitry of the battery-backed memory or by an external force (e.g., system firmware 420) on the battery-backed memory, critical data 412 will be flushed 1012 before non-critical data 416. The flush order 312 can be, for example, a bottom-up NVDIMM 404 flush order. Redundant copies 226 of data 118 (especially non-critical data 416) can also be retained 1126 in case the replicated data is not flushed 1012 in time to save 1102 it. The battery-backed memory apertures 220 are sized 1106 and positioned 1106 according to their battery characteristics 406, and can be repositioned 1115 or resized 1114 as conditions change. A flush defragmentation 1120 is performed to optimize the use of the aperture 220, especially within the portion 414 of the aperture that stores critical data 412.

[0286] Embodiments are understood to also include or benefit from tested and appropriate security controls and privacy controls, such as, for example, the General Data Protection Regulation (GDPR). It should be understood that appropriate measures should be taken to help prevent the abuse of computing systems through the injection or activation of malware and to help avoid the tampering of any personal or private information that the enhanced system may process during program execution. The use of the tools and techniques taught here is compatible with the use of such controls.

[0287] Although specific embodiments are explicitly shown and described herein as processes, configured storage media, or systems, it can be understood that the discussion of one type of embodiment generally extends to other types of embodiments as well. For example, the description of a process in conjunction with Figure 9 、 10 and 11 also helps describe configured storage media and helps describe the technical effects and operations of systems and manufacturers, as those discussed in conjunction with other figures. This does not mean that the limitations from one embodiment must be read into another embodiment. Specifically, a process is not necessarily limited to the data structures and arrangements presented when discussing systems or manufacturers (e.g., configured memory).

[0288] Those skilled in the art will understand that implementation details may be related to specific code, such as specific APIs, specific fields, specific types of components (hardware or software), and specific sample programs, and thus do not need to appear in every embodiment. Those skilled in the art will also understand that the program identifiers and some other terms used when discussing details are implementation-specific and thus do not need to be associated with every embodiment. However, although they do not necessarily have to appear here, these details can help some readers by providing context and / or by illustrating several of the many possible implementations of the techniques discussed here.

[0289] By paying due attention to the items provided here, including technical processes, technical effects, technical mechanisms, and technical details, which are illustrative but not comprehensive of the embodiments of all claims or claimable claims, those skilled in the art will understand that the present disclosure and the embodiments described herein are not directed to a subject matter outside the technical field, or to any idea as such of a primary or original cause or motivation, or to a result per se, or to a thought process or thought step, or to a business method or a prevalent economic practice, or to a method or to something or a process occurring in nature, or to a living being or a part of a living being, or to a mathematical formula per se, or to an isolated software per se, or to a conventional computer only, or to any completely imperceptible or any abstract concept per se, or to a trivial post-solution activity, or to any method implemented entirely on an unspecified device, or to any method that fails to produce a useful and concrete result, or to a preemption of all fields of use, or to any other subject matter that does not qualify for patent protection under the laws of the jurisdiction seeking or being licensed or implementing patent protection.

[0290] Citing an embodiment having a certain feature X here and citing an embodiment having a certain feature Y elsewhere in this document does not exclude an embodiment having both feature X and feature Y, unless such an exclusion is expressly stated here. All possible negative claim limitations are within the scope of the present disclosure, that is, any feature declared to be part of an embodiment can also be expressly removed from another embodiment, even if that specific exclusion is not given in any example here. Here, the term "embodiment" is only used as a more convenient form of "process, system, article, configured computer-readable storage medium, and / or other examples of the teachings herein applied in a manner compliant with applicable law". Thus, a given "embodiment" can include any combination of the features disclosed here, as long as the embodiment is consistent with at least one claim.

[0291] Not every item shown in the figures needs to be present in every embodiment. Instead, an embodiment can include (a) item(s) not explicitly shown in the figures. Although some possibilities are illustrated in the text and drawings here by specific examples, an embodiment can deviate from these examples. For instance, a particular technical effect or technical feature of an example can be omitted, renamed, differently grouped, repeated, instantiated in hardware and / or software in a different manner, or be a mixture of effects or features that appear in two or more examples. In some embodiments, a function shown in one location can also be provided in a different location; those skilled in the art recognize that functional modules can be defined in various ways in a given implementation without necessarily omitting a desired technical effect from the set of interacting modules taken as a whole. Due to space limitations or for convenience, different steps can be shown together in a single box in the figures, but can still be executed separately. For example, in a given execution of a method, one step can be executed without another step.

[0292] The drawings are referenced throughout by reference numerals. Any apparent inconsistencies in the wording associated with a given reference numeral in the drawings or text should be understood as simply broadening the scope of what the reference numeral refers to. Even when using the same reference numeral, different instances of a given reference numeral can refer to different embodiments. Similarly, a given reference numeral can be used to refer to a verb, a noun, and / or corresponding instances of each. For example, processor 110 can process 110 instructions by executing instructions.

[0293] As used herein, terms such as "a," "an," and "the" include one or more of the indicated item or step. Specifically, in the claims, a reference to an item generally means there is at least one such item, and a reference to a step means at least one instance of performing the step. Similarly, where context permits, the singular verb forms "is" and others should be understood to include the possibility of the plural forms "are" and others to avoid grammatical errors or misunderstandings.

[0294] The headings are for convenience only; information about a given topic can be found outside of the section where the heading indicates that topic.

[0295] All claims and the submitted abstract are part of the specification.

[0296] To the extent that any term used herein implies or otherwise refers to an industry standard, and to the extent that applicable law requires identification of a particular version of such a standard, the present disclosure should be understood to refer to the latest version of that standard, at least in draft form (if newer, then in final form), published prior to the earliest priority date of the present disclosure under applicable patent law.

[0297] Although the exemplary embodiments have been shown in the drawings and described above, it will be apparent to those of ordinary skill in the art that many modifications can be made without departing from the principles and concepts set forth in the claims, and such modifications need not encompass the entire abstract concept. Although the subject matter is described in language specific to structural features and / or process acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific technical features or acts described in the foregoing claims. It is not necessary to present or use every component or aspect or technical effect identified in a given definition or example in every embodiment. Rather, the specific features and acts and effects described are disclosed as examples to be considered when implementing the claims.

[0298] It cannot cover the entire abstract concept, but all changes within the meaning and equivalents of the claims should be incorporated within the full scope permitted by law.

Claims

1. A data storage subsystem in a computing system, the data storage subsystem comprising: a battery-backed memory having an aperture, the memory comprising digital data storage locations addressed in a physical address space, the aperture comprising at least a portion of the digital data storage locations, the aperture having a physical address flush order, the flush order being an order in which data is copied from the aperture to a non-volatile storage device associated with the aperture in response to a data integrity threat, the physical address flush order defining a first-flushed physical address end of the aperture and a last-flushed physical address end of the aperture; a data storage circuit in operable communication with the battery-backed memory, the data storage circuit being configured to perform a data storage step, the data storage step comprising (a) receiving a request to store a data set A-data in the aperture, the A-data comprising data designated as critical data, (b) identifying a portion of A-memory of unallocated memory of the aperture, the A-memory being large enough to hold the A-data, the A-memory having a physical address closer to a first flush physical address end of the aperture than any other physical address of any other unallocated memory of the aperture, and (c) storing the A-memory (d) receiving a request to store a data set Z-data in the aperture, the Z-data not including any data designated as critical data, (e) identifying a portion of Z-memory of unallocated memory of the aperture, the Z-memory being large enough to hold the Z-data, the physical address of the Z-memory being closer to the last flushed end of the aperture than any other physical address of any other unallocated memory of the aperture, and (f) marking the Z-memory as allocated and placing a copy of the Z-data in the Z-memory; The data retention subsystem provides a higher likelihood of successfully flushing critical data and thereby preserving it than by storing data in the battery backed memory without regard to data criticality.

2. The data retention subsystem of claim 1 , wherein the battery-backed memory aperture resides in or is controlled by a network node X, and the request to store A-data in the aperture is sent from a different network node Y.

3. The data-holding subsystem of claim 2, wherein the network node X comprises a storage node in a cloud, and the network node Y comprises a computing node in the cloud.

4. The data storage subsystem of claim 2, wherein the network node X and the network node Y are associated by a storage relationship R, wherein R is defined such that two nodes M and N are associated by R when M stores data on behalf of N, and wherein R is not a one-to-one relationship between the network node X and the network node Y.

5. The data-save subsystem of claim 1, wherein the battery-backed memory comprises NVDIMM memory.

6. The data-saving subsystem of claim 1, wherein the computing system includes a virtual machine, the data set A-data is sent from the virtual machine VM-A to the data-saving subsystem, and the data set Z-data is sent from the virtual machine VM-Z to the data-saving subsystem.

7. A method for data storage in a computing system, the method comprising: receiving a request to store a data set A-data in an aperture of a battery-backed memory, the memory comprising digital data storage locations addressed in a physical address space, the aperture comprising at least a portion of the digital data storage locations, the aperture having a physical address flush order, the flush order being an order in which data is copied from the aperture to a non-volatile storage device associated with the aperture in response to a data integrity threat, the physical address flush order defining a first-flush physical address end of the aperture and a last-flush physical address end of the aperture, the A-data comprising data designated as critical data; identifying a portion of A-memory of unallocated memory of the aperture, the A-memory being large enough to hold the A-data, the physical address of the A-memory being closer to a first flush physical address end of the aperture than any other physical address of any other unallocated memory of the aperture; marking the A-memory as allocated and placing a copy of the A-data in the A-memory; receiving a request to store a data set Z-data in the aperture, the Z-data excluding any data designated as critical data; identifying a portion of a Z-memory of unallocated memory of the aperture, the Z-memory being large enough to hold the Z-data, the physical address of the Z-memory being closer to a last flush physical address end of the aperture than any other physical address of any other unallocated memory of the aperture; as well as marking the Z-memory as allocated and placing a copy of the Z-data in the Z-memory; By thereby placing the critical data before non-critical data in the flushing order in the battery backed memory, the likelihood of successfully flushing the critical data and thereby preserving it is increased.

8. The method according to claim 7, further comprising at least one of the following: determining a size of the aperture based at least in part on battery characteristics of the battery backed memory; determining a position of the aperture based at least in part on battery characteristics of the battery backed memory; adjusting a size of the aperture based at least in part on a change in a battery characteristic of the battery backed memory; or The aperture is moved based at least in part on a change in a battery characteristic of the battery backed memory.

9. The method of claim 7, further comprising flush defragmenting at least one defragmented portion of the aperture, the defragmented portion comprising critical data.

10. The method of claim 7, further comprising designating at least a portion of the A-data as critical data based on at least one of the following criteria: workload criticality, virtual machine priority, or customer status.

11. The method of claim 7, further comprising detecting the data integrity threat, flushing all of the critical data from the aperture to the non-volatile storage device, and failing to flush at least some non-critical data from the aperture to the non-volatile storage device.

12. The method of claim 7 further comprising detecting the data integrity threat, flushing at least all of the critical data from the aperture to the non-volatile storage device, and then restoring all flushed data from the non-volatile storage device to at least one of: volatile memory, battery-backed memory from which the data was flushed, or battery-backed memory different from the battery-backed memory from which the data was flushed.

13. The method of claim 7, further comprising retaining at least one copy of the Z-data outside of the aperture and outside of the non-volatile storage associated with the aperture.

14. The method of claim 7, wherein the battery-backed memory aperture resides in a physical machine, denoted herein as M, and a request to store A-data in the aperture is sent from a different physical machine, denoted herein as N.

15. The method of claim 7, comprising storing the A-data in only one storage node, and storing copies of the Z-data in a plurality of storage nodes.

16. A computer-readable storage medium configured with data and instructions, the instructions, when executed, causing a data-saving subsystem to perform a method for data saving in a computing system, the method comprising: receiving a plurality of requests, each request seeking to store a respective data set in an aperture of a battery-backed memory, the memory comprising digital data storage locations addressed in a physical address space, the aperture comprising at least a portion of the digital data storage locations, the aperture having a physical address flush order, the flush order being an order in which data is copied from the aperture to a non-volatile storage device associated with the aperture in response to a data integrity threat, the physical address flush order defining a first-flush physical address end of the aperture and a last-flush physical address end of the aperture, each respective data set comprising data designated as critical data; for at least two of the requests, identifying respective portions of unallocated memory of the aperture, each portion of unallocated memory being large enough to hold the respective data set, a physical address of the identified respective portion of unallocated memory being closer to a first flush physical address end of the aperture than any other physical address of any other unallocated memory of the aperture; for at least one of the requests, marking the corresponding portion of the identified unallocated memory as allocated and placing a copy of the corresponding data set therein; detecting said data integrity threat; flushing all critical data copied to the aperture from the aperture to the non-volatile storage device; The method thereby preserves all of the critical data copied into the aperture despite the data integrity threat.

17. The computer-readable storage medium of claim 16, wherein the method further comprises specifying at least one of an aperture size or an aperture address, and wherein the specifying is based on at least one of: a battery capacity, a battery life, or a battery reliability value.

18. The computer-readable storage medium of claim 16, wherein the flushing comprises executing firmware having a unified extensible firmware interface.

19. The computer-readable storage medium of claim 16, wherein the method further comprises reserving at least ten percent of the aperture to hold data that is not designated as critical data.

20. The computer-readable storage medium of claim 16, wherein the method further comprises at least one of the following: moving data not designated as critical data to a storage location outside the aperture to make room for data designated as critical data; At least a portion of the aperture is defragmented to reduce or eliminate an unallocated portion previously located between two allocated portions of the aperture.

Images