A method, system and computer program product for unified and automated installation, deployment, configuration and management of software-defined storage assets

By using a unified management and orchestration plane, the installation and configuration of multiple storage stacks are automated, solving the problems of storage stack integration difficulties and inconsistent resource utilization in existing technologies, and achieving efficient management and low-cost operation across storage types.

CN113678100BActive Publication Date: 2024-11-15EMC IP HLDG CO LLC
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202080026899.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-01-31
Filing Date
2020-01-20
Publication Date
2024-11-15
Estimated Expiration
2040-01-20

AI Technical Summary

Technical Problem

Existing technologies struggle to integrate various software-defined storage stacks, making system automation difficult, resource utilization inconsistent, installation and configuration complex, and failing to meet diverse storage type requirements, resulting in high costs and insufficient flexibility.

Method used

It provides a unified management and orchestration plane that automates the installation, configuration, and management of multiple storage stacks by identifying storage type requirements, and leverages cloud-native ecosystems such as Kubernetes and Pivotal Cloud Foundry to achieve unified management and deployment across storage types.

Benefits of technology

It enables automated installation, configuration, and management of various storage types, reducing the workload of IT administrators, improving the utilization efficiency of storage resources and the flexibility of the system, and reducing operating costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113678100B_ABST
    Figure CN113678100B_ABST
Patent Text Reader

Abstract

An access consumption request, the consumption request including a stack parameter and a resource characteristic parameter. The stack parameter specifies at least one type of storage asset requested. The resource characteristic parameter specifies at least one functional capability required for the storage asset. A set of one or more first storage assets that can satisfy the consumption request is determined based on the stack parameter. For each undeployed first storage asset in the set, a first workflow is generated, the first workflow being configured to deploy the corresponding undeployed first storage asset in the set. For each second storage asset in the set that lacks the resource characteristic parameter, a second workflow is generated, the second workflow being configured to implement the missing resource characteristics in the corresponding second storage asset. By running the first workflow and the second workflow, the set storage assets are configured to satisfy the consumption request.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates at least generally to apparatus, systems and methods for data storage and data processing in computer systems, including systems for managing data and resources in a virtualized environment. More particularly, the present application relates at least to methods for improving the configuration, deployment and installation of software-defined data storage. Background Art

[0002] As the value and use of information continue to increase, individuals and businesses seek more ways to process and store information. One option available to users is an information processing system. An information processing system typically processes, compiles, stores and / or communicates information or data for commercial, personal or other purposes, thereby allowing users to take advantage of the value of this information. Because the needs and requirements of technology and information processing vary between different users or applications, information processing systems may also differ in terms of what information to process, how to process information, how much information to process, store or communicate, and how quickly and efficiently information can be processed, stored or communicated. The variation of information processing systems allows information processing systems to be general or configured for specific users or specific purposes (such as financial transaction processing, airline reservations, enterprise data storage or global communications). In addition, an information processing system may include various hardware and software components that can be configured to process, store and communicate information, and may include one or more computer systems, data storage systems and network systems.

[0003] In a data storage system, users of different storage technologies store large amounts of data on different storage devices. Users can select any combination of storage devices from a storage array of storage devices to create a virtual storage resource or logical unit. Computer systems are constantly improving in terms of speed, reliability, and processing power. As known in the art, a computer system that processes and stores large amounts of data typically includes one or more processors that communicate with a shared data storage system in which the data is stored. A data storage system may include one or more storage devices that typically have fairly robust properties and can be used for storage across various temporary needs, such as disk drives. One or more processors use the storage system to perform their corresponding operations. A mass storage system (MSS) typically includes an array of multiple disks with onboard intelligence and communication electronics and software for making the data on the disks available.

[0004] In addition, enterprises process more data than ever before, and traditional infrastructure no longer provides the reliability, performance, and efficiency required for enterprise workloads (e.g., enterprise information processing systems). This reality is challenging information technology (IT) managers to adopt simpler, more streamlined, and more cost-effective approaches, such as software-defined infrastructure (SDI). SDI enables enterprises to enjoy the power and efficiency of IT as a service by separating hardware and software and focusing on extensive automation and orchestration. This allows IT to provide enterprises with scalable, flexible resources for cloud-based and web-based services and native cloud workloads.

[0005] Moreover, information handling systems increasingly utilize reconfigurable virtual resources to meet changing user needs in an efficient, flexible and cost-effective manner. For example, cloud computing and storage systems implemented using virtual resources have been widely adopted. Other virtual resources currently widely used in information handling systems include Linux containers. Such containers can be used to provide at least a portion of the virtualization infrastructure of a given information handling system. Summary of the invention

[0006] The present summary is provided to introduce a selection of concepts in a simplified form to provide a basic understanding of one or more embodiment embodiments, which will be further described in the detailed description below. The present summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0007] In certain embodiments, a computer-implemented method for deploying storage assets is provided. A consumption request for accessing a storage asset is provided, the consumption request comprising a stack parameter and a resource characteristic parameter, wherein the stack parameter specifies at least one type of storage asset requested, and the resource characteristic parameter specifies at least one functional capability required of the storage asset. A set of one or more first storage assets that can satisfy the consumption request is determined based on the stack parameter. For each first storage asset in the set that is not deployed, a first workflow is generated, the first workflow being configured to deploy the corresponding first storage asset in the set that is not deployed. For each second storage asset in the set that lacks a resource characteristic parameter, a second workflow is generated, the second workflow being configured to implement the missing resource characteristic in the corresponding second storage asset. The set of storage assets is configured to satisfy the consumption request, wherein the configuration comprises running the first workflow and the second workflow.

[0008] In some embodiments of the computer-implemented method, the computer-implemented method further comprises: determining that a third storage asset resource can be created to satisfy the consumption request; and generating a third workflow configured to create and deploy the third storage asset resource. In some embodiments, the third storage asset resource is formed within at least one of the first storage asset and the second storage asset.

[0009] In certain embodiments of the computer-implemented method, the consumption request is used to provision a file share, and the computer-implemented method further comprises: contacting one or more LINUX hosts to determine and store host information; registering the one or more LINUX hosts; requesting that a storage asset of a predetermined type be deployed as a corresponding service; and creating a corresponding file share.

[0010] In another embodiment of the computer-implemented method, the consumption request includes a query (including query parameters) related to the retrieval of the inventory of the storage asset, and the computer-implemented method also includes: predefining a set of templates applicable to the storage asset, each corresponding template including the configuration of the corresponding storage asset and the prerequisites for deploying the storage asset; and generating a rule set configured to be executed on the inventory of storage assets, the rule set being configured to determine whether a template is implementable for a given storage asset based on one or more rules in the rule set.

[0011] In certain embodiments of the computer-implemented method, a list of deployment templates applicable to the storage asset is received, the list generated in view of the rule set, wherein the list of deployment templates includes at least one of the following: reasons why a first portion of the list of deployment templates can be deployed; and reasons why a second portion of the list of deployment templates cannot be deployed.

[0012] In certain embodiments of the computer-implemented method, the storage asset comprises a software-defined storage asset. In certain embodiments, at least a portion of the computer-implemented method is performed in a cloud-native environment.

[0013] In another aspect, a system is provided, the system comprising a processor and a non-volatile memory, the non-volatile memory being in operative communication with the processor and storing computer program code, the computer program code, when executed on the processor, causing the processor to execute a process operable to perform certain operations. An operation comprises accessing a consumption request to consume a storage asset, the consumption request comprising a stack parameter and a resource characteristic parameter, wherein the stack parameter specifies at least one type of storage asset requested, and the resource characteristic parameter specifies at least one functional capability required of the storage asset. An operation comprises determining a set of one or more first storage assets capable of satisfying the consumption request based on the stack parameter. An operation comprises generating a first workflow for each first storage asset in the set that is not deployed, the first workflow being configured to deploy the corresponding first storage asset in the set that is not deployed. An operation comprises generating a second workflow for each second storage asset in the set that lacks the resource characteristic parameter, the second workflow being configured to implement the missing resource characteristic in the corresponding second storage asset. An operation comprises configuring the set of storage assets to satisfy the consumption request, wherein the configuring comprises running the first workflow and the second workflow.

[0014] In a further embodiment of the system, the operations performed further include: determining that a third storage asset resource can be created to satisfy the consumption request; generating a third workflow, the third workflow being configured to create and deploy the third storage asset resource. In certain embodiments, the third storage asset resource is formed within at least one of the first storage asset and the second storage asset. In certain embodiments, the consumption request includes a query (including query parameters) related to retrieval of an inventory of the storage asset, and the computer-implemented method further includes computer program code that, when executed on the processor, causes the processor to perform the following actions: pre-define a set of templates applicable to the storage asset, each respective template including a configuration of the respective storage asset and a prerequisite for deploying the storage asset; and generating a rule set configured to be executed on the inventory of the storage asset, the rule set being configured to determine whether a template is implementable for a given storage asset based on one or more rules in the rule set.

[0015] In certain embodiments of the system, the operations include receiving a list of deployment templates applicable to the storage asset, the list generated in view of the rule set, wherein the list of deployment templates includes at least one of the following: reasons why a first portion of the list of deployment templates can be deployed; and reasons why a second portion of the list of deployment templates cannot be deployed.

[0016] In certain embodiments of the system, the storage asset comprises a software defined storage asset.In certain embodiments of the system, at least a portion of the computer program code is executed in a cloud native environment.

[0017] In yet another aspect, a computer program product is provided, the computer program product comprising a non-transitory computer-readable storage medium having computer program code encoded thereon, the computer program code, when executed on a processor of a computer, causing the computer to operate a storage system. In certain embodiments, the computer program product comprises computer program code for accessing a consumption request for consuming a storage asset, the consumption request comprising a stack parameter and a resource characteristic parameter, wherein the stack parameter specifies at least one type of storage asset requested, and the resource characteristic parameter specifies at least one functional capability required of the storage asset. In certain embodiments, the computer program product comprises computer program code for determining a set of one or more first storage assets capable of satisfying the consumption request based on the stack parameter.

[0018] In some embodiments, the computer program product includes computer program code for generating a first workflow for each first storage asset in the collection that is not deployed, the first workflow being configured to deploy the corresponding first storage asset in the collection that is not deployed. In some embodiments, the computer program product includes computer program code for generating a second workflow for each second storage asset in the collection that lacks the resource characteristic parameter, the second workflow being configured to implement the missing resource characteristic in the corresponding second storage asset. In some embodiments, the computer program product includes computer program code for configuring the collection of storage assets to satisfy the consumption request, wherein the configuring includes running the first workflow and the second workflow.

[0019] In further embodiments, the computer program product includes computer program code for determining that a third storage asset resource may be created to satisfy the consumption request, and computer program code for generating a third workflow configured to create and deploy the third storage asset resource.

[0020] In additional embodiments, the consumption request includes a query (including query parameters) related to the retrieval of an inventory of the storage asset, and the computer program product also includes computer program code for predefining a set of templates applicable to the storage asset, each respective template including a configuration of the respective storage asset and prerequisites for deploying the storage asset, and computer program code for generating a rule set configured to be executed on the inventory of storage assets, the rule set being configured to determine whether a template is implementable for a given storage asset based on one or more rules in the rule set.

[0021] In some embodiments, the computer program product includes computer program code for receiving a list of deployment templates applicable to the storage asset, the list generated in view of the rule set, wherein the list of deployment templates includes at least one of the following: reasons why a first portion of the list of deployment templates can be deployed; and reasons why a second portion of the list of deployment templates cannot be deployed. In some embodiments, the storage asset includes a software-defined storage asset. Details related to these and other embodiments will be described more fully herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The objectives, aspects, features and advantages of the embodiments disclosed herein will be more fully understood from the following detailed description, the accompanying claims and the accompanying drawings, in which similar reference numerals represent similar or identical elements. Reference numerals introduced in the specification in conjunction with the drawings may be repeated in one or more subsequent figures without the need for additional description in the specification to provide context for other features. For clarity, not every element may be marked in every figure. The drawings are not necessarily to scale, but rather focus on illustrating the embodiments, principles and concepts. The drawings are not meant to limit the scope of the claims included herein.

[0023] Figure 1 is an illustrative functional diagram of a management and operations (M&O) plane, the M&O plane encapsulating an exemplary software defined storage platform at a high level, according to at least one illustrative embodiment of the present disclosure;

[0024] Figure 2 is an illustrative diagram showing how multiple data services exist together on a server in an exemplary system according to an illustrative embodiment of the present disclosure, the data services including applications run by clients;

[0025] Figure 3 is a simplified diagram of an architecture for implementing a software defined storage suite according to an illustrative embodiment of the present disclosure;

[0026] Figure 4 According to an illustrative embodiment Figure 3 A more detailed diagram of the architecture shown;

[0027] Figure 5 FIG. 1 is a diagram showing an exemplary embodiment of the present disclosure. Figure 4 A detailed diagram of the architecture of the controller service diagram is shown;

[0028] Figure 6 is an illustrative user interface configured to work with the various architectures described herein in accordance with at least one disclosed embodiment;

[0029] Figure 7 According to an embodiment, Figure 5 A first illustrative diagram of components within a stack deployment orchestration controller;

[0030] Figure 8 In accordance with at least one embodiment, Figure 5 a second illustration of additional components used within the stack deployment orchestration controller;

[0031] Fig. 9 According to one embodiment, Figure 3-Figure 8 A first simplified flowchart of an exemplary method of storage service deployment for use with a system;

[0032] Fig.10 According to one embodiment, Figure 3-Figure 8 A second simplified flow chart of an exemplary method of service storage deployment for use with a system;

[0033] Fig.11 According to at least one embodiment Figure 3-Figure 8 Figure 2 shows a diagram and exemplary sequence of actions that occur in the architecture of a storage service (occurs during storage service deployment), showing when a customer requests a network attached storage (NAS) share until the workflow is assembled and delivered to Figure 3-Figure 8 The actions that occur when a workflow executes a service.

[0034] Fig.12 According to at least one embodiment, Figure 3-Figure 8 An illustration of the automatic vertical functions occurring in the architecture of the architecture;

[0035] Fig.13 FIG. 1 is a diagram showing a method for detecting a Figure 3-Figure 12 A simplified flowchart of a method for registering and deploying hosts and storage of an architecture;

[0036] Fig.14 According to at least one embodiment Fig.12 The architecture and Fig.13An illustrative diagram and exemplary sequence of actions occurring in a simplified flow chart of FIG. 1 showing components that function during host registration and deployment of software-defined shared block storage (based on direct attached storage), such as VxFlex OS and software-defined network attached storage (SDNAS);

[0037] Fig.15 According to at least one embodiment Fig.12 The architecture and Fig.13 An illustrative diagram and exemplary sequence of actions occurring in a simplified flow chart of FIG. 1 , showing components that function during host registration;

[0038] Fig.16 According to at least one embodiment Fig.12 The architecture and Fig.13 An illustrative diagram and exemplary sequence of actions occurring in a simplified flow chart of FIG. 1 , showing components that function during deployment of IO Scale as a Service;

[0039] Fig.17 According to at least one embodiment Fig.12 The architecture and Fig.13 An illustrative diagram and exemplary sequence of actions occurring in a simplified flow chart of FIG. 1 , showing components that come into play during deployment of SDNAS as a Service;

[0040] Fig.18 According to at least one embodiment Fig.12 The architecture and Fig.13 An illustrative diagram and exemplary sequence of actions occurring in a simplified flow chart of FIG. 1 showing components that function during the creation of a network file share (NFS);

[0041] Fig.19 is an illustrative diagram showing components that function during host registration and requesting a deployment template, according to at least one embodiment;

[0042] Fig. 20 According to at least one embodiment, at least Figure 3-Figure 8 An illustrative simplified flow diagram of actions occurring during host registration and requesting deployment of a template in an architecture of

[0043] Fig.21 is an illustrative simplified flow chart according to at least one embodiment, showing a process for responding to at least Figure 3-Figure 8 The architecture provides a method for supplying a request to a store, the request including an indication of which configurations are available in an inventory and the capabilities of the inventory.

[0044] Fig. 22 According to at least one embodiment Fig.21 An illustrative diagram and exemplary sequence of actions for implementing a method of requesting a template;

[0045] Fig.23 According to at least one embodiment Fig.21 An illustrative diagram and exemplary sequence of actions for implementing a method of obtaining an installed storage service;

[0046] Fig.24 According to at least one embodiment Fig.21 An illustrative diagram and exemplary sequence of actions for implementing a method for obtaining real-time installed storage service details;

[0047] Fig.25 According to at least one embodiment Fig.21 An illustrative diagram and exemplary sequence of actions for implementing a method of deploying a storage stack;

[0048] Fig.26 According to at least one embodiment, for at least Figure 2-Figure 19 A simplified flowchart of an exemplary method for identifying a pool of hardware available for consumption by an SDS platform and for elastically expanding / contracting an existing storage stack;

[0049] Fig. 27 According to at least one embodiment Fig.28 A simplified flowchart of a method for consuming a request for user information;

[0050] Fig.28 A simplified flow chart of a method for parsing a consumption request for storage type information according to at least one embodiment; and

[0051] Fig.29 According to at least some embodiments, it is possible to implement Figure 1-Figure 28 A simplified block diagram of an apparatus that implements at least a portion of a system and method. DETAILED DESCRIPTION

[0052] At least some illustrative embodiments will be described herein with reference to exemplary information processing systems and associated computers, servers, storage devices, and other processing devices. However, it will be appreciated that the embodiments described herein are not limited to use with the specific illustrative system and device configurations shown. Additionally, embodiments may include, but are not limited to, devices, systems, methods, and computer program products including processor-readable storage media.

[0053] It is contemplated that at least some of the embodiments described herein are feasible in various types of storage systems, not just in software-defined storage systems. In some embodiments, the current disclosure enables the disclosure of logical storage resources and allows enterprise IT departments and cloud service providers to manage heterogeneous storage environments through a simple, robust Representational State Transfer (REST) ​​API and command line interface (CLI).

[0054] Before describing embodiments of the concepts, structures, and techniques for which protection is sought herein, some terms are explained and some relevant background patents are cited. The following description includes several terms with definitions that are generally known in the art. However, the following glossary definitions are provided to clarify the subsequent description and to aid in understanding the specification and claims.

[0055] As used herein, the term "storage system" is intended to be broadly interpreted to cover, for example, private or public cloud computing systems for storing data, and systems for storing data including and excluding virtual infrastructure. As used herein, the terms "client," "host," and "user" interchangeably refer to any person, system, or other entity that uses a storage system to read / write data. In some embodiments, the term "storage device" may also refer to a storage array including a plurality of storage devices. In certain embodiments, a storage medium may refer to one or more storage media, such as a hard drive, a combination of hard drives, a flash memory, a combination of flash memories, a combination of hard drives, a combination of flash memory and other storage devices, and other types and combinations of computer-readable storage media, including those storage media that have not yet been thought of. A storage medium may also refer to both physical storage media and logical storage media, and may include multiple levels of virtual-to-physical mapping and may be or include an image or disk image. A storage medium may be computer-readable, and may also be referred to herein as a computer-readable program medium.

[0056] In some embodiments, the term "information processing system" may be used to cover processing systems including, for example, cloud computing and storage systems, as well as other types of processing systems including various combinations of physical and virtual processing resources. Thus, an information processing system may include, for example, at least one data center or other type of cloud-based system including one or more clouds hosting tenants accessing cloud resources. Cloud computing is intended to refer to all variants of cloud computing, including but not limited to public, private, and hybrid cloud computing. In addition, information processing resources may broadly refer to any component system, device, or equipment of an information processing system, including but not limited to processors, buses, memories, input-output devices and / or interfaces, storage resources, network interfaces, motherboards, electromechanical devices (e.g., fans), displays, and power supplies.

[0057] In certain embodiments, the term "computer-readable medium" may include any tool or collection of tools that can retain data and / or instructions for a period of time. Computer-readable media may include, but are not limited to: storage media such as direct access storage devices (e.g., hard drives or floppy disks), sequential access storage devices (e.g., magnetic tape disk drives), compact disks, CD-ROMs, DVDs, random access memories ("RAM"), read-only memories ("ROM"), electrically erasable programmable read-only memories ("EEPROM"), and / or flash memories; and communication media such as wires, optical fibers, microwaves, radio waves, and other electromagnetic and / or optical carrier waves; and / or any combination of the foregoing.

[0058] In some embodiments, the term "I / O request" or simply "I / O" may be used to refer to an input or output request, such as a data read or data write request, which may originate from a host, a user, or any other entity in operative communication with a computer system.

[0059] In certain embodiments, the term "storage device" may refer to any non-volatile memory (NVM) device, including hard disk drives (HDDs), solid-state drives (SSDs), flash memory devices (e.g., NAND flash memory devices), and similar devices that can be accessed locally and / or remotely (e.g., via a storage attachment network (SAN) (also referred to herein as a storage array network (SAN))).

[0060] In certain embodiments, a storage array (sometimes referred to as a disk array) may refer to a data storage system for block-based, file-based, or object storage, where the storage array may include, for example, dedicated storage hardware containing spinning hard disk drives (HDDs), solid-state disk drives, and / or all-flash drives (e.g., XtremIO all-flash drives available from DELL / EMC of Hopkinton MA). In certain embodiments, a data storage entity may be any one or more of the following: a file system, an object store, a virtualized device, a logical unit, a logical unit number, a logical volume, a logical device, a physical device, and / or a storage medium.

[0061] In some embodiments, a logical unit (LU) may be a logical entity provided by a storage system for accessing data from the storage system, and as used herein, a logical unit may be used interchangeably with a logical volume. In many embodiments herein, LU and LUN (Logical Unit Number) may be used interchangeably with each other. In some embodiments, a LUN may be a logical unit number for identifying a logical unit; it may also refer to one or more virtual disks or virtual LUNs, which may correspond to one or more virtual machines. A LUN may be divided into smaller logical areas to balance the load between system modules, wherein each such small logical area is referred to as a sub-LUN.

[0062] In some embodiments, a physical storage unit may be a physical entity for storing data in a storage location accessible according to an address, such as a disk or a disk array, wherein a physical storage unit may be used interchangeably with a physical volume. In some embodiments, a data storage entity may be any one or more of the following: a file system, an object storage, a virtualization device, a logical unit, a logical unit number, a logical volume, a logical device, a physical device, and / or a storage medium.

[0063] In some embodiments, a storage controller may refer to any system, device or apparatus operable to manage data communications between a processor and a storage resource of a storage array. In some embodiments, a storage controller may provide functionality including, but not limited to, disk aggregation and redundant array of independent disks (RAID), I / O routing, and error detection and recovery. A storage controller may also have features that support shared storage and high availability. In some embodiments, a storage controller may include a PowerEdge RAID controller (PERC) manufactured by Dell EMC Inc. A storage controller may be located in a housing or housing other than a housing or housing of an information processing system controlled by the storage controller, such as a storage housing that includes one or more physical storage resources of a given storage array.

[0064] In some embodiments, data replication includes the process of copying storage data (e.g., data stored on a data storage entity) to a remote or local system to help provide an enhanced level of redundancy in the event of a failure of a primary or primary storage backup system. In some embodiments, an image can be a copy of a logical storage unit at a specific point in time. In some embodiments, a clone can be a copy or replica of one or more images, and / or one or more drives at a first location at a second location. In some embodiments, a clone can consist of a group of objects.

[0065] In some embodiments, the data service may be a service for receiving, processing, storing, and protecting data. In some embodiments, the data service provides a high-level data and storage management capability of the system.

[0066] In some embodiments, single tenant refers to a single instance of software and supporting infrastructure that serves a single customer. In the case of single tenant, each customer has his or her own independent database and instance of the software. In some embodiments, multi-tenancy refers to a situation where a single instance of the software and its supporting infrastructure serves multiple customers. Each customer shares the software application and also shares a single database. The data of each tenant is isolated and remains invisible to other tenants. In some embodiments, a tenant represents an organization operating within the data storage system. Tenants can also be created in the system for security isolation purposes.

[0067] In some embodiments, the storage stack refers to a software defined storage (SDS) stack, where SDS is a storage architecture that uses a software layer to provision, orchestrate and manage physical data storage capacity on industry standard servers. In some embodiments, the SDS product can be designed and supported as a single product hyper-converged system that utilizes servers such as Dell EMC PowerEdge servers, but this is not limiting. SDI / SDS can automate infrastructure configuration, allowing solutions to be deployed faster and meet real-time application needs more quickly.

[0068] In some embodiments, the physical disk is a disk that can be used for data service consumption. As those skilled in the art understand, the physical disk can be locally attached to other devices (e.g., storage stacks), but this is not required for all types of physical disks.

[0069] In some embodiments, cloud computing has the following five characteristics or qualities: (1) on-demand self-service; (2) widespread network access; (3) resource pools; rapid elasticity or expansion; and (5) measurable service. In some embodiments, a cloud computing architecture includes front-end components and back-end components. Cloud computing platforms, referred to as clients or cloud clients, may include servers, fat or thin clients, zero (ultra-thin) clients, tablets, and mobile devices. These client platforms interact with cloud data storage via applications (middleware), via web browsers, or through virtual sessions. For example, the front end in a cloud architecture is the visual interface that a computer user or client sees through their web-enabled client device. The back-end platform of a cloud computing architecture may include single tenant physical servers (also referred to as "bare metal" servers), data storage facilities, virtual machines, security mechanisms, and services, all of which are built according to a deployment model and are jointly responsible for providing services.

[0070] In some embodiments, Software as a Service (SaaS) is a type of cloud computing that provides a software distribution model in which a third-party provider hosts applications and makes them available to customers over a network such as the Internet. Other types of cloud computing may include Infrastructure as a Service (IaaS) and Platform as a Service (PaaS).

[0071] In certain embodiments, a cloud-native ecosystem is a highly distributed, elastic, and container-based cloud system that can be used as a modular computing abstraction.

[0072] Kubernetes is an open source container management platform that provides a portable, scalable, open source platform for managing containerized workloads and services that facilitates both declarative configuration and automation. Kubernetes can be viewed as at least a container platform, a microservices platform, and a portable cloud platform. Kubernetes typically runs containers, such as Docker containers. At the time of writing this application, Kubernetes is an open source project managed by the Cloud Native Computing Foundation (CNCF) in San Francisco, CA.

[0073] Pivotal Cloud Foundry is a cloud-agnostic Paas solution that provides an open source multi-cloud application platform as a service. At the time of writing this application, Pivotal Cloud Foundry is managed by the Cloud Foundry Foundation in San Francisco, CA.

[0074] In some embodiments, a stack data service is a collection of data services that consume storage. In some embodiments, some data services cannot currently coexist on the same computer with other data services.

[0075] In certain embodiments, a stack client service is a collection of services that provide storage-related services to applications or other software-defined storage stacks.

[0076] In certain embodiments, public cloud services are a set of public offerings used to offload cold data or provide additional archival capacity.

[0077] In certain embodiments, a software defined infrastructure (SDI) brain provides unified management of software defined storage core components.

[0078] In some embodiments, orchestration describes the automatic arrangement, coordination, and management of complex computer systems and services. Orchestration may involve "stitching" together software and hardware components to distribute defined services or connecting workflows and automating workflows when applicable to distribute defined services. In the field of cloud computing, orchestration may include workflows and be configured to provide guided actions for purposes or goals. For example, in some embodiments herein, orchestration is run to meet goals such as wanting to "deploy a specific SDS service to a specific server or a series of servers", where orchestration is completed by running several workflows (e.g., task sequences or multiple instruction sets) that each execute certain automatic processes or workflows to achieve the purpose of overall deployment of specific SDS services (e.g., workflows, such as workflows or task lists for deploying storage stacks, workflows or task lists for registering hosts, workflows or task lists for deploying stack clients as services, etc.). In contrast, in some embodiments, for automation purposes, workflows are processed as a whole and completed as processes within a single domain.

[0079] Although vendor-specific terminology may be used herein to facilitate understanding, it should be understood that the concepts, techniques, and structures sought to be protected herein are not limited to use with any particular commercial product. In addition, in order to ensure clarity of the present disclosure, fully understood methods, procedures, circuits, components, and products are not described in detail herein.

[0080] The phrases "such as," "for example," "exemplary," and variations thereof are used herein to describe non-limiting embodiments, and are used herein to mean "serving as an example, instance, or illustration." Any embodiment described herein via these phrases and / or variations is not necessarily to be construed as preferred or advantageous over other embodiments and / or to exclude incorporating features from other embodiments. Additionally, the word "optionally" is used herein to indicate that a feature, process, etc. is provided in some embodiments but not in other embodiments. "Any particular embodiment may include multiple "optional" features unless such features conflict.

[0081] In addition, in the following implementations, it should be understood that although many specific embodiments use products of specific brands and names (e.g., Dell EMC products, Ansible, Microsoft and Amazon cloud services, etc.), none of the embodiments described herein are intended to be limited to using products from any supplier. At least some embodiments are intended to be completely hardware agnostic. However, in some embodiments, additional features (e.g., more things that can be automatically supplied, more actions, etc.) can be obtained with the use of certain supplier products, in which it is known exactly how to obtain the advantages and characteristics of the supplier products. For example, in one embodiment, if Dell EMC PowerEdge servers are used in conjunction with data services and client services are also from Dell EMC, SLO-based supply (Dell EMC brand supply based on or based on service levels) can be provided compared to other types of supply. In contrast to supply that must be based on "commodity" hardware, where special advantages, features and characteristics may not be known, those skilled in the art will understand the various advantages that can be achieved based on understanding specific hardware characteristics. In addition, it should be understood that the embodiments described herein are intended to work within any customer ecosystem, not just within the depicted Dell EMC ecosystem.

[0082] IT departments are increasingly demanding storage requirements that meet new requirements that are not easily met with traditional purpose-built appliances. Edge and near-edge workloads and tight IT budgets are driving the need for flexible software-defined storage solutions. These solutions allow customers to build their own hardware infrastructure based on their requirements using servers from the lowest bidder.

[0083] Despite these and other significant advances, substantial barriers still exist in certain information processing environments. For example, implementing data analysis functionality can be difficult under current practices. It can also be difficult to install, configure, and deploy software defined storage assets from multiple providers or multiple different types of software defined storage (SDS). As a result, companies that sell data storage systems, information processing systems, and the like are very interested in providing customers with efficient data storage solutions that minimize costs while meeting customer data storage needs. For these companies, it would be beneficial to have a way to reduce the complexity of implementing data storage.

[0084] One obstacle to data storage optimization is the difficulty in integrating the storage used (e.g., various software-defined storage stacks), especially making the integrated systems work together automatically. Customers often request that SDS have a unified or standardized interface. One reason why it is difficult to achieve this goal is because standardized user interfaces for various stacks are often lacking. For example, sometimes, each software-defined storage stack has its own deployment process and installation footprint. This can lead to inappropriate and inconsistent resource utilization. In addition, each software-defined storage stack often has its own management interface, although many SDS stacks can manage resources in a similar manner. In addition, each software-defined storage stack can disclose the consumable resources of the stack in its own way. Some stacks may not be industry standards or inconsistent with other stacks. As a result, existing software-defined storage stacks may require customized user and / or component interfaces, which requires a lot of manpower and time, and is therefore more expensive. In addition, this means that a given system is not flexible enough to adapt to changing customer needs and new or updated hardware / business requirements and respond to them.

[0085] Another issue is the diversity of storage types. IT administrators want to be able to fulfill requests for many different storage types: block, file shares, objects and streams, among others. Each storage type is unique in its protocols, access patterns, and performance and usage characteristics. As a result, each storage type requires specific resources and configuration to boot. This diversity forces customers to make heavy installations and increases overall costs.

[0086] Furthermore, IT administrators need to be able to use the minimum set of hardware to provide maximum capacity. Because different storage types require exclusive access to hardware resources, effectively balancing resources to achieve maximum capacity across all storage types with the least resources is a real challenge. Combination of usage is also a factor. A single computer server may contain more hardware than a single storage type requires. Therefore, combining storage types on a single computer server can give IT administrators more flexibility. This requires isolating computing resources through an agent that understands the predetermined strategy / policy of the storage type.

[0087] Customers use storage stacks and other storage resources in an appliance-based manner, where the purchased hardware is often pre-configured in the factory to ensure that the customer can be up and running quickly after delivery. In a typical scenario, the customer purchases its own hardware, fixes it in a data rack / stack in the data center, and uses a control plane or other software to request that the purchased system be deployed using software-defined storage so that applications can run on those servers or on other computers adjacent to those servers. The problem with this arrangement is that each individual storage device has its own installation and storage method, with various wizards or scripts to help the user, but the completely different interfaces can be difficult and inconvenient for the customer, especially when the installation, configuration, and deployment experience is different for each new device in the system.

[0088] Another issue that may affect storage system configuration and deployment is the remaining and / or unused capacity. In many systems, each storage stack (e.g., products from DellEMC of Hopkinton, Massachusetts, such as EMC ECS, VxFlex OS, SDNAS, Nautilus, and products such as Dell EMC's cloud tiering devices and / or cloud arrays, and products configured to provide the ability to move data into arrays rather than having data mobility in separate servers or devices) is installed manually (using installation scripts / wizards or forms), and configured using a command line interface (CLI) or user interface (UI) for each corresponding product using a prescribed worksheet. This practice can result in stagnant or unused storage and / or computing capacity.

[0089] To improve the simplicity of hardware installation and configuration, the industry has tried a solution, the CEPH storage product from Red Hat Software of Raleigh, North Carolina. CEPH provides a single defined open stack SDS platform, which can help some smaller installations. However, customers are still looking for an enterprise-class solution that provides at least the features of CEPH as well as additional features and benefits.

[0090] In certain embodiments described herein, the experience of installing, configuring, and deploying new hardware and software-defined storage in the data center and on the cloud is improved and unified. In certain embodiments, a configuration is provided that includes a software suite that can provide unified storage management across all software-defined storage assets, understand what needs to be deployed and connected together to service storage requests, and accelerate or decelerate the software-defined storage stack due to predictive analytics, policies, and usage at the request of IT administrators.

[0091] In certain embodiments, customers may also reduce operating expenses if they can access a single management pane around available SDS to have an easily deployed and managed product that falls into a modern cloud native ecosystem (e.g., Pivotal CloudFoundry–Pivotal Container Service (PCF-PKS) based on Kubernetes), thereby avoiding the use of high-cost hardware appliances or "resource-exclusive" virtual applications ("vApps"). PCV-PKS is a system that utilizes "production-grade" Kubernetes to enable deployment and consumption of container services. As previously described, Kubernetes provides an open source container orchestration system that is used to automate the deployment, scaling, and management of containerized applications across clusters of hosts.

[0092] Certain embodiments described herein can at least partially meet this need. In certain embodiments described herein, a mechanism is identified by which multiple storage types (block, file, object, and stream) can be installed, configured, consumed, and managed through a single management and orchestration plane that understands the requirements of each storage type and applies policies to hardware inventory to roll out enterprise-grade software-defined storage to commodity assets.

[0093] In certain embodiments, an arrangement is provided, the arrangement being configured to:

[0094] Give IT administrators a mechanism to identify the servers they wish to use for software-defined storage consumption;

[0095] Allows IT administrators to deploy from multiple storage types and even profiles within each storage;

[0096] Automates the installation of software-defined storage stack components such as storage server software (server drivers), data path software (client drivers), and management software for each storage type

[0097] Automating the configuration of software-defined storage stack components to reduce the touch points required by IT administrators.

[0098] Expose a consumption interface (API, SDK or industry standard enabler plugin) to IT administrators or other appropriate entities to accelerate consumption of storage

[0099] Monitor storage for alerts and other actionable information

[0100] ·Automate the management of the storage stack to facilitate expansion, contraction, or maintenance operations

[0101] In at least some embodiments, the arrangements described herein provide at least these advantageous features:

[0102] A unified mechanism / platform for software-defined storage storage assets.

[0103] · Automatic / unattended installation and deployment of storage stacks based on policy versus available inventory.

[0104] Hyper-converged storage stack ecosystem (multiple stacks on a single server).

[0105] It is contemplated that the embodiments described herein may be useful when utilizing at least the following technologies:

[0106] Dell EMC Elastic Cloud Storage (ECS); Dell EMC VXFLEX OS; Software Defined Network Attached Storage (SDNAS); Dell EMC NAUTILUS (a proprietary software-defined solution for storing and analyzing large amounts of streaming Internet of Things (IoT) data in real time); Glider, and storage stacks that correspond to specific storage characteristics, protocols, or “storage types” (block, file, object, stream)

[0107] Ansible: A general-purpose IT orchestration tool that can be easily embedded to facilitate orchestration of the installation, configuration, and management of the above storage stack; Ansible is an open source automation platform that can help with configuration, management, application deployment, task automation, and IT orchestration (for example, when you need to run tasks in a sequence and create a chain of events, events must occur on several different servers or devices).

[0108] Installation utilities, such as Dell EMC installation utilities.

[0109] There are known ways to install and configure various types of storage stacks (including but not limited to the aforementioned Dell EMC Elastic Cloud Storage (ECS); Dell EMC VXFLEX OS, SDNAS, and Dell EMC NAUTILUS). For example, in some instances, each storage stack is manually installed (using an installation script / wizard or form) and configured using a command line interface (CLI) or user interface (UI) for each respective product. The experience for each product is different and not uniform at all. The different experiences also mean that users cannot try to manually "share" servers between storage stacks.

[0110] In certain embodiments, as described herein, these problems are at least partially solved via a single method that automates the installation, configuration, and deployment of all types of storage stacks via a unified interface. Certain embodiments herein can describe and protect the capabilities of software in a consolidated manner, thereby enabling comprehensive control, management, deployment, resolution, and configuration of many different types of software-defined storage (SDS) assets. Certain embodiments described herein are configured to control, manage, deploy and configure DELL EMC software defined storage (SDS) assets, including but not limited to DELL EMC SCALE IO, DELL EMC VCX OS, storage connected via the Container Storage Interface (CSI is an open source dynamic container storage provisioning environment based on Kubernetes to which developers can issue persistent volume requests for specific classes of services and then present these requests to Kubernetes), appliance-based CSI storage (such as Dell EMC PowerMax and Unity), software defined ECS or object storage, ECS flex, DELL EMC ISILON FLEX and ISILON software defined (where DELL EMC ISILON is a type of scale-out network attached storage system designed for demanding enterprise file workloads), and virtually any type of data system or information system related to data protection, including but not limited to products such as DELL EMC Data Domain Virtual Edition. References herein to specific products from DELL EMC are intended to be illustrative and not limiting. Those skilled in the art will appreciate the applicability of the embodiments described herein to storage assets from many different vendors, etc.

[0111] Furthermore, certain embodiments described herein are applicable to a variety of usage categories including, but not limited to, server factory installation, server management installation, server management, service deployment management, service "care and delivery," and service usage.

[0112] With respect to server factory installation, in one example embodiment, servers from vendors (such as DELL EMC PowerEdge servers) may be pre-installed with a base operating system (OS) and prerequisite software to accelerate participation in a software-defined storage suite (SDSS) ecosystem. With respect to server management installation, the embodiments described herein permit the SDSS ecosystem to be deployed in a customer's cloud-native infrastructure. Through server management use, certain embodiments described herein can provide complete management capabilities, including quickly identifying servers for suite (SDSS) use, adding servers or a series of servers for SDSS use, non-destructively removing, replacing, or upgrading servers for SDSS use, and monitoring server health. In addition, it should be understood that in this disclosure, the terms software-defined storage system (SDSS), software-defined interface (SDI) brain, and VIKI are all used interchangeably.

[0113] For service deployment management, certain embodiments described herein deploy a particular software-defined storage service to a particular server or series of servers, configure the initial characteristics or constraints that each service is allowed to consume, and / or remove or move software-defined services from a server. For service "care and delivery", certain embodiments described herein monitor the health and capacity of software-defined services on a server or group of servers, use the user's existing reporting and monitoring systems to notify users or other entities of failures or potential failures of services, predict service shortages, and provide (semi-)automatic repair or expansion of required data services to remain within SLAs. For service usage, certain embodiments described herein use industry standard specifications to create, expand, and remove volumes, shares, buckets, streams, or protections.

[0114] In certain embodiments, various configurations are provided that advantageously provide one or more of the following features:

[0115] Deploy multiple services on shared infrastructure

[0116] Provide services to many tenants.

[0117] Elastically scale platforms and services

[0118] Flexibly shift capacity between services

[0119] Providing operational consistency across services

[0120] Provide visibility to operators and account users.

[0121] Figure 1An illustrative functional diagram 100 of a management and orchestration (M&O) plane 102 (also referred to herein as a layer) encapsulating an exemplary software defined storage platform at a high level, in accordance with at least one illustrative embodiment of the present disclosure. Figure 1 A management plane 102 is depicted running within a customer's existing cloud native ecosystem, which installs, configures, and manages various storage stacks 106-120 (discussed further below). Figure 1 In the exemplary diagram 100 of FIG. 1 , the M&O (which in some embodiments is also referred to as the controller or “brain” of the system) is shown as being part of a cloud computing ecosystem (e.g., by being part of Kubernetes or PCF, as shown in FIG. 1 ). Figure 1 ). The M&O plane 102 is operable to communicate with a plurality of potential software defined storage stacks that may be used in conjunction with a cloud computing ecosystem. Figure 1 In the example of , these storage stacks include a software defined (SD) block 106, a software defined network attached storage (SDNAS) block 108, a cloud gateway 110, a user application 112, DELL EMC Data Domain Virtual Edition 114, DELL EMC Elastic Cloud Storage (ECS) 116, DELL EMC NAUTILUS 118 and DELL EMC ISILON 120.

[0122] exist Figure 1 , SD block 106 , SDNAS 108 , cloud gateway 110 , user application 112 , and DDve 114 are all in operable communication with DELL EMC VxFlex OS 104 . It should be noted that the "OS" in "VxFlex OS" is part of the product name of this exemplary product, formerly known as Dell EMC ScaleIO; "OS" does not stand for "operating system"). VxFlexOS104 provides large-scale, high-performance elastic base storage. VxFlex OS is a software-defined storage product that creates a server-based storage area network (SAN) from local application server storage using existing customer hardware or EMC servers. In some aspects, VxFlex OS converts direct-attached storage into shared block storage. In some embodiments, VxFlex OS aggregates compute resources and commodity storage into a single-tier architecture. VxFlex OS can be deployed as storage only, or as a converged infrastructure that combines storage, compute, and networking resources into a single block. The capacity and performance of all available resources are aggregated and provided to each participating VxFlex OS server and application. Storage tiers can be created with media types and drive types that match ideal performance or capacity characteristics to best suit application needs.

[0123] SD block 106 provides advanced block data services, data reduction, snapshots, replication, tiering, D@RE (Dell EMCUnity data at rest encryption (D@RE) and multi-tenancy. SDNAS block 108 provides multi-tenant NAS. Cloud gateway 110 is connected to external cloud services 109, such as Amazon Web Services, Microsoft Azure and Web Stream (these are illustrative and not limiting). User application 112 represents a plurality of user applications that can be used in this arrangement, including but not limited to NoSQL DB, graph DB, stream processor, map reduce, elastic search and traditional applications. As described above, DDve 114 provides a link to enterprise data protection and can also be used with cloud service 109. Amazon cloud service. In some embodiments, ECSSD 116 is used with objects, Nautilus 118 is used for storage streams, and Isilion 120 is used with large-scale NAS. In some embodiments, software-defined (SD) NAS, a high-performance NAS type, is another data service that can be used.

[0124] In at least one embodiment, Figure 1 The system 100 may be implemented using DELL EMC POWEREDGE Blade servers and services, which may include a software defined storage stack (SDS0 (i.e., Figure 1 However, it should be understood that the embodiments herein are not limited to PowerEdge servers in any respect, including but not limited to the M&O 102 or storage stack deployment itself; advantageously, in some embodiments, the M&O 102 is hardware-independent.

[0125] Figure 2200, according to an illustrative embodiment, shows how multiple data services (e.g., applications 112, VxFlex OS 104, SD block 106, SDNAS 108, DDve 114, cloud gateway 116, ECS, Nautilus, Isilon) coexist together on a server 202 present in an exemplary system, including applications run by customers. For example, the top rack of servers on the far right includes seven data services, including four applications 112 (represented by the letter "A"), VxFlex OS 104 (represented by the letter "S"), SD block 106 (represented by the letter "C"), and SDNAS 108 (represented by the letter "N"). Those skilled in the art will understand that other racks in the server can be similarly interpreted using the letter codes in the figure. Advantageously, as further described herein, the M&O control plane 102 will provide a unified management interface for configuration, deployment, provisioning, management, etc. across all of these storage-defined assets. The M&O plane handles management, orchestration, deployment, configuration, and registration.

[0126] Figure 3 is a simplified diagram of an architecture 300 implementing a software defined storage suite according to one illustrative embodiment. Figure 4 According to an illustrative embodiment Figure 3 A more detailed diagram 400 of the architecture is shown. Figure 3 and Figure 4 Depicts the same architecture but at a different level of detail. Figure 4Multiple different types of stacks in the stack data services 324 section (described further below) are also described in detail, for example, DELL EMC Nautilus, SDNAS, DELL EMC VxFlex OS (also referred to herein as SIO), etc., as well as cloud data services, and as those skilled in the art will appreciate, there may also be other types of storage stacks from different vendors. As those skilled in the art will appreciate, each type of stack (including the cloud data interface) may have its own proprietary management interface and management software running "under" the proprietary management interface. For users who want to control, manage, deploy, provision, configure, etc. any of these stack resources (these users may include not only IT personnel, but also non-human entities such as controllers, processors, computing systems, applications), doing so is complex, time-consuming, and costly. Each product may have its own unique requirements to utilize the product's own wizards or scripts to complete these tasks, which increases the burden on users / clients who have to learn different installation and configuration techniques and use the techniques for each system. One way that at least some of the embodiments described herein help address this problem is by providing a unified and automated experience of controlling, managing, deploying, provisioning, and configuring storage assets, particularly software-defined storage assets, such as stack data services. According to at least some of the embodiments further described herein, systems, methods, and apparatus are provided to provide an architecture with a control plane configured to provide a unified user experience. This will be further described herein.

[0127] refer to Figure 3 and Figure 4 , Figure 3 Architecture 300 and Figure 4 The architectures 400 of FIG. 300 each include a software defined storage (SDS) management and orchestration (M&O) plane 102 (also referred to herein as layer 102) as described above, which interacts with the surface area 302 and the plurality of storage stacks 306. Note also that the M&O control plane 102 of FIG. 300 explicitly shows an element called a "stack management service 302" (discussed further below) outside the M&O control plane 102; such a configuration is not Figure 4 The same applies, but Figure 4 The level of detail shown does not make this configuration easy to see.

[0128] exist Figure 3 and Figure 4In the exemplary architecture of FIG. 3 , considering these architectures starting from the bottom to the top, the storage stack 306 includes a stack client service 332, a stack data service 334, a plurality of physical disks 336a to 336c, support 338, and one or more public cloud services 340. Generally, in some embodiments, the storage stack 306 is implemented using an existing software defined storage stack. The physical disk 336 includes a set of (usually locally attached) physical disks that are available for consumption by the data service.

[0129] The stack data services 334 provide a set of data services to consume storage. As will be appreciated by those skilled in the art, some data services cannot currently coexist on the same computer with other data services, and in certain embodiments, it is not required that these data services coexist on the same computer. The stack client services 322 provide a set of services that provide services to applications or other storage aspects of the software defined storage stack ( Figure 3 or Figure 4 The layering / dependencies of the services are not shown).

[0130] In some embodiments, public cloud service 340 includes a set of public products that are not limited to offloading cold data or providing additional archival capacity; in some embodiments, it is possible to move the entire workload of data to public cloud service 340 or another data center. Figure 4 As shown, in an exemplary embodiment, these may include public cloud services 340 (including, but not limited to, Amazon Web Services (AWS), Virtustream, and Microsoft Azure, which may be used as public cloud services 340). Support 338 may include any kind of support available for the M&O plane 102. In some embodiments, support 228 is DELL EMC support, such as Secure Remote Services (SRS) call center support for M&O. In some embodiments, services such as SRS maintain an inventory of customer assets; therefore, in some embodiments, Dell support 338 may also be used to populate inventory services 402 in the M&O layer 102 by helping to verify which storage stacks are "healthy" or at least have no issues that affect expected performance. This may be more convenient for the user because the need to manually provide input information is eliminated.

[0131] The stack management service 330 is a service that performs management (control path) of an already existing storage stack. Advantageously, if the stack management service 330 is not already enabled, it can be promoted to a cloud native cluster. In some embodiments, the stack management service 330 is configured to be "pluggable", meaning that the high-level features and changing functionality of the service can be driven by plugins and automatically reflected in the M&O layer without the need for additional adaptation by the unified service. Figure 4As shown, the stack management service 330 includes a plurality of management services corresponding to / matching a specific stack data service 334 (e.g., management services named “ECS management”, “SD block management”, “DDve management”, etc.)

[0132] The M&O plane 102 is also referred to herein as the software defined interface (SDI) “brain” and, in certain embodiments, provides unified management of software defined storage core components. Figure 4 As shown, in some embodiments, the M&O plane 102 includes a cloud native ecosystem 323 that provides excellent external expansion, configuration, and management capabilities, such as Kubernetes / Pivotal CloudFoundry. Figure 3 and / or Figure 4 The user / customer of the architecture of the software-defined storage stack 306 already has a cloud native ecosystem in its private cloud. In some embodiments, the server management service 322 is a set of open source tools that are used to perform the installation, deployment, and configuration of the software-defined storage stack 306.

[0133] The unified software-defined management service 304 is a set of services that understand the managed servers and services and allow users to create and store any policies around deployment. Given the deployed hardware, the unified software-defined management service 304 also stores configuration templates and applies the configuration templates to the storage stack to best practice configuration.

[0134] Monitoring and reporting (M&R) components 321 are external M&R products and services that are integrated into the unified software-defined management service to provide dashboard notifications of server or service failures, as well as enabling the platform to perform elastic scaling of specific stacks based on policies and M&R alerts (this will be discussed further herein). In some embodiments, products configured to allow users to monitor, analyze, and troubleshoot a given storage environment from any perspective, such as DELL EMCCLOUDIQ, can be used to implement this aspect of the embodiment.

[0135] The management REST (Representational State Transfer) endpoint 316 is exposed through the REST endpoint. In some embodiments, the management REST endpoint 316 needs to comply with industry standard specifications. In some embodiments, the surface area layer 302 exclusively provides the desired touch points.

[0136] The surface area plane or layer 302 includes enablers 308, a software developer kit (SDK) 310, playbooks 312, and a user interface 314. The surface area 302 describes how to expose the SDI brain 323 capabilities to customers. It should be understood that in this disclosure, the terms software defined storage system (SDSS), software defined interface (SDI) brain, and VIKI are all used interchangeably. In certain embodiments, it is advantageous if the surface area 302 includes a "pre-packaged" industry standard SDK and enablers.

[0137] The user interface (UI) 314 can be implemented in various ways in various embodiments. In one embodiment, the UI 314 is a native UI that communicates directly through the SDI brain management REST endpoint 316 (in this case, a portal service can be added to the unified software-defined management service 304 of the SDI brain). In one embodiment, the UI 3114 is a web-based graphical interface that cloud administrators and users can access to manage compute, storage, and networking services, such as an OpenStack Horizon-based UI (in some embodiments, it is present with the OpenStack core plugin and the Horizon plugin elsewhere in the system 400).

[0138] In certain embodiments, Figure 3 and Figure 4 The management REST endpoint 316 includes a set of platform REST endpoints, combined with Figure 3 , Figure 4 , Figure 5 , Figure 7 and Figure 8 , these endpoints are discussed further below. Figure 5 FIG. 1 is a diagram showing an exemplary embodiment of the present disclosure. Figure 4 A detailed diagram of the architecture of a controller service is shown in FIG500.

[0139] Figure 5 The platform REST endpoint 502, native service 504, stack deployment request mapper 605, stack coordinator orchestration component 508, computing controller 509, open source service 510, DELL EMC service 512 (the service is different from Figure 3 and Figure 4 328), API gateway 514, load balancer 516, platform configuration management service 518, task service 520, workflow execution controller 522, Ansible library 524, consumption controller 526, consumption request mapper 528, placement analysis 530 and stack deployment orchestration controller 532. These will be further described below.

[0140] Figure 5 The VIKI platform service in the cloud runs in the customer's existing cloud native environment (such as Kubernetes cloud environment 323). It should be understood that in the present disclosure, the terms software defined storage system (SDSS), software defined interface (SDI) brain and VIKI are all used interchangeably. VIKI is deployed to many nodes for high availability, and service orchestration is achieved through Helm charts, which can be used to simplify packaging and deploying common applications on Kubernetes. Requests to the platform will be served by multiple services running independently. The API gateway 514 multiplexes incoming uniform resource locators (URLs) to services that can serve requests in the platform. In some embodiments, several identical services may exist within the cloud native ecosystem 323. A load balancer 516 such as NGINX spreads the request load (relatively) evenly across the request load services.

[0141] In some embodiments, native services 504 are all services implemented through the VIKI platform, except AWX.

[0142] The compute controller 509 exposes a REST interface for adding servers to the inventory. This inventory is used to deploy and install the software defined storage stack.

[0143] The stack deployment orchestration controller 532 exposes a REST interface for deploying storage stacks on servers, which also configures and guides the stack based on user input or best practice templates stored in the policy manager. The consumption controller 526 exposes a REST interface that allows consumers to consume storage of each corresponding storage stack type. The simplest form is a pass-thru to the underlying stack. In a complex flow, it dynamically drives the elastic expansion or deployment of the storage stack to meet requests.

[0144] Open source services 510 are leveraged to reduce infrastructure development costs. Open source services are used by many of the native services 504. Dell EMC services 512 are services used to help facilitate cross-functional activities, such as M&R ​​and support / call center. Platform configuration management services 518 (also referred to herein as base services in VIKI) are independent of or shared by controllers, such as Figure 5 The computing controller 509, the stack deployment orchestration controller 532 and the consumption controller 526 are provided.

[0145] In certain embodiments, Figure 5 The controller service in is the VIKI controller service. Figure 5 The platform REST endpoint 502 at the top of the includes everything related to requests to install, configure, or register SDS assets or other storage all the way to consumption requests, that is, everything the management system must provide. Figure 5 Below the platform REST endpoint 502 in the native service 504 block, other endpoints in the native service 504 block can be divided into deployment REST endpoints and consumption REST endpoints. In some embodiments, like deployment requests, consumption requests are passed in through the native service 504, and then these consumption requests are delivered to the consumption request manager 528.

[0146] Figure 6 According to at least one disclosed embodiment, the Figure 3-Figure 8 600 for working with the architecture of the present invention. In some embodiments, this user interface (UI) 600 is implemented as a browser extension, but this is not limiting. The management (admin) field 602 in the exemplary UI 600 enables the UI 600 to function as a browser extension. Figure 3-Figure 8 The UI 600 is a central management interface for the Software Defined Storage Infrastructure (SDSI) of the present invention. Via an “Object” button 604 on the UI 600, the UI 600 provides a way for the user to request to roll up active software defined data services into logical groupings. Via an “Add Service” button 606 in the UI 600, the user is able to select, for example, “Add Service” and the UI 600 understands how the storage stack is deployed and configured (this will be discussed more exclusively herein). Via a “Cloud Gateway” button 608, the UI 600 allows the user to leverage predictive analytics and events from M&R to elastically scale services to new nodes (this will be discussed in more detail herein). Via a “Virtual Flow” button 610, the UI 600 simplifies deployment and configuration for the user because the system understands the storage stack dependencies and how to orchestrate the deployment and configuration.

[0147] Figure 7 For an example according to an embodiment Figure 5 700 of the components within the stack deployment orchestration controller 532. In some embodiments, the overall purpose of the dashed box 714 (and other similar boxes next to it) is to push specific types of logistics and business logic down to a pluggable architecture so that the top layer can be simple and not affected by individual technology requirements. In some embodiments, the dashed box 714 depicts Figure 5 Any of the block controllers, fire controllers, object controllers, projections, and flow controllers shown. The overall purpose of boxes 705, 708 within box 714 (and any other boxes adjacent thereto within the dashed box 714) is to push technology-specific configuration parameters and business logic down into a pluggable architecture.

[0148] Still refer to Figure 7 , from top to bottom, the components in this part of the architecture are configured as follows:

[0149] Northbound requests to deploy a specific type of storage service enter the stack deployment request mapping layer 506. The responsibility of this service is to validate parameters, dispatch to the stack coordinator orchestration service and return a task ID and other relevant information.

[0150] The stack coordinator orchestration service 508 receives a request to dispatch a storage service from the request mapper service 506. The stack coordinator orchestration service 508 can receive multiple dependent deployment requests (such as SDNAS on VxFlex OS) in the same request. The stack coordinator orchestration service 508 is responsible for loading the corresponding stack type and calling the necessary interface to achieve the requested result. The stack coordinator orchestration service supports previewing and executing corresponding operations.

[0151] The first plug-in capability 702 is implemented by the service in the following dashed box 714, which implements the REST interface contract and registers itself as a provider of the stack orchestrator orchestration with respect to the service registry.

[0152] <Type>Controller (Block, File, Object, Stream, Protection) 704 implements a basic set of REST interfaces and registers services with respect to registrations with a specific type. In this example embodiment, only one primary controller per <Type> is allowed, and the type is fixed.

[0153] Each main type has a set of REST entry points to implement according to the second plug-in capability 706. For deployment, the block selection analysis is the deployment entry point that determines the appropriate service to deploy given a deployment request.

[0154] The stack implementation (block / VxFlex OS, file / SDNAS, etc.) 708 registers itself with respect to the service registry and implements knowledge about: what resources are needed to service such a request (create a new pool, deploy a new stack, etc.); how to describe the actions that need to be performed for a preview request; and how to initiate a deployment or management operation using the bedrock service, which determines how to configure and execute Ansible scripts to provide automation of IT management tasks.

[0155] The southbound driver 710 of each stack is responsible for talking to the deployed storage stack management interface (VxFlex OS management or "MDM") to get information about how to satisfy the request. In some embodiments, the southbound driver 710 also dispatches to the bedrock to get basic information about the host and utilities, and the stack implementation can use this information.

[0156] Figure 8 In accordance with at least one embodiment, Figure 4FIG80 is a second diagram of additional components used within stack deployment orchestration controller 532 of FIG80. Stack deployment orchestration controller 532 allows for installing a stack, changing a stack, retrieving information about a stack, creating new resources within a stack (e.g., a new pool, in some embodiments), and returning task objects for long-running operations.

[0157] Fig. 9 According to one embodiment, Figure 3-Figure 8 A first simplified flowchart 900 of an exemplary method of storage service deployment for use with a system of the present invention, and Fig.10 According to one embodiment, Figure 3-Figure 8 A second simplified flow chart of an exemplary method of service storage deployment for use with the system. Fig. 9 Describes the Fig.10 A similar approach is used, while Fig.10 Further details are provided for each major type of analysis performed. In certain embodiments, Fig. 9 and Fig.10 The methods can be used for various applications, such as deploying a particular software-defined storage service to a particular server or a series of servers. For example, in some embodiments, customers may want to deploy their software-defined storage stack before consumption requests come in. According to some embodiments herein, the systems, methods, and apparatus described herein (including but not limited to Figure 3-Figure 8 architecture) has the following capabilities: (a) when a client / customer request comes in, understand the stack and resources potentially available for use; and (b) serve the request by deploying the stack or simply partitioning the resources within the stack to meet the details of the storage requirements.

[0158] exist Fig. 9 In the example of FIG. 9 , upon receiving a request (block 910), three types of analysis are completed: mapping analysis (block 910), stack analysis (block 920), and resource analysis (block 930), and based on the results of these analyses, the orchestration (block 940) executed will be able to deploy and couple the content required to serve the received storage request. In some embodiments, the received request will include stack parameters and resource characteristic parameters, wherein the stack parameters specify at least one type of storage asset, and the resource characteristic parameters specify at least one functional capability required of the storage asset.

[0159] As an example, consider the case where a client / client request for a NAS share is received ( Fig. 9 Block 905, Fig.10 In some embodiments, a request for a type of storage share is referred to as a consume request ( Fig.10 block 1050). According to Fig.10, analyzing the client / client request (as further described herein) to determine what is being requested. In this example, assume that the received consume request indicates that the NAS share (e.g., stack parameter) must be located on very fast storage (first resource characteristic parameter) with in-line deduplication capability (second resource characteristic parameter).

[0160] First, refer to Fig. 9 and Fig.10 , run a mapping / placement analysis ( Fig. 9 Block 910, Fig.10 Block 1052 of FIG. 1054 ) analyzes the request to determine the type of stack required to service the request ( Fig.10 Block 1052). The systems, methods, and apparatus herein are also capable of determining (in response to a received request) whether resources are available and deployed via analysis of blocks 910-940 (and blocks 1050-1066). In this hypothetical example, assume that: in response to a request for a NAS share of extremely fast storage with in-line deduplication capabilities (block 1052), the list received at block 1054 indicates that the received request can only be serviced by a combination of SDNAS, SD Block, and VxFlexOS storage stack.

[0161] In some embodiments, stack analysis (block 920) involves checking the status of a storage stack that can satisfy the storage request (or is needed / required to satisfy the storage request) to determine whether the stack is deployed to satisfy the storage request (block 1056) based on the status (block 1054). The current storage is analyzed, and the status of the storage is received (block 1054). In the current assumption, the status indicates that the SIO (also referred to herein as DELL EMC VxFlex OS) and SDNAS products are deployed within the available storage. Fig.10 In the method of , if these two products are the only stack required, then the answer at block 1056 can be "yes" at this time.

[0162] However, in this example, SIO is not the only stack required, so the analysis continues for each necessary stack. For example, the stack analysis may determine that the software defined (SD) block is not deployed at all because the current suite receiving the customer / client request has SIO deployed, but there is only one pool on the hard disk drive (HDD) and no solid state drive (SSD). In this assumption, it is known that the SD block required to service the request is not deployed (the answer at block 1056 is "no").

[0163] In this case, a workflow (e.g., a set of tasks or scripts) is added to the workflow list to be run at orchestration time (when the storage or share is available to the customer / client. For example, for each undeployed stack, one or more workflows are added to the orchestration as needed to deploy the undeployed stack. Therefore, after the stack analysis (block 920) is completed, methods 900 and 1000 then perform a resource analysis (blocks 930, 1060-1064), wherein the resource analysis of each stack (whether deployed or not) has the resource characteristics required for the customer's requested performance level and has the required characteristics (block 1060). Based on this, the SDS suite / framework will be able to respond to the request to automatically provide resources.

[0164] refer to Fig. 9 Method 900 and Fig.10 Method 1000, performs mapping analysis (block 910). In some embodiments, for example, the SDS suite / architecture determines in the mapping analysis block that the received request (block 905) can only be serviced by the SDNAS, SD block, SIO combination (block 910).

[0165] As part of the stack analysis (block 920), the SDS suite / architecture determines whether the required resources have been deployed. In some embodiments, the stack analysis includes evaluating an open inventory (e.g., of storage resources) against a best practice template. In this example, the SDS suite / architecture determines that SIO has been deployed, SDNAS has been deployed, and the software-defined (SD) block has not been deployed. In some embodiments, the SD block implements a set of additional capabilities that can be used for block storage by VxFlex OS. For example, in a hypothetical case, because the SD block is not deployed, as the last part of the stack analysis, the SDS suite / architecture determines that it should add a script or workflow to the orchestration to enable the script or workflow to be deployed, such as "Run an Ansible script for deploying the SD block against best practices" (this is exemplary and not limiting).

[0166] Reference again Fig. 9 , Fig.10 After the stack analysis is complete, a resource analysis (block 930) is run to determine whether each stack (whether deployed or not) has the required resource characteristics and required features for the performance level (block 1060). In the resource analysis block (block 930), a check is made to determine whether the available and deployed stacks meet the other requirements of the request. More details on these types of checks are further described herein. If the resource is insufficient in the required features or characteristics (i.e., the answer at block 1062 is "yes"), a workflow / script is added to the orchestration to generate the required features or characteristics.

[0167] For example, in the assumption, if the SIO resource does not have a pool that meets the high performance (which is specified in the request), a workflow / script is added that adds "Create SSD-based SIO pool" (at the time of writing this application, SSD devices are generally devices with higher performance than HDD devices, which is known in the art). In another example related to the assumption, if the SD block resource requires corresponding mapping to provide in-line deduplication capabilities, a workflow / script is added to the orchestration, which states "Create in-line deduplication volume mapping in SD block". In another example, if the resource analysis indicates that SDNAS needs to create a resource (share) suitable for the profile, then in clock 1064, a workflow / script configured as "Create SDNAS file share" is added to the orchestration.

[0168] When there are no longer missing resources in the required features and characteristics (ie, the answer at 1062 is "no"), the run orchestration is executed (block 940, block 1066), which runs all scripts added to the orchestration.

[0169] Fig.11 According to at least one embodiment Figure 3-Figure 8 Figure 1100 and an exemplary sequence of actions that occur in the architecture of FIG. 1100 (occurs during storage service deployment), illustrating the operations that occur when a customer requests a network attached storage (NAS) share until the workflow is assembled and delivered to the Figure 3-Figure 8 The workflow execution service ends. Fig.11 The sequence of actions is shown on the right side of the figure as actions numbered (1) to (9), with corresponding numbered labels in the architecture on the left side of the figure showing which elements of the architecture help implement the sequence.

[0170] Fig.11 The architecture and Figure 4 The architecture of is substantially the same, but some of the elements in the architecture are shown in greater detail. Figure 4 The detailed reference identification is not included in Fig.11 to improve clarity, but it is expected that similar elements have similar numbers. This sequence is implemented to achieve the same as above with respect to Fig. 9 and Fig.10 The flowchart discusses the assumptions, where Fig. 9 and Fig.10 The method shown still applies. Fig.11 In more detail, for example Figure 3 and Figure 4 How can the components of the architecture be implemented? Fig. 9 and Fig.10 method.

[0171] refer to Fig. 9 , Fig.10 , Fig.11, assuming the same use case as discussed above: a customer requests a NAS share for ultra ("uber") fast storage with in-line deduplication. The current SDS storage suite in the system receiving the request has SIO deployed, but only one pool is on HDD (slow storage) and not on SSD (fast storage), and the SD blocks are not deployed at all. Starting from step 1 in the sequence, at step (1), a request for a NAS share arrives at the fabric, in this example via the management REST endpoint 316 ( Figure 3 , Figure 4 At step (2), a placement analysis (also referred to herein as a mapping analysis, e.g., Fig. 9 Block 910 and Fig.10 As shown in block 1052 of Fig.11 As shown, in this embodiment, the request mapping service 318 performs placement / mapping analysis (2), stack analysis (3), (4), and (5), and resource analysis (6), (7), and (8), each of which is performed in a manner and with respect to Fig. 9 and Fig.10 The method described above is basically similar. Fig.11 Also shown is a stack driver 414 ( Figure 4 ) helps communicate information associated with steps (3), (4) and (5) of stack analysis and steps (6), (7) and (8) of resource analysis to associated stack management services 330 (e.g., SDNAS management and SIO management). The workflow execution service 410 then runs the orchestration at (9).

[0172] Fig.12 To illustrate the invention according to at least one embodiment Figure 3-Figure 8 An illustration of the automatic vertical functionality that occurs within the architecture of FIG1200. In particular, Fig.12 The architecture shown is Figure 4 The architecture 400 is compared to the "after" architecture to better illustrate the actions that occur. In this exemplary embodiment of the architecture, the architecture 1200 implements a management platform that automates the installation, deployment, and configuration of a particularly simplified subset of storage (e.g., SDNAS on VxFlex OS using the industry standard front-end Ansible Tower for consuming only file shares in this exemplary embodiment). Fig.12 As shown, the architecture 1200 includes, from bottom to top, the software defined storage (SDS) management and orchestration (M&O) plane / layer 102 (also referred to herein as layer 102) as described above, which interacts with the surface area 302 and one or more storage stacks 306. Figure 4 Likewise, in some embodiments, the stack management service 302 is external to the M&O control plane 102 .

[0173] consider Fig.12 Starting from the bottom up, the storage stack 306 includes a plurality of physical disks 336d-336c, a stack data service 334, and a stack client service 332. Generally, in some embodiments, the storage stack 306 is implemented using an existing software defined storage stack, such as the illustrated SIO data service 1248 (however, such data services are illustrative and not limiting). The physical disk 336 includes a set of physical disks available for consumption by data services.

[0174] The stack data service 334 provides a set of data services to consume storage. The stack client service 322 provides a set of services that provide services to the storage aspects of the application or other software defined storage stack (the layers / dependencies of the services are described in Fig.12 not specified in the table).

[0175] The stack management service 330 is a service that performs management (control path) of an already existing storage stack. Advantageously, if the stack management service 330 is not already enabled, it can be promoted to a cloud native cluster. In some embodiments, the stack management service 330 is configured to be "pluggable", meaning that the high-level features and changing functionality of the service can be driven by plugins and automatically reflected in the M&O layer without the need for additional adaptation by the unified service. Fig.12 As shown, for example only, stack management services 330 include SIO management services and SDNAS management services. As will be appreciated, stack management services layer 330 will typically include management services for client services and data services that are actually in stack client services layer 332 and stack data services 324, respectively.

[0176] The M&O plane 102 is also referred to herein as the software defined interface (SDI) “brain” and, in certain embodiments, provides unified management of software defined storage core components. Fig.12 As shown, in some embodiments, the M&O plane 102 includes a cloud native ecosystem 323, such as Kubernetes / Pivotal Cloud Foundry. In some embodiments, assuming the use of Fig.12 The user / customer of the architecture may already have a cloud native ecosystem in its private cloud. In some embodiments, if the user or customer does not have a cloud native ecosystem in its private cloud, the architecture 1200 can provide a cloud native ecosystem.

[0177] The server management service 322 is a set of open source tools that are used to perform the installation, deployment, and configuration of the software defined storage stack 306. Fig.12 In the example embodiment of , the illustrative server management source used is Ansible, but this is not limiting.

[0178] The unified software-defined management service 304 is a set of services that understand the managed servers and services and allow users to create and store any policies around deployment. Given the deployed hardware, the unified software-defined management service 304 also stores configuration templates and applies the configuration templates to the storage stack to best practice configuration.

[0179] The management REST (Representational State Transfer) endpoint 316 is exposed through the REST endpoint. In some embodiments, the management REST endpoint 316 needs to comply with industry standard specifications. In some embodiments, the surface area layer 302 exclusively provides the desired touch points.

[0180] The surface area plane or layer 302 includes a base or "basic" user interface (UI) 1202, such as reuse of existing UI assets. Advantageously, in certain embodiments, the implementation leverages the customer's existing IT ecosystem, such as Ansible Tower, VMware vRO, and / or other orchestration and management software, such that some of at least some of the embodiments herein can be easily "plugged" into the existing ecosystem. The surface area layer 302 also includes a base organizational unit or collection of scripts that define the work of automated server configuration for consumption by the platform, such as Ansible scripts 1204. The surface area 302 depicts how the SDI brain 323 capabilities are exposed to customers. It should be understood that in this disclosure, the terms software defined storage system (SDSS), software defined interface (SDI) brain, and VIKI are all used interchangeably.

[0181] Fig.13 FIG. 1 is a diagram showing a method for detecting a Figure 3-Figure 12 A simplified flowchart 1300 of a method for registering and deploying hosts and storage of an architecture. Fig.14 According to at least one embodiment Fig.12 The architecture and Fig.13 1400 and an exemplary sequence of actions occurring in a simplified flowchart of the present invention, showing the components that come into play during host registration and deployment of software-defined shared block storage (based on direct attached storage) such as VxFlex OS and software-defined network attached storage (SDNAS). Fig.13 Flowchart and Fig.14 The corresponding diagrams and sequences are specifically referred to in this article Fig.12 However, it will be appreciated that such description is also applicable to Figure 3-Figure 4 architecture.

[0182] Reference now Figure 12-14In one embodiment, the overall method illustrates a process for a customer / client to provision file shares in a faster and simpler manner than the current manner. Where a customer can use the architecture to quickly and automatically register a host with the system and deploy stack data services (e.g., SIO) and stack client services (e.g., SDNAS) to quickly provision file shares. Fig.13 In block 1310, the client / client starts the process to quickly provision file shares. Fig.14 In step (1) of the related sequence, a request to register a host is passed in via the management REST endpoint 316, and the request is routed to the compute controller (see Fig.14 ). Register the LINUX host for VIKI use ( Fig.13 This registration process involves computing the controller ( Fig.14 ) Contact the LINUX host, discover basic information about the LINUX host, and store the learned inventory information ( Fig.14 step (2) of the sequence). Fig.15 According to at least one embodiment Fig.12 The architecture and Fig.13 Schematic diagram and exemplary sequence of actions occurring in a simplified flow chart of FIG. 1 , showing the components that function during LINUX host registration.

[0183] Fig.16 According to at least one embodiment Fig.12 The architecture and Fig.13 An illustrative diagram and exemplary sequence of actions occurring in a simplified flowchart of FIG1500 showing the components that come into play during the deployment of Scale IO as a Service. Fig.15200. An exemplary sequence of steps (1) of the sequence, in which, as part of registration, the management REST endpoint 416 shares the LINUX host information. In step (2), the compute controller contacts the bedrock controller with the password information. In step (3), the bedrock controller uses an existing script from Caspian to pass ssh keys to facilitate later playback of the Ansible script. In step (4), the bedrock controller "hides" the asynchronous nature of the ansible script and returns status 202 to the compute controller. In step (5), the compute controller also passes the ssh keys so that the compute controller can access inventory / discovery information. In step (6), the compute controller executes the Ansible built-in module to obtain the discovery information. In step (7), the compute controller stores the host information in the form of an Ansible consumable file or artifact that can be referenced by the bedrock controller or Ansible itself given the grouping / labels during the deployment request. In step (8), the management REST endpoint 416 response is returned to the entity that requested to register the LINUX host.

[0184] Still refer to Fig.13 and Fig.14 In response to a request to deploy VxFlex OS as a service (blocks 1330-1332), the request mapping service 318 identifies an appropriate script to execute (e.g., by communicating with an Ansible script, such as Fig.14 As another part of the request, the workbook is executed, providing feedback to the user ( Fig.14 The workflow execution service 410 executes the workflow and installs the components of the VxFlex OS ( Fig.14 1204), wherein in this embodiment, VxFlex OS is deployed with default values. This request may be passed in via a number of sources (e.g., via REST API 316, user interface 1202, or Ansible script 1204). This registration process will also be discussed briefly below. Fig.16 The sequence is described in further detail.

[0185] Fig.16 According to at least one embodiment Fig.12 The architecture and Fig.13 A schematic diagram and exemplary sequence of actions occurring in a simplified flowchart of FIG1600 showing the components that come into play during the deployment of Scale IO as a Service. Fig.16, which is used to deploy a stack data service 324 (in this case, VxFlex OS), in step (1), a VxFlex OS deployment POST request is received at the management REST endpoint 416 and the request is associated: the group / host tags are picked up by the Bedrock Controller of the request mapping service 318. In step (2) of the sequence, the Bedrock Controller retrieves the group / host from the REST request payload from the "Ansible file" (alternatively, in some embodiments, if the underlying Ansible execution can retrieve the "Ansible file", this step may be unnecessary). In step (3), the Bedrock Controller returns the "task id" to the caller (i.e., whatever entity is requesting the deployment of VxFlex OS as a service) for subsequent status updates of the deployment. In step (4), the Bedrock Controller executes the Ansible script to deploy the VxFlex OS components on the corresponding nodes. (MDM management, SDC application connector, and SDS storage server). In step (5), the Ansible script deploys the components. In step (6), the Ansible script creates a basic storage pool in the VxFlex OS (eg, in the stack management service 330).

[0186] In step (7), the Bedrock Controller polls the Ansible output and "stores" important final state information for each host / node and component deployed, as well as simple status fields. In some embodiments, because the final state information is stored, there is no need to provide an overall status. In some embodiments, it is necessary to know the unique identifier of the VxFlex Os storage pool so that the identifier can be used when creating NFS files through SDNAS (as discussed further herein). In step (8), a GET request with "task id" from the northbound API retrieves the cached status information from step (6) above until all messages associated with the task id are in a final state.

[0187] Now refer to it again Figure 12-14 ,exist Fig.13 Flowchart and Fig.14 In the sequence, now in Fig.13 Blocks 1340-1342 and Fig.14 Step (8) where SDNAS will be deployed as a service. Fig.17 The process for deploying SDNAS as a service is basically similar to the process flow described above for deploying VxFlex OS as a service.

[0188] Fig.17 According to at least one embodiment Fig.12 The architecture and Fig.13A schematic diagram and exemplary sequence of actions occurring in a simplified flowchart of FIG. 10 , showing the components that come into play during the deployment of SDNAS as a Service. Fig.17 In step (1) of the sequence, a VxFlex OS deployment POST request is received at the management REST endpoint 416 and the group / host tags are picked up by the Bedrock Controller of the request mapping service 318. In step (2) of the sequence, the Bedrock Controller retrieves the group / host from the REST request payload from the "Ansible file" (in one embodiment, if the underlying Ansible execution can retrieve the "Ansible file", this step may be unnecessary). In step (3), the Bedrock Controller returns the "task id" to the caller (the entity requesting the deployment of SDNAS as a service) for subsequent status updates of the deployment. In step (4), the Bedrock Controller executes the Ansible script to deploy the SDNAS cluster management and NASlib (network attached repository) components on the corresponding nodes. In step (5), the Ansible script deploys the components. In step (6), the Bedrock Controller polls the Ansible output and "stores" important final state information and simple status fields for each host / node and component deployed. In some embodiments, this means that no overall status is required. In step (7), a GET request with "task id" from the northbound API retrieves the cached state information from step 6 above until all messages associated with the task id are in a final state.

[0189] Reference again Fig.13 and Fig.14 , now in Fig.13 Block 1350 and Fig.14 The above workflow has returned detailed status and the provisioned pool capacity ( Fig.14 At step (9)), a file share such as an NFS (Network File System) file share may be sent to the stack management service 330 (i.e., in this example embodiment, this is the entry point for the request). Fig.13 Block 1350, Fig.18 The sequence then runs to provision the file share via the SDNAS management entry point.

[0190] Fig.18 According to at least one embodiment Fig.12 The architecture and Fig.13 A schematic diagram and exemplary sequence of actions occurring in a simplified flow chart of FIG. 10, showing the components that come into play during the creation of a Network File Share (NFS). Fig.18 In step (1), use POST to request a new NFS share on the SDNAS management API (such as Fig.18 In step (2), the NFS share is displayed on the client (shown as Fig.18 bottom screenshot in 1804).

[0191] In another embodiment, using Figure 3-Figure 8 The complete architecture shown may implement additional features. For example, in some embodiments, in addition to Figure 3-Figure 18 In addition to the above functions implemented in the embodiments, the described systems, methods and devices can provide users with more customized automatic settings, with more templates suitable for the installation, control, management, deployment, configuration, etc. of storage assets (such as software-defined storage assets) adjusted based on the characteristics and features of storage assets to meet specific user requirements.

[0192] For example, in certain embodiments, the described systems, methods, and apparatus are capable of returning a list of software defined storage (SDS) stack configurations available to a user given the user's inventory and the capabilities of the inventory. In certain embodiments, being able to return a list of available SDS stack configurations given a user's inventories and inventory capabilities establishes a basis for being able to describe software defined storage stack installability based on a number of other factors, including, but not limited to, server capabilities (memory, disk, CPU, network), licenses, existing stack installations, tenant restrictions, and network topology and configuration.

[0193] Fig.19 is an illustrative diagram 1900 showing components that function during host registration and requesting a deployment template, wherein the applicable architecture is as discussed above, according to at least one embodiment. Figure 3-Figure 5 architecture. Fig. 20 For illustrative purposes, a simplified flowchart is shown in which at least Figure 3-Figure 8 The actions that occur during host registration and request for deployment templates in the . Figure 5 Compared to the architecture 500 shown, Fig.19 The architecture shown is a "disassembled" architecture to better illustrate the actions taking place.

[0194] refer to Fig.19 , Fig.19 The M&O layer and surface area layer 302 are illustrated and include platform REST endpoints 502, native services 504, compute controllers 509, API gateways 514, deployment template management services (DTMS) 1906, multiple templates 1914a-1914e within the deployment template management service, open source services 1906, policy managers 1908, template CRUD services 1910, template definition files 1912, and stack deployment orchestration controllers 532. These components are discussed further below or are equivalent to the components previously described with respect to Figure 4-Figure 5or as described elsewhere herein.

[0195] Requests to the platform REST endpoint 502 will be served by multiple services running independently. The API gateway 514 multiplexes incoming uniform resource locators (URLs) to services that can serve requests in the platform. In some embodiments, several identical services may exist within the cloud native ecosystem 323. The user interface (UI) 1902 communicates the request to the API gateway 415. In some embodiments, the UI 1902 is implemented as a basic UI using the Vaadin framework, wherein the Vaadin framework is a Java UI framework and library that simplifies web application development, wherein the framework's code is written in Java and executed on the server's Java virtual machine (JVM) and the UI is rendered in the browser as Hypertext Markup Version 5 (HTML5). The Vaadin framework is an open source product provided by Vaadin of Turku, Finland. In some embodiments, the UI 1902 is implemented via Ansible Tower / AWX integration, wherein Ansible Tower is a web-based interface for managing Ansible, thereby providing a UI for managing rapid deployment and monitoring configuration.

[0196] In some embodiments, native service 504 is similar to the one described above with respect to Figure 5 The native services 504 described above include all services implemented by the VIKI platform, except Ansible AWX / Tower. The compute controller 509 exposes a REST interface for adding servers to the inventory. This inventory is used to deploy and install the software-defined storage stack.

[0197] The stack deployment orchestration controller 532 has Figure 5 The stack deployment orchestration controller in has similar functionality and exposes a REST interface that deploys the storage stack on the server and configures and guides the stack based on user input or best practice templates stored in the policy manager 1908 and / or template definition files 1912.

[0198] In certain embodiments, Fig.19 The controller service in is the VIKI controller service. Fig.19 The platform REST endpoint 502 at the top includes everything related to requests to install, configure, or register SDS assets or other storage all the way to consumption requests, i.e., everything that the management system must provide, as previously described. In some embodiments, native services 504 are all services implemented through the VIKI platform, with the exception of AWX, as previously described.

[0199] The template definition file 1912 is a file that describes the configuration of the storage stack and the prerequisites required to deploy the storage stack configuration. The Template Create / Read / Update / Delete (CRUD) Service 1910 records the template definition and inventory information, providing the template definition and inventory information to the DTMS 1906 when requested, checks whether the template definition has changed, and stores the template definition file in memory. The DTMS 1906 collates the data, template, and inventory information and provides the information to the Policy Manager 1908, receives a set of results from the Policy Manager 1908, filters the results based on the query parameters, and provides the response to the caller / user.

[0200] refer to Fig.19 and Fig. 20 Flowchart 2000, in Fig.19 In the architecture 1900 of FIG. 1 , a client / user may register / identify a LINUX host for use with VIKI (block 2010), request a deployment template that the client / user may execute (block 2020), obtain a detailed list of templates that may be deployed (block 2030), and obtain reasons why the client / user may not deploy other templates (block 2040). This will be further explained herein.

[0201] Fig.21 is an illustrative simplified flow chart according to at least one embodiment, showing a process for responding to at least Figure 3-Figure 8 and Fig.19 The architecture provides a method for supplying a request to a store, the request including an indication of which configurations are available in an inventory and the capabilities of the inventory. Fig.21 At a high level, this includes Figure 19-Figure 25 The overall action shown in , with details of the action in some blocks. Figure 19-Figure 25 Reference is now made to Fig.19 and Fig.21 , the method first predefines the templates and rules to be applied to the software-defined storage assets (block 2110). The predefinement includes creating a template definition file 1912 describing the configuration of the storage stack and the prerequisites for deploying the configuration (block 2112), and creating a rule set 1909 for the policy manager 1908 (block 2114). The rule set 1909 can be executed against the inventory and configuration information to determine whether the template can be implemented.

[0202] The Template CRUD service 1910 is started (block 2120), which includes loading the Template definition file 1912 and making it resident in memory (block 2122). The Template definition file describes the prerequisites for the storage stack configuration and deployment configuration (block 2122). As part of this block, the Template CRUD service 1910 checks to see if the definition (Template definition file 1912) changed when GET was called and a certain amount of time has passed. (optional) (block 2124). A request for a template is made (block 2130). In some embodiments, this block involves executing Fig. 22 The actions listed in the sequence in Fig. 22 According to at least one embodiment Fig.21 2200 and an exemplary sequence of actions for implementing a request template method. Fig. 22 , the numbers in the figure correspond to components in the architecture that help perform the corresponding numbered steps.

[0203] Brief reference Fig.19 and Fig. 22 In the enumerated sequence, in step (1), a user submits a request to see which possible storage stacks can be deployed to the Deployment Template Management Service (DTMS) 1906. The user request may provide query parameters to constrain the query. In step (2), the DTMS 1906 requests the template definition from the Template CRUD Service 1910. In step (3), the DTMS 1906 requests the template definition from the Inventory CRUD Service 2202 ( Fig. 22 ) requests inventory information. In step (4), DTMS1906 collects data and templates and sends the data and templates to policy manager 1908. In step (5), the policy manager service applies the input and rule set to implement a result set returned to DTMS1906. In step (6), DTMS1906 filters the results based on the query parameters sent in step (1). In step (7), DTMS1906 formulates a response body with an OK status code to the caller. In step (8), the user experience (UX experience) or user interface shows the template based on the query. Fig.21 These templates are returned in block 2130 of .

[0204] Reference again Fig.19 and Fig.21 , in block 2140, obtaining a list of installed storage services (block 2140), and Fig.23 According to at least one embodiment Fig.21 An illustrative diagram and exemplary sequence 2300 of actions for implementing a method of obtaining an installed storage service. Fig.23and the sequence shown therein, in step (1), a request is submitted via the platform REST endpoint 502 to obtain a list of existing storage stacks installed in the VIKI platform (GET / v1 / storage-service). In step (2), for each storage stack, the storage deployment service sends a request to the compute controller to obtain a list of nodes with the storage stack that has this universally unique identifier (UUID) installed on it. (GET / v1 / compute / storage-service / {uuid}). In step (3), the compute controller returns a list of nodes with components of the installed storage stack to the REST endpoint. In step (4), the SDS compiles a response and sends it back to the consumer. Fig.23 Included is an example of a payload 2302 that may be returned to the client in step (4). The example payload (which is for illustration purposes only) includes a list of storage services, including the UUD, type (e.g., block, file, etc.), stack name (e.g., VxFlex OS, SDNAS), and a list of nodes on which the stack is installed.

[0205] Reference again Fig.19 and Fig.21 After obtaining the list of installed storage services (block 2140), the method obtains real-time installed storage service details (block 2140), and Fig.24 According to at least one embodiment Fig.21 An illustrative diagram and exemplary sequence 2400 of actions for implementing a method for obtaining real-time installed storage service details. Fig.24 , in step (1), a request is submitted to the storage deployment service 2402 to obtain details about an existing storage stack installed in the inventory (GET / v1 / storage-service / {uuid} / details). In step (2), the storage deployment service 2402 issues a REST call to the stack coordinator (SC) service 2404 (GET / v1 / storage-service / {uuid} / details). In step (3), the SC service 2404 issues a REST call to the service registry 2408 (GET / v1 / service-registry / {type}) to obtain the type of controller access (e.g., block, file). In step (4), the SC service 2408 issues a REST call to the registered controller 2406 (GET / v1 / storage-service / <type> / {uuid} / details). In step (5), the controller 2406 issues a REST call (GET / v1 / service-registry / {type} / {stack}) to the service registry 2408 for access by the stack implementation (e.g., VxFlex OS, etc.). In step (6), the controller issues a REST call (GET / v1 / storage-service / <type> / <stack> / {uuid} / details). In step (7), the southbound driver calls the stack management interface to get specific details. In step (8), the storage deployment service 2402 compiles the stack details and returns a response to the user.

[0206] Reference again Fig.19 and Fig.21 Once the architecture has the template, installed storage service details, and live installed storage service details, the storage stack may be deployed (block 2160). Fig.25 According to at least one embodiment Fig.21 An illustrative diagram and exemplary sequence 2500 of actions for implementing a method of deploying a storage stack. Fig.25 , in step (1), a request is submitted to the Storage Deployment Service to deploy the storage stack (POST / v1 / storage-service / change-request). In step (2), the Storage Deployment Service sends a request to the Task Service to create a new task. (POST / v1 / tasks). In step (3), the Storage Deployment Service issues a REST call to the Stack Coordinator Orchestration (SCO) service (POST / v1 / storage-service / change-request). In step (4), the task is returned to the user so that the user can monitor the deployment execution. In step (5), the SCO issues a REST call to the service registry (GET / v1 / service-registry / {type}) to obtain the type of controller to be accessed (e.g., block, file). In step (6), the SCO issues a REST call to the correct controller (POST / v1 / storage-service / <type>In step (7), the controller issues a REST call to the service registry (GET / v1 / service-registry / {type} / {stack}) to obtain the stack implementation to be accessed (e.g., VxFlex OS, SDNAS, etc.).

[0207] In step (8), the controller issues a REST call (POST / v1 / storage-service / ) to the stack implementation controller which will call the southbound driver to deploy the storage stack. <type> / <stack> / deployment). In step (9), the stack controller analyzes the resources and orchestrates the operations to be performed. In step (10), the orchestration component requests the deployment controller from the service registry. In step (11), the orchestration component issues a REST call to the deployment controller. In step (12), the deployment controller calls Bedrock and polls for tasks. In step (13), when a task is completed, a message is put on the bus. In step (14), the orchestration component subscribes to messages from the bus about ongoing tasks. In step (15), the SCO subscribes to the message bus and consumes the task status. In step (16), the SCO marks the task with the appropriate status (completed or failed). This also makes Fig.21 The method ends.

[0208] As the above embodiments demonstrate, a single, unified management and orchestration (M&O) layer or pane with the above architecture can be configured to automatically install, configure, deploy, and consume storage assets from a variety of different vendors, and can understand what is deployed and connect together to service storage requests, and accelerate or decelerate the software-defined storage stack due to predictive analytics, policies, and usage at the request of IT administrators.

[0209] IT departments use software-defined storage platforms to reduce the excessive capital and operating expenses associated with deploying and maintaining dedicated equipment (storage arrays) associated with the hardware. This was initially achieved by standardizing and purchasing a group of commodity computing servers, putting them together in a rack with simple networking, and installing a software-defined storage stack that provides block, file, object or stream storage through IP-based protocols.

[0210] However, challenges can arise when trying to maximize the use of infrastructure across many storage types. One attempted solution is to lock specific servers in a rack into specific roles. For example, have servers 1-4 provide block storage, servers 5-8 provide object storage, and so on. These worksheet-based configuration practices tend to lead to stranded capacity, and stranded capacity on one storage type means potential lack of capacity for another storage type in the future.

[0211] At least some of the foregoing embodiments illustrate and describe unified configuration management, including deployment, installation, configuration, and consumption of software-defined storage assets. At least some of these embodiments provide a way to canonically and manually identify these roles and apply policies to an inventory of hardware to automatically install a software-defined storage stack for future consumption.

[0212] In further aspects of these embodiments, additional embodiments are provided that enable the canonical manual deployment steps to be completely removed. At least some embodiments provide a mechanism where policies and predictive analytics are applied to consumption requests (requests to create volumes, buckets, file shares, etc.) from users and to available hardware inventory to dynamically install and configure storage software. This allows customers to start with just the hardware inventory and provides customers with consumption capabilities where management software guides the storage stack and makes configuration changes as requests are passed into the system. In some embodiments, this can provide consumption-based elastic deployment and reconfiguration of software-defined storage.

[0213] At least some embodiments provide an automatic mechanism to elastically deploy storage stacks suitable for requests to consume storage. At least some embodiments provide a way to responsibly shrink (reduce size) or uninstall storage stacks when they are no longer in use. At least some embodiments expand existing storage stacks to additional nodes or drives to predictively increase capacity. At least some embodiments provide controls for IT administrators to control operations based on the type of operation. (Install vs. expand vs. shrink). At least some embodiments enable automation of installation, deployment, and configuration of storage stacks, as previously described herein. At least some embodiments elastically expand and / or shrink storage usage policies, predictive analysis (past usage predicts future results). At least some embodiments perform installation / deployment as part of a consumption request.

[0214] In certain embodiments, a configuration is provided in which an IT administrator identifies a pool or inventory of hardware (e.g., servers) that can be consumed by a software-defined storage platform. In some embodiments, the servers may have only a bare-bones operating system on them that is supported. In contrast to a manually configured rack of servers where a specified set of storage types are installed on a user-defined fixed group of nodes before an end user issues a request to consume storage (create a volume, share, or bucket), in certain embodiments, this configuration instead starts with no storage stack installed before allowing a consumption request.

[0215] Temporary reference Figure 3 and Figure 4 , at least some embodiments include a component within the M&O management layer / plane referred to as a resilient configuration service 328 that operates to implement at least the Figure 26-28 Associated functions.

[0216] Consider a scenario where a customer / client is existing in a computer-based or information processing ecosystem (such as a cloud ecosystem) in a client-server configuration, but the customer / client does not deploy a single storage stack anywhere in the ecosystem (yet). However, the customer has access to an architecture such as the M&O plane / layer described above. The customer / client runs an application such as an ORACLE database on the server, and the application hypothetically requires 10GB of storage.

[0217] With respect to the embodiments described herein, a client / client is able to request a 10GB volume from / for a server (unlike other use cases described herein), and in certain embodiments, the architecture provided herein should be able to discern what type of storage is actually needed, what resources can accomplish this functionality, based on the request for 10GB of storage, and also be able to roll out and actually install, deploy, and configure a storage stack so that the request for 10GB can be satisfied, and finally invoke the management software of the storage stack to actually create the volume. Thus, it is possible to start from "nothing" (without accessing any data storage), so one can start from scratch and then actually have the storage stack automatically deployed and ready. This also provides an important advantage to the client / client, because the client / client may not have to pay for the additional storage (the additional 10GB of data storage) until it actually uses the additional storage, and this advantage also applies in cloud storage configurations.

[0218] In at least some embodiments, this mechanism allows an IT administrator to identify a pool or inventory of hardware (servers) that can be consumed by a software-defined storage platform. A server may have only a supported bare operating system on it. In contrast to a manually configured rack of servers, where a specified set of storage types are installed on a user-defined fixed group of nodes before an end user issues a request to consume storage (create a volume, share, or bucket), this mechanism instead starts with no storage stack installed before allowing the consumption request, and then automatically provides the storage stack in response to certain instructions. Therefore, this can advantageously enable a "just-in-time" installation and deployment of storage assets to be performed, which makes costs and resources more efficient because storage assets are not installed or deployed until the storage assets are actually needed for consumption, thereby saving clients and customers money by not having to install and deploy storage assets until they are needed.

[0219] Fig.26 According to at least one embodiment, for at least Figure 2-Figure 19 A simplified flowchart of an exemplary method 2600 for identifying a hardware pool that can be consumed by an SDS platform and for elastically expanding / contracting an existing storage stack. This flowchart 2600 will be discussed with respect to an illustrative example in which a management platform (e.g., at least Figure 3-Figure 19 In some embodiments, the SDS M&O 102 is installed, but this example is not limiting and is provided only to better understand how the process steps work. In some embodiments, the M&O is installed in a cloud native orchestration ecosystem such as Kubernetes, Pivotal Cloud Foundry, or Pivotal Container Service (PKS). Fig.26 Before the method begins, the IT administrator identifies a rack of servers for use by the M&O 102 management platform. In some embodiments, potentially available servers may be identified according to the description of Figure 3-Figure 26 The described architecture and method (including a unified interface for configuration management) are automatically recognized (and provisioned, deployed, configured, etc.).

[0220] In at least one embodiment, Fig.26 The method is implemented in the following configuration: the M&O management platform 102 includes a fixed inventory to match incoming requests and supported storage topologies. Although the use of a fixed inventory is not limiting, having a fixed inventory improves the efficiency of analyzing assets to assess the ability of assets to meet customer needs. In one example, the fixed inventory that matches incoming storage requests and supported storage topologies may include the following exemplary features, but this is certainly not limiting:

[0221] Storage type: block, file, object, stream

[0222] Storage technologies: VxFlex OS, PowerMax, Unity, ECS Flex, IsilonSD, Nautilus, AWSS3, SDNAS (and the corresponding storage types they support)

[0223] Storage protocols: iSCSI, FC, SDC (VxFlex OS), NAS

[0224] Reference now Fig.26 , in block 2605, a request is received from a user (e.g., at management REST endpoint 316) using a predefined interface (e.g., user interface (UI) 314 as described above) to request a block volume of predetermined size "X" from a predetermined host "Y". In this example, the following parameters are assumed:

[0225] User: asics

[0226] Request Type: Supply Storage

[0227] ·storage:

[0228] Type: Block

[0229] Protocol: iSCSI

[0230] Size: 10GB

[0231] Host: dbhost1.localhost, dbhost2.localhost

[0232] In block 2010, the consumption request is parsed to obtain information about: specific tenant / host; user; request type; storage type; storage protocol. Thus, the management platform 102 interprets the received consumption request as a specific type for a specific tenant and host. Elastic Deployment Service 328 ( Figure 3 and Figure 4 ) is responsible for matching consumption requests with existing storage technology installations / deployments, or expanding the capacity of existing deployments or installing new storage technology deployments to service requests. In some embodiments, the following request fields affect the elastic deployment service 328: user, request type, storage type, and storage protocol. As part of this analysis, in some embodiments, execution Figure 27-Figure 29 The process (block 2611) and a determination is made as to what type of request it is (block 2612) and whether the storage technology can provide the required storage protocol (block 2613). This will be explained further below.

[0233] Brief reference Fig. 27 , Fig. 27 According to at least one embodiment Fig.28 Analysis of flowchart of a simplified flowchart 2700 of a method for consuming requests for user information. In block 2710, user fields are checked for data isolation based on user / tenant, service layer, and policy. Based on the policy and service layer paid by the customer, the elastic deployment service 328 must ensure that the data of this user is isolated from the data of other users. ("Tenant" can be replaced with user here, because all organizations / companies may want to isolate their data from other companies' data). Therefore, at block 2720, a check is made to see whether the customer user data is isolated from other user data. If the answer at block 2720 is "no" (i.e., the customer user data does not have to be isolated), then at block 2722, a check is made to see whether the user already has one or more storage stacks that already coexist with other customers. If the answer at block 2722 is "yes", the elastic deployment service 328 may treat the existing storage stack as block 2605 ( Fig.16 ). An example of when the answer at block 2722 may be "yes" is a situation where a user or tenant can pay for a less expensive policy, where the user's or tenant's storage stack coexists with other customers. In this case, if the storage stack already exists, it can be considered available for this request.

[0234] If the answer at block 2722 is "no" (meaning that the data does not need to be isolated, but the user does not yet have a storage stack deployed), the elastic deployment service 328 must install / deploy new storage (e.g., using the methods described herein for Figure 3-Figure 25 methods already described, etc.)

[0235] Referring again to block 2720, if the answer at block 2720 is "yes" (customer data must be isolated), then, depending on the policy and service tier for which the customer paid, the elastic deployment service 328 must ensure that this user's data is isolated from other users' data. (In some embodiments, "tenant" can be replaced with user here, because all organizations / companies may want their data to be isolated from other companies' data). Therefore, if the answer at block 2720 is "yes", a check is made to see if the user already has a storage stack of the same type (e.g., "block" type) deployed to the same (storage) network (block 2730). If the answer at block 2730 is "yes", the elastic deployment service 328 may treat the existing storage stack as Fig.26 Candidates for a request for block 2605 (eg, a block volume request for a 10GB volume) (block 2724).

[0236] If the answer at block 2730 is “no”, then if the user does not have a storage stack of the same type already deployed, the elastic deployment service 328 knows that it needs to install / deploy a new storage stack (block 2740), e.g., using Figure 3-Figure 25 Then, the method returns to Fig.26 Block 2612 (block 2750).

[0237] Reference again Fig.26 At block 2612, the request type is checked to see if the request type is one of: provision, expand, remove / delete, connect, or remove connectivity (block 2612). For example, the request may be a request to create (provision) storage for a host / cluster, expand existing storage, or remove / delete storage. The request may also be for connecting existing storage to an additional host / cluster or for removing connectivity of storage to or from a host.

[0238] In block 2613, a check is made to determine if any available storage technology can provide the required storage protocol (the required protocol is named in the request). The storage protocol (e.g., one of FC, iSCSI, SDC [VxFlex OS], NAS) drives the policy within the management platform because each storage technology can only provide certain protocols. For example, if the request at block 2605 specifies "iSCSI" as the protocol, and the list of available storage technologies is: only SDNAS and Isilon, then block 2613 may choose to deploy Isilon because Dell EMC SDNAS cannot provide the iSCSI protocol. Processing then flows from block 2613 to block 2615. It should also be noted that in Fig.26 The order in which the fields are checked when parsing the consumption request in block 2610 is illustrative and not limiting; the fields may be checked in any order.

[0239] Referring to blocks 2615 and 2620, the current SDS is analyzed to determine if there is any storage of the requested type. The management platform 328 uses this field to determine if a deployed storage stack matching the requested type exists. The management platform has an inventory of supported storage types and the technologies that can service those types. As an illustrative (but non-limiting) example, in one embodiment, the inventory list includes:

[0240] Block: VxFlex OS (default), PowerMax, Unity

[0241] File: Dell EMC SDNAS [Software Defined NAS] (default), Isilon SD

[0242] Target: Dell EMC ECS (default), AWS S3

[0243] Stream: Dell EMC Nautilus

[0244] The elastic deployment service 328 is configured to know which storage technology to use via two mechanisms: if a non-default storage stack is not deployed, the default storage stack technology is used, but if a non-default storage stack technology is not identified in the request payload, the default storage stack technology is used. Otherwise, the policy will dictate the non-default storage technology to use for the request. Fig.26 In block 2615, the method temporarily jumps to Fig.28 (Block 2616), Fig.28 A simplified flow chart 2800 of a method for parsing a consumption request for storage type information according to at least one embodiment.

[0245] Temporary reference Fig.28 ,from Fig.26 Starting from block 2616, Fig.28 Processing in starts at block 2810. Analyze storage type (block 2820) to determine whether there is a deployed storage stack that matches the requested type (e.g., one of block type, file type, object type, and stream type). In order to determine what storage technology to use, elastic deployment service 328 checks whether any non-default storage stacks have been deployed (block 2830). If the answer is "no", then the default storage stack technology (block 232) is used. If the answer at block 2830 is "yes", i.e., (already) deployed non-default storage stacks, then check to see if the request payload has identified the non-default storage stack to be used (block 2840). If the answer at block 2840 is "no", this means that the request payload does not seem to specify a specific storage technology, so the default storage stack technology is allowed to be used for the system (block 2832).

[0246] If at block 2840, the answer is "yes", i.e., the request payload includes a non-default storage technology therein, then the request should be configured to use the specified non-default storage technology (block 2850). Then, processing returns to block 2860. Fig.26 Block 2616.

[0247] Reference again Fig.26 , at block 2616, a result is returned indicating whether storage of the requested type exists (blocks 2615 and 2620). Therefore, in block 2620, if storage of the requested type exists (the answer at block 2620 is yes), the storage stack is reconfigured to satisfy the request (block 2637), and optionally, a notification that the new stack has been deployed may be sent (block 2639). Deployment may occur automatically, depending on IT administrator preferences. In certain embodiments, an optional preview / pause step (between blocks 2620 and 2637) may occur, where an entity such as an end user or IT administrator may give final approval prior to deployment.

[0248] Referring again to block 2620, if storage of the requested type does not exist (the answer at block 2620 is "no"), then in certain embodiments, the elastic deployment service 328 described herein is configured to analyze the current landscape of software-defined storage assets and has determined that storage of the requested type does not exist. In certain embodiments, to assist in deploying, configuring, and provisioning the appropriate or suitable type of storage, in block 2625, the elastic deployment service considers the following to identify a suitable template (a collection of deployment attributes) of the type block to be applied to the inventory:

[0249] The requested size (some sizes cannot be satisfied by some hosts in the inventory)

[0250] The host that requested it (hosts with previous requests generally prefer subsequent storage from the same source)

[0251] Past performance of such requests (scores) and subscribed deployments (analytics to predict future performance of operations)

[0252] Performance / resources of inventory (other storage stacks may be in use, and depending on policy and conditions, you may want to avoid installing a new stack on top of other storage stacks)

[0253] • Compatibility of the storage stack (some storage stacks are adjacent to other storage stacks, however, some storage stacks are not) to determine whether multiple stacks on a single server are allowed.

[0254] In block 2630, the elastic deployment service 328 receives the service analysis results according to the preferences. The elastic deployment service analysis generates various supported possibilities (depending on the IT administrator's preferences). If the request can be automatically satisfied (the answer at block 26350 is "yes"), then without administrator intervention, the storage stack is deployed (e.g., if the storage is available, but not yet deployed) or the storage stack is reconfigured (e.g., the storage is deployed, but the allocation of the storage must be changed in some aspect to satisfy the request) to satisfy the request (block 2637). Optionally, if notification is required, a notification can be sent (e.g., to a user, IT administrator, or other entity for receiving notification) to warn that a new stack has been deployed (block 2639).

[0255] If the request cannot be automatically satisfied (i.e., the answer at block 2635 is "no"), it may be that the configuration is such that some manual action (such as approval or review of the change, or guided execution) must occur before the request is executed. In block 2640, a check is made to see if the request may be satisfied with user input (such as review or guided execution). If the answer at block 2640 is yes, then the elastic deployment service 328 is configured to provide the IT administrator with a "menu" of options to guide execution, thereby providing better prescriptive automation of future requests.

[0256] For example, the elastic deployment service 328 may provide a recommended template that satisfies the received request (block 2642), and deploy or reconfigure a new storage stack according to the guided user selections (block 2644).

[0257] Another result at block 2640 is a "no" answer, because the request may not satisfy the user input, so that there is no possible way to satisfy the request, resulting in the request failing (block 2645). Failure can occur for a variety of reasons, including but not limited to, the inventory does not exist, the inventory is insufficient, or the resources of the inventory have been allocated to other storage types or stacks.

[0258] Because about Figure 3 , Figure 4 Targeted at Figure 26-Figure 28 The above description shows that the elastic deployment service 328 accepts consumption requests for creation, deletion, and expansion of storage in some embodiments and determines the best practices provided. In some embodiments, an advantage of at least some embodiments of the architecture and method described herein is to help establish a minimum deployment, because the template-based deployment, installation, and configuration of the storage stack take into account potential future uses to reduce the overall time-consuming installation process of the storage stack. In addition, another advantage of at least some embodiments described herein is the ability to maximize capacity, because at least some embodiments of the elastic deployment service 328 help prevent shelving capacity by allocating only a portion of the hard drive space of the server when possible. In addition, another advantage of at least some embodiments described herein is to help maximize computing, because elastic deployment combines storage stacks on a single server to maximize computing resources. Moreover, another advantage of at least some embodiments described herein is to increase the ability to reallocate resources, because at least some embodiments of the elastic deployment described herein can move the storage stack from one node to another in order to meet service level objectives.

[0259] Testing of at least some embodiments described herein involved case studies in which block and file storage stack components were successfully installed (deployed, configured, etc.) on the same node (with unique local storage volumes allocated for each stack).

[0260] In the above-mentioned flow charts and sequence diagrams, certain elements (e.g., rectangular elements, diamond elements, and statements preceded by numbers in circles) indicated as "processing blocks" herein represent computer software instructions or groups of instructions. Alternatively, a processing block may represent steps performed by a functionally identical circuit (such as a digital signal processor (DSP) circuit or an application-specific integrated circuit (ASIC)). The flow charts and sequence diagrams do not depict the syntax of any particular programming language, but rather illustrate the functional information required by a person skilled in the art to manufacture a circuit or generate computer software to perform the processing required by a particular device. It should be noted that for the sake of clarity, many routine program elements, such as initialization of loops and variables and the use of temporary variables, may be omitted. The specific sequence of the blocks described is illustrative only and may be changed without departing from the spirit of the concepts, structures, and techniques sought to be protected herein. Therefore, unless otherwise specified, the blocks described below are unordered, which means that, when possible, the functions represented by the blocks may be performed in any convenient or desired order.

[0261] Furthermore, the processes and operations described herein may be performed by a computer specially configured for the desired purposes or by a general-purpose computer specially configured for the desired purposes through another computer program stored in a computer-readable storage medium or in a memory.

[0262] Fig.29 According to at least some embodiments, it is possible to implement Figure 1-Figure 28 Simplified block diagrams of devices that illustrate at least a portion of systems, architectures, sequence diagrams, and methods.

[0263] like Fig.29 As shown, the computer 2900 may include a processor 2902, a volatile memory 2904 (e.g., RAM), a non-volatile memory 2906 (e.g., one or more hard disk drives (HDDs), one or more solid state drives (SSDs), such as flash drives, one or more hybrid magnetic and solid state drives, and / or one or more virtual storage volumes, such as cloud storage; or a combination of physical storage volumes and virtual storage volumes), a graphical user interface (GUI) 2910 (e.g., a touch screen, a display, etc.), and an input and / or output (I / O) device 2908 (e.g., a mouse, a keyboard, etc.). The non-volatile memory 2904 stores, for example, log data 2904a, metadata 2904b, and a pre-allocated memory area 2904c. In some embodiments, the non-volatile memory 2906 may include an operating system 2914 and computer instructions 2912 and data 2916. In some embodiments, the computer instructions 2912 are configured to provide several subsystems, including a routing subsystem 2912a, a control subsystem 2912b, a data subsystem 2912c, and a write cache 2912d. In some embodiments, the computer instructions 2912 are executed by the processor / CPU 2902 out of the volatile memory 2904 to perform Figure 2-Figure 8 The program code may also be applied to data inputted or received from I / O device 2908 using input device or GUI 2910.

[0264] Figure 1-Figure 28 The systems, architectures, sequences, flow charts, and processes described herein are not limited to use with the hardware and software described and illustrated herein, and may be applied in any computing or processing environment and with any type of machine or group of machines that may be capable of running a computer program. The processes described herein may be implemented in hardware, software, or a combination of both. The logic for executing the method may be embodied as Fig.29 Part of the system described in, which can be used to perform reference such as Figure 1-Figure 28 The methods described in the embodiments shown. The processes and systems described herein are not limited to the specific embodiments described, nor are the processes and systems particularly limited to the specific processing order shown. Specifically, any of the blocks of the processes may be reordered, combined or removed, executed in parallel or serially as necessary to achieve the results stated herein.

[0265] Processor 2902 may be implemented by one or more programmable processors that execute one or more computer programs to perform the functions of the system. As used herein, the term "processor" describes an electronic circuit that performs a function, operation, or sequence of operations. Functions, operations, or sequences of operations may be hard-coded into an electronic circuit, or soft-coded by means of instructions stored in a memory device. A "processor" may use digital values ​​or use analog signals to perform a sequence of functions, operations, or operations. In some embodiments, a "processor" may be embodied in one or more application specific integrated circuits (ASICs). In some embodiments, a "processor" may be embodied in one or more microprocessors with associated program memory. In some embodiments, a "processor" may be embodied in one or more discrete electronic circuits. A "processor" may be analog, digital, or mixed signal. In some embodiments, a "processor" may be one or more physical processors or one or more "virtual" (e.g., remotely located or "cloud") processors.

[0266] The various functions of the circuit elements may also be implemented as processing blocks in a software program. Such software may be used, for example, in one or more digital signal processors, microcontrollers, or general-purpose computers. The described embodiments may be implemented in hardware, a combination of hardware and software, software, or software executed by one or more physical or virtual processors.

[0267] Some embodiments may be implemented in the form of methods and apparatus for practicing those methods. The described embodiments may also be implemented in the form of program code, for example, stored in a storage medium, loaded into and / or executed by a machine, or transmitted through some transmission medium or carrier (such as by wire or cable, by optical fiber, or via electromagnetic radiation). Non-transitory machine-readable media may include, but are not limited to, tangible media such as magnetic recording media, including hard drives, floppy disks, and magnetic tape media; optical recording media, including compact disks (CDs) and digital versatile disks (DVDs); solid-state memory, such as flash memory, hybrid magnetic and solid-state memory, non-volatile memory, volatile memory, etc., but essentially excluding temporary signals. When embodied in a non-transitory machine-readable medium and program code is loaded into and executed by a machine such as a computer, the machine becomes an apparatus for practicing the methods.

[0268] When implemented on one or more processing devices, the program code segments combine with a processor to provide a unique device that operates similarly to a specific logic circuit. Such processing devices may include, for example, a general-purpose microprocessor, a digital signal processor (DSP), a reduced instruction set computer (RISC), a complex instruction set computer (CISC), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic array (PLA), a microcontroller, an embedded controller, a multi-core processor, and / or other devices, including combinations of one or more of the above. The described embodiments may also be implemented in the form of a bit stream or other sequence of signal values, the signal values ​​are transmitted through a medium electrically or optically, magnetic field changes are stored in a magnetic recording medium, etc., generated using methods and / or devices as recited in the claims.

[0269] For example, when program code is loaded into a machine and executed by the machine, such as Fig.29 The computer of the present invention is transformed into a device for practicing the present invention. When implemented on one or more general-purpose processors, the program code combines with such processors to provide a unique device that operates similarly to specific logic circuits. Thus, a general-purpose digital machine can be transformed into a special-purpose digital machine. Fig.29 Program logic 2924 is shown as embodied on a computer readable medium 2920 as shown, and wherein the logic is encoded in a computer executable code configured to perform the reservation service process of the present invention and thereby form a computer program product 2922. The logic may be the same logic loaded on a memory on a processor. The program logic may also be embodied in a software module, embodied as a module, or embodied as a hardware module. The processor may be a virtual processor or a physical processor. The logic may be distributed across several processors or virtual processors to execute the logic.

[0270] In some embodiments, the storage medium may be a physical or logical device. In some embodiments, the storage medium may be composed of a physical or logical device. In some embodiments, the storage medium may be mapped across multiple physical and / or logical devices. In some embodiments, the storage medium may exist in a virtualized environment. In some embodiments, the processor may be a virtual or physical embodiment. In some embodiments, the logic may be executed across one or more physical or virtual processors.

[0271] For purposes of illustrating the present embodiments, the disclosed embodiments are described in a specific configuration and embodied using a particular logical arrangement, but those skilled in the art will appreciate that the apparatus is not limited to a specific configuration, but is limited only by the claims included with this specification. In addition, it is expected that during the life of a patent growing from this application, many related technologies will be developed, and the scope of the corresponding term is intended to include all such new technologies a priori.

[0272] The terms "comprising," "including," "having," and variations of said terms mean at least "including but not limited to." As used herein, the singular forms "a," "an," "the," and "said" include plural references unless the context clearly dictates otherwise. The various elements described in the context of a single embodiment may also be provided separately or in any suitable subcombination. It will be further understood that various changes in the details, materials, and arrangements of the parts that have been described and illustrated herein may be made by those skilled in the art without departing from the scope of the following claims.< / stack> < / type> < / type> < / stack> < / type> < / type>

Claims

1. A computer-implemented method for deploying storage assets, the method include: a consumption request for accessing a consumed storage asset, the consumption request comprising a stack parameter and a resource characteristic parameter, wherein the stack parameter specifies at least one type of storage asset requested and the resource characteristic parameter specifies at least one functional capability required of the storage asset; determining, based on the stack parameters, a set of one or more first storage assets capable of satisfying the consumption request; For each undeployed first storage asset in the collection, generating a first workflow, the first workflow being configured to deploy the corresponding undeployed first storage asset in the collection; For each second storage asset in the group that lacks the resource characteristic parameter, generating a second workflow, the second workflow being configured to implement the resource characteristic in the corresponding second storage asset; as well as The collection of storage assets is configured to satisfy the consumption request, wherein the configuring includes running the first workflow and the second workflow.

2. The computer-implemented method of claim 1, further comprising: include: determining that a third storage asset resource can be created to satisfy the consumption request; as well as A third workflow is generated, the third workflow being configured to create and deploy the third storage asset resource.

3. The computer-implemented method of claim 2, wherein the third storage asset resource is formed within at least one of the first storage asset and the second storage asset.

4. The computer-implemented method of claim 1, wherein the consumption request is for provisioning a file share, and wherein the computer-implemented method further include: Contacting one or more LINUX hosts to determine and store host information; Registering the one or more LINUX hosts; Requesting to deploy a storage asset of a predetermined type as a corresponding service; as well as Create the corresponding file share.

5. The computer-implemented method of claim 1, wherein the consumption request comprises a query related to retrieval of an inventory of the storage asset, the query comprising a query parameter, and wherein the computer-implemented method further include: predefine a set of templates applicable to the storage asset, each corresponding template including configuration of the corresponding storage asset and prerequisites for deploying the storage asset; as well as A rule set configured to be executed on an inventory of storage assets is generated, the rule set configured to determine whether a template is achievable for a given storage asset based on one or more rules in the rule set.

6. The computer-implemented method of claim 5, further comprising receiving a list of deployment templates applicable to the storage asset, the list generated in view of the rule set, wherein the list of deployment templates includes at least one of: The reason why the first part of the list of deployment templates can be deployed; and The second part of the list of deployment templates is the reason why they cannot be deployed.

7. The computer-implemented method of claim 1, wherein the storage asset comprises a software-defined storage asset.

8. The computer-implemented method of claim 1, wherein at least a portion of the computer-implemented method is performed in a cloud-native environment.

9. A computer-implemented system for deploying storage assets, wherein include: processor; and a non-volatile memory in operative communication with the processor and storing computer program code which, when executed on the processor, causes the processor to perform a process operable to: a consumption request for accessing a consumed storage asset, the consumption request comprising a stack parameter and a resource characteristic parameter, wherein the stack parameter specifies at least one type of storage asset requested and the resource characteristic parameter specifies at least one functional capability required of the storage asset; determining, based on the stack parameters, a set of one or more first storage assets capable of satisfying the consumption request; For each undeployed first storage asset in the collection, generating a first workflow, the first workflow being configured to deploy the corresponding undeployed first storage asset in the collection; For each second storage asset in the set that lacks the resource characteristic parameter, generating a second workflow, the second workflow being configured to implement the resource characteristic in the corresponding second storage asset; as well as The collection of storage assets is configured to satisfy the consumption request, wherein the configuring includes running the first workflow and the second workflow.

10. The system of claim 9, further comprising computer program code that, when executed on the processor, causes the processor to: determining that a third storage asset resource can be created to satisfy the consumption request; and A third workflow is generated, the third workflow being configured to create and deploy the third storage asset resource.

11. The computer implemented system of claim 10, wherein the third storage asset resource is formed within at least one of the first storage asset and the second storage asset.

12. The system of claim 9, wherein the consumption request comprises a query related to retrieval of an inventory of the storage asset, the query comprising a query parameter, and wherein the computer-implemented method further comprises computer program code that, when executed on the processor, causes the processor to: predefine a set of templates applicable to the storage asset, each corresponding template including configuration of the corresponding storage asset and prerequisites for deploying the storage asset; and A rule set configured to be executed on an inventory of storage assets is generated, the rule set configured to determine whether a template is achievable for a given storage asset based on one or more rules in the rule set.

13. The system of claim 12, further comprising computer program code that, when executed on the processor, causes the processor to: Receiving a list of deployment templates applicable to the storage asset, the list generated in view of the rule set, wherein the list of deployment templates includes at least one of: The reason why the first part of the list of deployment templates can be deployed; and The second part of the list of deployment templates is the reason why they cannot be deployed.

14. The system of claim 9, wherein the storage assets comprise software defined storage assets.

15. The system of claim 9, wherein at least a portion of the computer program code is executed in a cloud-native environment.

16. A computer program product comprising a non-transitory computer-readable storage medium having computer program code encoded thereon, the computer program code, when executed on a processor of a computer, causing the computer to operate a storage system, the computer program product include: Computer program code for accessing a consumption request to consume a storage asset, the consumption request comprising a stack parameter and a resource characteristic parameter, wherein the stack parameter specifies at least one type of storage asset requested and the resource characteristic parameter specifies at least one functional capability required of the storage asset; computer program code for determining, based on the stack parameters, a set of one or more first storage assets capable of satisfying the consumption request; Computer program code for generating, for each first storage asset in the collection that is not deployed, a first workflow configured to deploy the corresponding first storage asset in the collection that is not deployed; Computer program code for generating, for each second storage asset in the collection that lacks the resource characteristic parameter, a second workflow configured to implement the resource characteristic in the corresponding second storage asset; as well as Computer program code for configuring storage assets of the collection to satisfy the consumption request, wherein the configuring comprises running the first workflow and the second workflow.

17. The computer program product of claim 16, further comprising: include: computer program code for determining that a third storage asset resource can be created to satisfy the consumption request; as well as Computer program code for generating a third workflow configured to create and deploy the third storage asset resource.

18. The computer program product of claim 16, wherein the consumption request comprises a query related to retrieval of an inventory of the storage asset, the query comprising a query parameter, and the computer program product further include: Computer program code for predefining a set of templates applicable to said storage asset, each respective template including a configuration of a respective said storage asset and prerequisites for deploying said storage asset; as well as Computer program code for generating a rule set configured to be executed on an inventory of storage assets, the rule set configured to determine whether a template is achievable for a given storage asset based on one or more rules in the rule set.

19. The computer program product of claim 18, further comprising computer program code for receiving a list of deployment templates applicable to the storage asset, the list generated in view of the rule set, wherein the list of deployment templates includes at least one of: The reason why the first part of the list of deployment templates can be deployed; and The second part of the list of deployment templates is the reason why they cannot be deployed.

20. The computer program product of claim 16, wherein the storage asset comprises a software defined storage asset.

Citation Information

Patent Citations

  • Offload stack for network, block and file input and output

    CN101171573A

  • System and method for electronic key provisioning and access management in connection with mobile devices

    CN106489248A

  • Custom resources in a resource stack

    CN108268323A

  • Automated generation of deployment workflows for cloud platforms based on logical stacks

    CN109155755A

  • Virtual data storage service with sparse provisioning

    US20150026430A1