A disc slicing system and method

Abstract

A disk slicing system for analyzing and optimizing the performance of storage disks within a distributed network is disclosed. The system comprises at least one storage disk, at least one benchmark engine, and at least one orchestrator. The orchestrator interacts with the storage disk and benchmark engine to direct the configuration of a performance profile by determining the optimal ratio of read / write input / output operations per second (IOPS) to the disk's total capacity. The orchestrator enforces this performance profile by performing a disk slicing operation on the disk.

Description

[0001] A DISK SLICING SYSTEM AND METHOD

[0002] FIELD OF THE INVENTION

[0003] The present invention relates in general to orchestration procedures for storage systems, and in particular to optimization of resource allocation.

[0004] BACKGROUND OF THE INVENTION

[0005] Providing data storage and data backup capabilities represent a significant concern as current computing systems, whether local, remote or cloud based (such as containers packages, private / public / multi -cloud systems, etc.), require ever more extensive data storage solutions for their proper operation. Usually, such data provision and management are made and offered by designated data centers and traditionally the provision of used or expected to be used data storage is provided by stacking physical data storing components, i.e. hybrid hard disk drive (HHD), hard disk drive (HDD), solid-state drive (SSD), etc. Because the methods by which data is stored and edited on different types of drives are so distinct, a similarly broad variety of network configurations and operating methods have emerged to meet the requirements of different network applications.

[0006] Many of these systems and methods include technical features which - whilst distinct - can serve similar functions in the very specific context in which they are disposed, albeit not functions that are independent of said context. Technical features relating to the storage, transfer, sensing, and management of data are employed in a variety of approaches, systems, methods, and network configurations, which have been developed to address a range of technical problems relating to data storage and network management broadly. Much of these are discussed below, in order to provide a broad overview of the relevant prior for the present invention.

[0007] An approach well established in the field of data storage is the operation of stacking data storing components to create what is termed “Storage Arrays ” (or alternatively “disk arrays ”) which are used for different kinds of data, broadly categorized by: block -based storage; file-based storage; object storage, among other data types. Rather than store data on a server, storage arrays use multiple drives in a collection capable of storing a huge amount of data, controlled by a local / central controlling system interfacing via storage network protocols to the server.

[0008] Traditionally, a storage array controlling system provides multiple storage services so as to keep track of storage capacity; the allocation of space to different datasets, the management of sections of data storage capacity known as “volumes ”; the periodic backup operation of the data to facilitate restoration and disaster recovery and the creation of point-in-time copy of the data, known as "snapshoiiin ". the identification and tracking of errors; the encryption of data communication to protect the integrity and privacy of data; the compression of data to conserve storage capacity; etc. Services of such type require significant computing capacity, metadata, data storage, accelerators, etc. - thus, such services require the designation of extensive infrastructure and budget capacities and resources.

[0009] Commonly, a storage array is separated from a system server's operability and is configured to implement system and application operations on dedicated hardware, for example a server stack, a storage array stack, one or more hard disk or solid state drive (HDD or SSD) and media input / output (I / O) devices configured to communicate with the servers via the storage stack.

[0010] Another approach well established in the field is the employment of an orchestrator, which is a software module logically situated in the control plane (CP) of a distributed network and is responsible for managing the operations of the data plane (DP), such as provisioning and resource coordination. The DP is the layer within a network architecture responsible for the movement, processing and storage of data through the distributed network, whilst the CP is an associated network layer that responsible for controlling how said data flows through the DP. Positioned in the CP, the orchestrator provides centralized management of the DP network, automating data flow across network nodes in accordance with predefined rules. Such coordination is particularly important in distributed network environments where resources such as computational power, storage and network bandwidth are spread across multiple nodes, often in different physical locations. Another approach well established in the field of data storage is the operation of redundant arrays of independent disks (RAID), which can be operated as a way of storing the same data in different places to protect data in the case of a system failure. RAID is a general approach and network configuration that virtualizes data and combines multiple physical disk drive components into one or more logical units. Persons skilled in the art will appreciate that the technical problem RAID operations are employed to address depend on the type of RAID operation undertaken: RAID 0 stripes data across multiple disks to address performance bottlenecks and capacity limitations; RAID 1 mirrors data across two or more disks to address data loss due to disk failure; RAID 2 stripes data at the bit level and uses Hamming code for error correction to address data errors and fault tolerance in high-reliability systems; RAID 3 stripes data at the byte level and uses a dedicated parity disk to address single-disk failure and sequential data access bottlenecks; RAID 4 stripes data at the block level with a dedicated parity disk to address single -disk failure and block-level performance bottlenecks; RAID 5 stripes data and distributes parity information across multiple disks to address single-disk failure and storage efficiency; RAID 6 stripes data with double distributed parity to address multiple disk failures and ensure data integrity; RAID 10 combines mirroring (RAID 1) and striping (RAID 0) to address performance bottlenecks and single-disk failure; RAID 01 mirrors two RAID 0 arrays to address performance bottlenecks and fault tolerance; RAID 50 combines RAID 5 arrays and stripes them using RAID 0 to address the performance and reliability limits of RAID 5; RAID 60 combines RAID 6 arrays and stripes them using RAID 0 to address the performance and redundancy limits of RAID 6; RAID 7 uses an embedded real-time OS and dedicated cache to improve performance and address bottlenecks associated with traditional RAID levels; RAID IE stripes mirrored data across an odd number of disks to address fault tolerance and performance in setups where an odd number of disks are available.

[0011] Another approach well established in the field of data storage is the operation of remote replication, which is the process of copying data to a device at a remote location for data protection or disaster recovery purposes. Remote replication may be either synchronous or asynchronous, the former writes data to the primary and secondary sites at the same time, and the latter at different times. Because asynchronous replication is designed to work over longer distances and requires less bandwidth, it is often considered a better option in the field for the recovery of data after a catastrophic disaster.

[0012] However, the operation of asynchronous replication also introduces several risks, not least the risk of loss of data during a system outage as said data at the target device isn't synchronized with the source data. Most enterprises today use data storage vendors that include replication software on their high- end and mid-range storage arrays, to partially mitigate this risk.

[0013] Another configuration well established in the field of data storage is software -defined storage (SDS), which enables communality of operation of different hardware. SDS configurations include the abstraction of data storage resources from the underlying physical storage hardware, and thereby are able to provide flexible exploitation of available hardware and data storage resources. Typically commercial off-the-shelf servers run a subset of SDS known as hyper-converged infrastructure HCI, in which the abstractions of both the area network and the underlying storage are implemented virtually in software, rather than physically in hardware.

[0014] Both conventional storage arrays and SDS configurations typically include an integrated “storage stack” - a layered software framework that organizes, manages and facilitates data storage, access and retrieval. Said storage stack t provides essential services such as data protection (e.g. backup, redundancy, recovery, etc.); space allocation; data optimization, backup and recovery, among other functions. Due to the broad array of functions required by SDSs, the integrated software stack is typically configured to have a high of reliability, and the efficiency of the code is also conventionally prioritized.

[0015] Another data storage configuration taught in the field is directed attached storage (DAS), which typically provides the direct local services (such as encryption, compression, RAID, etc.) in cases where central storage systems are not needed or desired. Conventionally, DAS configurations will exploit a robust collection of internal storage components, without which the means of operating said services would be insufficient for proper network function. Persons skilled in the art will appreciate that the technical problem DAS network configurations are employed to address is: the provision of data storage services in the absence of centralized data management nodes. DAS is mostly limited to non-critical applications due to an inherent drawback related to the fact that DAS is inherently tied to one host: server communication failure precludes data accessibility, typically limiting DAS to non-critical applications. This is in contrast to the SDS solutions previously described, which are typically accessible by multiple servers over the network; if one server or communication channel fails, other servers can still access the storage.

[0016] Another approach well established in the field of data storage is the operation of hot spares. Traditionally, hot spares act as standby drives in RAID 1, RAID 5, or RAID 6 volume groups, but they have also been applied to other network management approaches. Generally, if a drive fails, for example in a volume group, some control software will reconstruct data from the failed drive on a hot spare. When a drive fails in a storage array, a hot spare drive can be substituted without requiring a physical swap. Persons skilled in the art will appreciate that the technical problem hot spare configurations are employed to address is: minimizing downtime and ensuring quick recovery from disk failures in RAID and other storage systems. Another approach well established in the field of data storage is the operation of snapshots of data. A snapshot is used to represent the content of a particular part of a data stored on a storage system at a particular point in time. The source of snapshots are typically base volumes, which are usually referred to as “member volumes ” of a “consistency group”. The purpose of a consistency group is to facilitate the capture of simultaneous snapshot images of multiple volumes, thus obtaining copies of a collection of volumes at a particular point in time. In practice, most mid-range and high-end storage arrays create snapshot consistency groups within volumes inside the storage array. Persons skilled in the art will appreciate that the technical problems snapshot operations are employed to address are: loss prevention; data recovery; control of database version; rule compliance and auditing; monitoring of storage dynamics, among other technical problems.

[0017] Obtaining a local snapshot is enabled by a server operating system that includes a logical volume manager (LVM) - a software layer that abstracts physical storage disks into virtualized storage units (logical volumes) - enabling the obtaining of a local snapshot on a single virtualized volume. In distributed storage system, since the volumes are distributed across multiple servers, obtaining or creating a consistency group is not usually possible or supported, producing a number of data integrity risks. LVM works by partitioning the physical volumes (PVs) into physical extents (PEs), which are mapped onto logical extents (LEs) which are then pooled into volume groups (VGs), linked together as logical volumes (LVs). Persons skilled in the art will appreciate that the LVM approach is typically undertaken in order to address the technical problems posed by: inflexible partition sizes; fragmentation of disk space; limited scalability of storage infrastructure; complex mirroring and striping setups; difficulty in taking snapshots; and the efficient management of multi -disk systems.

[0018] Another approach well established in the field of data storage is quality of service (QoS), which is critical to deliver consistent storage performance applications where multiple workloads share a single limited resource by preventing the “noisiest neighbor” from disrupting the performance other applications on the same system. On physical storage arrays, QoS can be set for volumes as limits on data transfer. Unlike storage arrays, the distributed servers of storage stacks mean there isn’t a single point that can enforce QoS. Persons skilled in the art will appreciate that QoS is a general approach in data storage array management, which can be disposed to address a number of different challenges, including but not limited to: predictable performance in shared resources; performance spikes caused by noisy neighbors; difficulty maintaining SLA compliance; resource contention during peak loads; and the need for overprovisioning to avoid performance issues.

[0019] Another approach well established in the field of data storage is disk cloning, which is the process of making a copy of a part (or all) of a hard drive, typically undertaken at a particular point in time whilst hosts continue to access the data. Like QoS, this is an approach which is difficult to operate on shared storage stacks, since the source and target may reside on different physical entities. Persons skilled in the art will appreciate that disk cloning is typically undertaken in order to address the technical problems of: efficient data migration; disaster recovery; consistent system deployment; backup integrity, and the prevention of data loss due to hardware failure.

[0020] Another approach well established in the field of data storage is thick provisioning , where the complete amount of virtual disk storage capacity is pre -allocated on the physical storage when the virtual disk is created, rendering capacity unavailable for use by other volume. Persons skilled in the art will appreciate that the thick provisioning approach is typically undertaken in order to address the technical problems posed by: unpredictable availability of storage capacity; overcommitted storage resources; storage fragmentation, performance degradation, the risks of complex storage management; and the resultant shortages in capacity from said technical problems leading to data loss. In contrast to thick provisioning, yet another approach well established in the field of data storage is thin provisioning, where a virtual disk consumes only the space that it needs initially, and grows with time according to increase in demand. Whilst thinly provisioned storage consumes less disk space, it consumes significantly more RAM to store the metadata of the thin allocation. Additionally, thin provisioning consumes much more CPU on the I / O transmissions needed to facilitate intensive random access to translate logical addresses to physical, since it has to navigate through a tree -like data structure. Despite these limitations, thin provisioning is a widely undertaken approach to address a number of different technical problems of the field, persons skilled in the art will appreciate that said technical problems include but are not limited to: the inefficient utilization of storage; high upfront capital costs; difficulty in scaling storage; and the over-allocation of resources.

[0021] Another approach well established in the field of data storage is the Clustered Logical Volume Manager (CLVM), which is a set of clustering extensions to LVM, an approach discussed earlier. These extensions allow a cluster of computers to manage shared storage using LVM by locking access to physical storage while a logical volume is being configured. A single misbehaving node can impact the health of the entire cluster, introducing significant risk for the integrity of data stored on a data storage system. Persons skilled in the art will appreciate that the technical problems the CLVM approach is disposed to address include but are not limited to: uncoordinated access to shared storage introducing storage performance limitations; corruption of stored data from multiple read / write operations; limitations to the scalability of storage environments; low storage availability; inefficient data sharing; and the risks of high complexity in the management and expansion of shared storage.

[0022] Another approach well established in the field of data storage is the deployment of a hardware security module (HSM), which is a physical device that manages digital keys for strong authentication. Persons skilled in the art will appreciate that HSMs are typically deployed in order to address the technical problems posed by: secure key generation and storage; tamper detection and resistance; performance bottlenecks for cryptographic operations; regulatory compliance; controlled key access; secure cryptographic operations; auditing; and logging. Another approach well established in the field of data storage is the use of tunneling protocols, which are a communications protocols that allow for the movement of private data from one network to another across a public network, using a process called encapsulation. Persons skilled in the art will appreciate that tunneling protocols are typically operated in order to address the technical problems posed by: secure transmission of data over untrusted networks; bypassing network restrictions and firewalls; ensuring confidentiality and integrity of data in transit; preventing eavesdropping and man- in-the-middle attacks; encapsulating incompatible or sensitive protocols; and reducing exposure to external threats.

[0023] Other approaches have been taught in the art to address the challenges of secure communication in distributed storage environments, including virtual private networks (VPNs)_; reverse proxies; agentbased models; and secure APIs. VPNs and encrypted tunnels create secure connections, but add latency and require extensive setup. Reverse proxies and API gateways offer controlled access to storage servers by routing external requests through a single entry point, but they also add routing layers that create bottlenecks and increase complexity. Agent -based models, which rely on modules within a network to pull commands from the control software rather than receive them directly, help bypass firewall restrictions but delay orchestration by requiring periodic updates instead of real-time communication. Secure APIs, which rely on authentication protocols, provide direct access to storage resources but can be challenging to scale across large networks due to resource demands.

[0024] Similar to the challenges of security, many approaches have been taught in the art to address chattiness, which is when communication between servers consists of repetitive, non-essential notifications that create unnecessary traffic. This challenges is typically addressed using: traffic filtering; message batching; and rate limiting, which selectively blocks non-essential communications; aggregates multiple smaller messages into fewer transmissions; and restricting the volume of messages over a defined interval, respectively.

[0025] Not unlike solutions to chattiness, many systems and methods have been taught in the art to address the challenge of identification of servers within node-based cloud storage networks, particularly in multi-tenant environments. Conventional means for server identification typically rely on: IP address verification; hostname recognition; and basic authentication protocols such as API keys or token -based systems

[0026] Storage systems may be implemented as on-premises data centers, wherein servers and infrastructure are privately owned and managed, or as networked storage environments, such as those offered via cloud computing service providers. Cloud storage systems may exploit shared resources both for the storage media, which physically stores the data, and for the network infrastructure, which serves to connect the storage system to other systems and clients. In some configurations, storage systems utilize shared networks for general operations, while in others, dedicated networks may be required for each function in order to optimize performance and manage system resources more efficiently. Persons skilled in the art will appreciate that the choice of whether to employ shared or dedicated networks may depend on various technical factors, including but not limited to: workload types, data throughput requirements, latency considerations, as well as scaling requirements.

[0027] Node systems are critical components within a networked environment, acting as intermediaries that facilitate communication and data exchange across all devices in the network. In the context of a data communication network, a node refers to a distinct device capable of transmitting, receiving, or routing data. In addition to their fundamental role in data transmission, node systems often provide quality of service (QoS) monitoring capabilities, ensuring that data flow across the network meets predefined performance metrics. A particular implementation of node systems is cloud storage, which delivers scalable and flexible storage solutions for individuals and organizations. Cloud storage nodes operate in distributed environments and are thus capable of dynamically adjusting to accommodate fluctuating storage demands. Cloud storage systems face several technical challenges which impact performance and reliability.

[0028] Storage devices within distributed storage systems are highly variable in their performance, particularly under “real-world”, mixed workloads. While storage devices, such as flash drives, often come with technical specifications that outline their maximum read and write IOPS (input / output operations per second), these figures frequently reflect ideal conditions, such as read-only or write-only scenarios. In practice, most storage environments involve a combination of both read and write operations, and the performance of the device can degrade significantly under such mixed workloads. For example, the specified read IOPS capability of a storage device may decrease significantly when even a small number of write operations are performed simultaneously. As the demand for write operations increases, the read IOPS can continue to decline, and can drop to less than 5% of its stated read IOPS capacity. Currently, storage devices lack the ability to provide reliable performance guarantees under mixed workload conditions. Consequently, end users are unable to predict how storage volumes created from such devices will perform with respect to read and write IOPS.

[0029] Furthermore, storage devices experience performance degradation over time due to internal processes such as garbage collection, driven by hostile write patterns. These internal operations clean and rearrange data within the drive, especially when subjected to specific, intensive write patterns. Write operations can also lead to fluctuations in temperature, which can also impact the performance of the drive. It will be well appreciated by persons skilled in the art that this degradation, coupled with the unpredictable nature of mixed workloads, can lead to inconsistent performance, affecting the reliability and efficiency of distributed storage systems.

[0030] There is thus a need in the art for a system that ensures consistent and predictable performance of storage devices under mixed workload conditions. Such a system should address the variability caused by simultaneous read and write operations, as well as the effects of internal processes and temperature fluctuations. Additionally, there is a need for a system that provides end users with greater transparency and reliability regarding the performance capabilities of storage volumes, enabling more efficient and effective use of distributed storage systems.

[0031] SUMMARY OF THE INVENTION

[0032] The following embodiments and aspects thereof are described and illustrated in conjunction with systems, devices and methods which are meant to be exemplary and illustrative and not limiting in scope. In various embodiments, one or more of the above -described problems have been reduced or eliminated, while other embodiments are directed to other advantages or improvements. The following embodiments and aspects thereof are described and illustrated in conjunction with systems, devices and methods which are meant to be exemplary and illustrative and not limiting in scope. In various embodiments, one or more of the above -described problems have been reduced or eliminated, while other embodiments are directed to other advantages or improvements.

[0033] According to a first aspect of the invention, a disk slicing system configured to analyze and optimize the performance of storage disks within a distributed network comprises: (i) at least one storage disk; (ii) at least one benchmark engine; and (iii) at least one orchestrator configured to interact with the said storage disk and said benchmark engine, wherein the orchestrator is configured to direct the at least one benchmark engine to configure a performance profile for said disk by determining an optimal ratio between read / write input / output operations per second (IOPS) and the total capacity of said disk and enforce said performance profile to said disk by undertaking a disk slicing operation thereon.

[0034] According to another aspect of the invention, the at least one benchmark engine is configured to operate at set time intervals, and wherein the performance profile is updated thereby at said intervals, and wherein the locking ratio relating to said updated performance profile is enforced by the orchestrator.

[0035] According to some embodiments, the benchmark engine is configured to analyze the locked storage disk by generating and applying a variety of workload scenarios. These scenarios include mixed read and write operations to evaluate the performance characteristics of the storage media under at the locked ratio. The analysis encompasses parameters such as read and write input / output operations per second (IOPS), bandwidth, latency, and usable capacity.

[0036] According to another aspect of the invention, the at least one benchmark engine is configured to procure (i.e. enables procurement) a capacity-designated performance profile that prioritizes the optimization of capacity on the at least one disk. According to another aspect of the invention, the at least one benchmark engine is further configured to procure a performance-designated performance profile that prioritizes the optimization of performance of the at least one disk.

[0037] According to another aspect of the invention, the at least one benchmark engine is configured to create at least one catalog of the at least one disk provided by at least one cloud provider, the at least one catalog including performance profiles and metadata for each storage disk

[0038] According to another aspect of the invention, the catalog comprises at least two catalog entries for each disk provided by the cloud provider, one related to a first configuration that prioritizes capacity and a second related to a second configuration that prioritizes performance, wherein all of the at least two catalog entries are provided by the at least one benchmark engine.

[0039] According to another aspect of the invention, the at least one benchmark engine further determines the pricing and availability of the at least one storage disk and stores this data in the catalog.

[0040] According to another aspect of the invention, the at least one orchestrator is further configured to monitor pricing and availability of the at least one storage disk.

[0041] According to another aspect of the invention, analysis provided by the at least one benchmark engine is configured to consider numerous parameters of the at least one storage disk including capacity, type, operation system, and location.

[0042] According to another aspect of the invention, the at least one orchestrator is further configured to allocate storage disks from the cloud provider based on the performance profile, further incorporating parameters such as instance type, instance size, availability zone, and failure domain to address specific use case requirements or client needs.

[0043] According to another aspect of the invention, at least one planner service is configured to analyze data collected by the at least one orchestrator, including performance profiles and additional parameters provided to the at least one orchestrator by the at least one benchmark engine, and wherein said at least one planner service is designated to incorporate the analysis provided by the at least one planner service with user-specific requirements.

[0044] According to another aspect of the invention, the at least one orchestrator and / or at least one benchmark engine is implemented as a cloud-based service (SaaS)configured to provide a composable storage system by slicing and composing storage disks through cloud computing operations.

[0045] According to another aspect of the invention, the cloud-based slicing and composing of at least one storage disk is configured to optimize the utilization of storage resources, according to a performance profile provided by the at least one benchmark engine.

[0046] According to another aspect of the invention, the optimization of storage resources is configured to minimize unused storage capacity.

[0047] According to another aspect of the invention, the at least one benchmark engine is configured to analyze and monitor the performance of the at least one storage disk in relation to heat generated during its operation.

[0048] According to another aspect of the invention, the at least one benchmark engine is configured to analyze and monitor the performance of the at least one storage disk in relation to internal processes such as memory management operations.

[0049] According to some embodiments, the benchmark engine is designated to monitor storage disk performance by running worst-case read / write simulations that may include hostile write patterns.

[0050] According to another aspect of the invention, the at least one storage disk is solid-state drive (SSD) based.

[0051] According to another aspect of the invention, the at least one storage disk is storage class memory (SCM) based.

[0052] According to another aspect of the invention, the at least one storage disk is random access memory (RAM) based. According to another aspect of the invention, the at least one storage disk is hard disk drive (HHD) based.

[0053] According to another aspect of the invention, a method for analyzing and optimizing the performance of storage disks within a distributed network comprises: (i) providing at least one storage disk; (ii) providing at least one benchmark engine; (iii) providing at least one orchestrator configured to interact with the said storage disk and said benchmark engine; and (iv) directing, by the at least one orchestrator, the at least one benchmark engine to configure at least one performance profile for the said at least one storage disk, wherein the configuration comprises determining an optimal ratio between read / write input / output operations per second (IOPS) and the total capacity of the said at least one storage disk and enforcing the configured performance profile onto the said at least one storage disk by disk slicing.

[0054] According to another aspect of the invention, the at least one performance profile is configured by the at least one benchmark engine to optimize the use of storage resources in accordance with the intended application.

[0055] According to another aspect of the invention, the disk slicing allocates storage resources dynamically based on the performance profile.

[0056] BRIEF DESCRIPTION OF THE FIGURES

[0057] Some embodiments of the invention are described herein with reference to the accompanying figures. The description, together with the figures, makes apparent to a person having ordinary skill in the art how some embodiments may be practiced. The figures are for the purpose of illustrative description and no attempt is made to show structural details of an embodiment in more detail than is necessary for a fundamental understanding of the invention.

[0058] In the Figures:

[0059] FIG. 1 constitutes a schematic illustration of a typical data storage system. FIG. 2 constitutes a table of specific storage disk families, according to some embodiments of the invention.

[0060] FIG. 3 constitutes table illustrating the results of a benchmark engine’s analysis configured to test two aspects of a storage disk, according to some embodiments of the invention.

[0061] FIGS. 4A and 4B constitutes a tabular illustration of storage disk pricing and a schematic illustration demonstrating the analysis undertaken by the benchmark engine on a new disk, according to some embodiments of the invention.

[0062] FIG. 5 constitutes a schematic illustration of a disk slicing utilization, according to some embodiments of the invention.

[0063] FIG. 6. constitutes a schematic illustration of a controller operating as a declarative engine forming a part of the disk slicing system, according to some embodiments of the invention.

[0064] FIG. 7. constitutes a schematic illustration of a specific utilization of the disk slicing system, according to some embodiments of the invention.

[0065] DETAILED DESCRIPTION OF SOME EMBODIMENTS

[0066] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components, modules, units and / or circuits have not been described in detail so as not to obscure the invention. Some features or elements described with respect to one embodiment may be combined with features or elements described with respect to other embodiments. For the sake of clarity, discussion of same or similar features or elements may not be repeated.

[0067] Although embodiments of the invention are not limited in this regard, discussions utilizing terms such as, for example, “controlling” “processing,” “computing,” “calculating,” “determining,” “establishing”, “analyzing”, “checking”, “setting”, “receiving”, or the like, may refer to operation(s) and / or process(es) of a controller, a computer, a computing platform, a computing system, or other electronic computing device, that manipulates and / or transforms data represented as physical (e.g., electronic) quantities within the computer's registers and / or memories into other data similarly represented as physical quantities within the computer's registers and / or memories or other information non-transitory storage medium that may store instructions to perform operations and / or processes.

[0068] Unless explicitly stated, the method embodiments described herein are not constrained to a particular order or sequence. Additionally, some of the described method embodiments or elements thereof can occur or be performed simultaneously, at the same point in time, or concurrently.

[0069] The term "Controller" as used herein refers to any type of computing platform or component equipped with a Central Processing Unit (CPU) or microprocessor and capable of supporting multiple input / output (I / O) ports. The term “Node” as used herein refers to any system or device that serves as a connection point within a network, facilitating communication and data exchange. Nodes can take various forms, including servers, which provide resources or services such as data storage, website hosting, or application management, as well as computers, wireless access points, modems, gateways, switches, and routers, each fulfilling a distinct role in enabling connectivity and interaction.

[0070] The term “benchmark engine” as used herein, generally refers to a software tool or system designed to measure and assess the performance of hardware, software, or an entire computing system and may be configured to provide standardized and reproducible tests that enables evaluation of speed, efficiency and capabilities of different components or systems.

[0071] Reference is now made to FIG. 1, which schematically illustrates atypical data storage system 10. As shown, a data storage system 10 may comprise at least one target server 100 that may further comprise a storage media 104 and be configured to run an operating system (for example, a Uinux based operating systems such as Red Hat, Suse, etc.), wherein said operating system is designated to host data accessible over a DP network. According to some embodiments, at least one initiator server 102 may be configured to run an operating system (for example, a Linux based operating systems such as Red Hat, Suse, etc.), wherein said operating system is designated to access and be exposed to remote resource / s over the DP network.

[0072] According to some embodiments, at least one orchestrator 106 may be configured to interact with each of said target server / s 100 and / or initiator server / s 102 in order to control the CP of said DP network. According to some embodiments, a designated portion of the storage media 104 forming a part of the target server 100 may be exposed to the DP network, in other words, a designated physical space is reserved and specified in order to contribute a storage space used by the DP network.

[0073] According to some embodiments, orchestrator 106 is configured to utilize the designated portion of the storage media by orchestrating storage stack (SS) components and standard storage stack (SSS) of the operating system embedded within said target / initiator server / s 100 / 102, such that the initiator server 102 is configured to interact with the target server 100 via the DP network.

[0074] According to some embodiments, data storage system 10 may be designated to perform the following steps:

[0075] • Using the operating system installed on target the storage media 104 forming a part of the target server 100 and designated to host data accessible over the DP network, wherein said storage media 104 is used to utilize a persistent storage medium,

[0076] • Using the operating system’s logical volume manager (LVM) in order to split the storage media 104 to multiple partitions,

[0077] • Using the operating system installed on at least one initiator server 102 in order to access and consume the storage media 104’ partition / s over the DP network in order to utilize a remote media’s capacity and performance, Using the orchestrator 106 which is configured to interact with each of said target server / s

[0078] 100 and initiator server / s 102, wherein said orchestrator 106 is designated to control the CP of said DP network,

[0079] • Using the operating system to merge at least two network paths in order to utilize a single storage media partition as a single network device by creating a multipath component, and thus, enabling enhanced redundancy and efficiency.

[0080] According to some embodiments, the steps disclosed above may further include using a resource management component / s in order to provide dynamic allocation and de-allocation capabilities configured to be conducted by the operating system and affect, for example, on processor cores and / or memory pages, as well as on various types of bandwidths, computations that compete for those resources. According to some embodiments, the objective of the steps disclosed above is to allocate resources so as to optimize responsiveness subject to the finite resources available.

[0081] According to some embodiments and as disclosed above, at least one initiator server 102 is configured to interact with at least two target servers 100 using a multipath connection. According to some embodiments, a multipath connection may be used to improve and enhance the connection reliability and provide a wider bandwidth.

[0082] According to some embodiments, the coordination between the initiator server / s 102 and the target server / s 100 or vice versa, may be conducted using a local orchestrator 107 component configured to manage the CP and further configured to be physically installed on each server. According to some embodiments, installing a local orchestrator 107 on each server may provide a flexible way of utilizing the data storage system 10 as well as eliminate the need to provide data storage system 10 with access to internal software and processes of a client’s servers. According to some embodiments, the operations and capabilities disclosed in the current specification with regards to orchestrator 106, may also apply to local orchestrator 107 and vice versa. According to some embodiments, the communication between the orchestrator 106 and between either a target server 100 or the initiator server 102, may be conducted via the DP network by utilizing a designated software component installed on each of said servers.

[0083] According to some embodiments, the initiator server 102 is configured to utilize a redundant array of independent disks (RAID) storage stack component (SSC) configured to provide data redundancy originated from multiple designated portions of the storage media 104 embedded within multiple target servers 100.

[0084] According to some embodiments, the RAID SSC is further configured to provide data redundancy originated from combined multiple initiator paths originated from the designated portion of the storage media 104 of at least two target servers 100.

[0085] According to some embodiments, the target servers 100 may be located at different locations, such as, in different rooms, buildings or even countries. In this case, the orchestration procedure conducted by the orchestrator 106 is allocated across different resiliency domains. For example, the orchestrator 106 may consider various parameters regarding cyber security, natural disasters, financial forecasts, etc. and divert data flow accordingly. According to some embodiments, said orchestration procedure conducted by the orchestrator 106 and configured to utilize servers’ allocation, is conducted with a consideration of maintaining acceptable system balance parameters.

[0086] According to some embodiments, the orchestrator 106 may be configured to interact with server / s 100 / 102 using an administration protocol. According to some embodiments, the designated portion of the storage media 104 may be allocated using a logical volume manager (LVM) SSC. According to some embodiments, the storage media 104 may be solid-state drive (SSD) based, storage class memory (SCM) based, random access memory (RAM) based, hard disk drive (HHD) based, etc.

[0087] According to some embodiments, the orchestrator 106 may be a physical controller device or may be a cloud-based service (SaaS) and may be configured to command and arrange data storage and traffic in interconnected servers, regardless whether orchestrator 106 is a physical device or not. According to some embodiments, the operations on each server / s 100 / 102 may be implemented, wholly or partially, by a data processing unit (DPU), wherein said DPU may be an acceleration hardware such as an acceleration card, wherein hardware acceleration may be use in order to perform specific functions more efficiently when compared to software running on a general-purpose central processing unit (CPU), and hence, any transformation of data that can be calculated in software running on a generic CPU can also be calculated in custom-made hardware, or in some mix of both.

[0088] According to some embodiments, under a traditional SDS, the storage stack code would rely on proprietary software, requiring separate, independent installation and maintenance, whereas data storage system 10 is configured to rely on an already installed operating system’ capabilities combined with using the orchestrator 106 discussed above.

[0089] According to some embodiments, under a traditional SDS, the control protocol would rely on proprietary software requiring separate, independent installation and maintenance, whereas the data storage system 10 is configured to rely on the operating system capabilities using the orchestrator 106 discussed above.

[0090] Under a traditional SDS, the nodes interconnect would rely on proprietary software, whereas according to some embodiments, data storage system 10 is configured to rely on standard storage protocols using the orchestrator 106 discussed above.

[0091] Under a traditional SDS, the stack model that controls the storage array system 10 uses a single proprietary code and has components interleave both DP and CP, whereas according to some embodiments, data storage system 10 is configured to utilize the operating system to execute a dummy data plane while using the orchestrator 106 disclosed above in order to emulate a CP and execute its actual operations upon (among others), the DP.

[0092] Some advantages of the various embodiments disclosed above may facilitate ultra-high performance when compared to a traditional SDS operations, for example, with regards to the number of nodes in a cluster which under data storage system 10 are expected to be unlimited. As noted there are certain drawback and inefficiencies in coupling the together of DP and the CP with the storage when implemented in traditional networks, including SDNs. According to some embodiments, an SDN may be configured to decouple the DP from the storage, and the CP from the storage. According to some embodiments, a storage system may be built with dummy devices to forward and store data, from the CP of the network, which controls how the traffic will flow through the network while SDN is considered to enable much cheaper equipment, agility and limitless performance than other decoupling means, since more data plane resources can be flexibly added-on, such decoupling using an SDN enables scalability with no limitation and higher survivability rate due to a limited impact on the processed cluster. According to some embodiments, a single orchestrator may be provided to provide storage services to huge cluster since, data capacity, bandwidth and IOPS do not impact the CP services utilization. Such decoupled DP may be utilized for various data Services, such as: Protocols; RAID; Encryption; QoS Limits; Space Allocation; Data Reduction and others. Whereas such a decoupled CP may be utilized for various storage services and coordination, such as: Volume Lifecycle (Create, Delete, Resize, etc.); Storage Pool Management; Snapshot Coordination - Consistency Groups; failure Monitoring; Performance Monitoring; Failure Handling.

[0093] According to some embodiments, data storage system 10 may be designated to perform the following steps to obtain an SDN based CP and CD and storage decoupling:

[0094] Using the storage CP 104 on one of servers 100, to create storage DP; Using the storage CP on server 102, to create storage DP, to connect to said server 100 and consume the exposed drive chunk;

[0095] • Using the storage CP on another server 100, to create storage DP;

[0096] • Using the storage CP on server 102, to create storage DPe, to connect to said another server 100 and consume the exposed drive chunk; Using the storage cCP on server 102, to create storage DP, that includes multipath, RAID, encryption, compression, deduplication, LVM and replication services.

[0097] Reference is now made to FIG. 2, which tabularly illustrates storage disk families, according to some embodiments of the invention. As shown, in this list offered by AWS and presented as an example, various instances having differs characteristics such as vCPU, memory, storage capacity, and network bandwidth capabilities.

[0098] For example, AWS provides a variety of instance families, each optimized for different use cases (e.g., compute-optimized, memory-optimized, storage-optimized). Users can choose the instance type that best meets their performance and resource requirements.

[0099] Cloud-based storage enables clients to allocate resources based on application requirements. This capability is achieved by partitioning all physical resources of a specific instance into virtual slices and distributing them among different virtual machines. For instance, FIG. 2 illustrates a specific storage instances family, c6gd, offered by AWS.

[0100] According to some embodiments, the disk slicing capabilities are built upon the ability of cloudbased storage to allocate resources according to applications needs and constructs a data infrastructure from components within the public cloud.

[0101] According to some embodiments, the disk slicing capabilities may align seamlessly with specific workload requirements requested by the user or suggested by the system. According to some embodiments, said intelligent slicing and composition not only entails the selection of optimized storage media components but also may integrate elements such as network awareness, fault domains, application profiling and demands and ensure a dynamic and tailored provisioning of resources, optimizing the overall efficiency, storage capacity, and budget considerations for diverse applications and users.

[0102] Reference is now made to FIG. 3, a table illustrating the results of an operation conducted by a benchmark engine forming a part of the disk slicing system, according to some embodiments of the invention.

[0103] According to some embodiments, the benchmark engine is a key aspect of the disk slicing system’s approach in determining a performance profile. The benchmark engine may imbue the system with the capability to systematically test each type of storage disk under various workloads and conditions and generate a comprehensive catalog record which may include performance characteristics consistently and repetitively provided by each specific disk within the cloud storage system . Depending on the users’ need, the performance profile may emphasize higher storage capacity at the expense of IOPS or higher IOPS at the expense of storage capacity.

[0104] According to some embodiments, following the benchmark engine’s analysis, the disk slicing system is configured to lock a ratio between read / write IOPS and total capacity of the at least one storage volume in accordance with various need and constrains.

[0105] According to some embodiments and as previously disclosed, the benchmark engine may be configured to test two aspects of an instance, one that prioritizes capacity and one that prioritizes performance, for example, an instance that prioritizes capacity may have more disk space but provide a lower IOPS comparing to an instance that prioritizes performance and as a consequence, has less disk space available, sible.

[0106] According to some embodiments, the benchmark engine is configured to create a catalog of at least one cloud provider’s resources, wherein the catalog comprises at least two catalog entries for each instance provided by the cloud provider, one related to a first configuration that prioritizes capacity and a second related to a second configuration that prioritizes performance.

[0107] Reference is now made to FIGS. 4A and 4B, , a schematic illustration demonstrating the analysis undertaken by the benchmark engine on a new disk and a tabular illustration of storage media pricing, according to some embodiments of the invention. For the purposes of clarity and brevity, the at least one orchestrator required to form the network whose operation is illustrated, partially, FIGs. 4A-B is not shown. However, persons skilled in the art will appreciate that this embodiment of the invention relies fundamentally on the operation of the orchestrator as taught elsewhere in the present application. Said orchestrator may be configured to monitor pricing and availability and / or size and / or type of each instance across diverse regions and other parameters in accordance with the analysis conducted by the benchmark engine. According to some embodiments, the benchmark engine may conduct benchmark tests on emerging instances. For example, a c6gd.medium instance - as taught in FIG. 2 - operating in a Linux or RHEL environments and located in the US east or west coast may have a different hourly pricing which are configured to be recorded by the benchmark engine and provided to the orchestrator. The performance profiles produced by the benchmark engine enables the disk slicing system to allocate cloud storage resources in accordance to the users’ needs.

[0108] In operation 100 a new instance type is presented to the disk slicing system, in operation 102 the benchmark engine may analyze and determine performance characteristics 104, capacity characteristics 106 and location and price info 108 of said storage instance which may then be stored in a catalog 110 and record in designated catalogs 112 and 114.

[0109] Similarly, in the event of an instance deprecation by a cloud provider, the disk slicing system may promptly review and update its catalog, thereby guaranteeing that the data remains current and reliable, providing users with up-to-date information at all times.

[0110] According to some embodiments, the orchestrator is configured to optimize the use of cloud provider storage resources by selecting the most appropriate combination of failure domains, performance characteristics, resiliency levels, and other parameters to match the specific needs of each user.

[0111] According to some embodiments, a planner service is configured to analyze data collected by the orchestrator and evaluate it in conjunction with the user’s requirements. This analysis enables the planner service to optimize resource allocation and minimize the cost of the services provided by the disk slicing system.

[0112] According to some embodiments, the orchestrator and the benchmark engine included in the disk slicing system are a cloud-based service (SaaS), designated to provide a composable storage system by slicing and composing storage volumes using cloud computing operations.

[0113] According to some embodiments, said cloud computing based slicing and composing is configured to enable optimization of storage resources, wherein the optimization of storage resources is configured to create a storage volume having minimized storage resources not being used. For example, said cloud computing based slicing and composing may be configured to prevent over provisioning of storage resources by dynamically allocate resources in accordance with various needs and constrains.

[0114] According to some embodiments, the disk slicing system may be further configured to undergo an autonomous network profiling procedure designated to analyze and pre -calculate the shared network resources in order to allocate network storage resources to a storage volume / s.

[0115] According to some embodiments, the benchmark engine is designed to monitor system performance over a sufficient duration to provide a reliable analysis under a variety of test conditions. These tests may include assessing the performance of media storage while subjected to heat generated during operation and analyzing internal processes, such as the initiation and behavior of garbage collection mechanisms, to evaluate their impact on system performance. According to some embodiments, the data gathered by the benchmark engine is configured to be stored in a catalog accessible to the orchestrator. This catalog serves as a repository of detailed performance metrics, capacity characteristics, and operational insights, enabling the orchestrator to make informed decisions regarding resource allocation, workload distribution, and optimization of storage resources. By maintaining a comprehensive and up-to-date catalog, the system ensures that the orchestrator can efficiently manage storage configurations and adapt to changing requirements or conditions.

[0116] According to some embodiments, the storage volume may be a storage drive based on various technologies such as a solid-state drive (SSD), storage class memory (SCM), random access memory (RAM), hard disk drive (HHD), etc.

[0117] Although the present invention has been described with reference to specific embodiments, this description is not meant to be construed in a limited sense. Various modifications of the disclosed embodiments, as well as alternative embodiments of the invention will become apparent to persons skilled in the art upon reference to the description of the invention. It is, therefore, contemplated that the appended claims will cover such modifications that fall within the scope of the invention.

[0118] Reference is now made to FIG.5, which constitutes a schematic illustration of a specific utilization of the disk slicing system, according to some embodiments of the invention. Said system may be designated to analyze and categorize storage media based on key performance characteristics such as read / write speeds, latency, endurance, and capacity.

[0119] As shown, in operation 12, a server comprising a flash storage drive is identified by the orchestrator of the disk slicing system.

[0120] According to some embodiments, the capacity of a flash drive is typically identified by the amount of storage space it offers, usually measured in gigabytes (GB) or terabytes (TB). This capacity may be determined by the number of memory cells within the flash drive and the storage technology used. Flash drives also usually have a controller chip that manages the memory cells and handles read and write operations. Additionally, some storage space is reserved for wear-leveling, error correction, and other overhead functions, which may reduce the available storage capacity slightly. A flash drive's capacity is identified by the total amount of data it can store, which is determined by the type of NAND flash memory, the number of memory cells, and the technology used in its construction, wherein the actual usable capacity may be slightly lower due to formatting and overhead.

[0121] In operation 14, the storage disk is locked to a specific read / write ratio and is then configured to undergo a stress-testing procedure to confirm its performance under various conditions.

[0122] According to some embodiments, an (Input-Output) IO operation ratio of the storage disk is configured to be locked in a pre -calculated ratio in order to ensure pre-designated performance of a storage disk.

[0123] According to some embodiments, an IO operation ratio lock may be determined by the following operations:

[0124] • Data Collection: Data on the read and write operations performed by the storage device must be collected. This data is typically collected by monitoring the device's I / O operations using specialized tools or software. The monitoring process spans a specific period, which can range from minutes to days, depending on the level of detail required for the analysis. • Counting Reads and Writes: During the data collection period, the tool or software counts the number of read and write operations that occur on the device. Read operations involve retrieving data from the storage, while write operations involve storing data on the storage.

[0125] The read / write ratio can be calculated using the following formula:

[0126] Read / Write Ratio = (Number of Read Operations) / (Number of Write Operations)

[0127] The formula may be described as a percentage by dividing the number of read operations by the total number of operations (reads + writes) and multiplying by 100:

[0128] Read / Write Ratio (%) = [(Number of Read Operations) / (Number of Read Operations + Number of Write Operations)] x 100

[0129] The read / write ratio describes how often data is being read compared to how often it is being written. A high read / write ratio means that the device is primarily used for reading data, which is common for devices that store and serve data, like hard drives in file servers. Conversely, a high write ratio indicates a device that is frequently written to, like a database server or a device used for constant data logging.

[0130] In operation 16, stress testing is conducted to simulate worst-case scenarios for the storage drive under the predetermined read / write ratio. According to some embodiments, this involves applying a workload designed to push the drive to its operational limits, ensuring it can maintain performance and reliability. This testing assesses the drive's ability to handle intensive read and write operations, heat generation, and other factors that could impact performance during extreme conditions.

[0131] In operation 18, real-time performance metrics are assessed to evaluate how said flash storage device performs under typical operating conditions. Unlike stress testing, which simulates extreme workloads, real-time performance monitoring focuses on understanding the device's behavior during standard operations. This process measures key performance metrics such as read / write speeds, latency, and input / output IOPS. These insights are critical for analyzing performance under normal conditions and optimizing resource allocation within the system.

[0132] According to some embodiments, real-time performance assessment can be conducted by running benchmark tests on the storage media to obtain relevant performance metrics. Most benchmarking tools provide results for sequential and random read / write operations, as well as sequential read / write throughput. These tests can evaluate the storage device's ability to maintain consistent throughput over time, particularly in real-world scenarios. Random access performance can also be assessed, which is especially useful for applications with small random I / O operations, such as databases and virtual machines. Additionally, these tests analyze metrics such as IOPS, latency, and disk usage, while monitoring the device's temperature, as excessive heat can negatively affect performance and longevity.

[0133] If a flash storage disk is nearing capacity, it can lead to performance degradation, hence there is a need to ensure that there is sufficient free space and monitor performance over time. Tracking how performance changes as the storage media ages and fills with data provides valuable insights into its long-term behavior and reliability.

[0134] Assessing the real-time performance of storage media is an ongoing process. Regular monitoring and benchmarking facilitate the identification of any performance issues, enabling corrective actions and storage systems optimization.

[0135] Assessing the real-time performance of a storage volume may be performed by applying a specific pattern designated to cause the drive to perform in a specific way in real-time.

[0136] In operation 20, the disk slicing system 10 is configured to identify the physical location of the flash storage device in order to provide identification of physical location. In operation 22 the classification gathered while utilizing the steps above may be stored in a catalog, which is a data base containing various data pertaining to the performance of the flash drive. According to some embodiments, the disk slicing system 10 monitors performance of the storage disk, thereby enabling the performance guarantee system to optimize network allocation in accordance with the storage disks capabilities.

[0137] Reference is now made to FIG. 6, which schematically illustrates a controller operating as a declarative engine forming a part of the disk slicing system 10, according to some embodiments.

[0138] According to some embodiments, declarative programming is a programming paradigm in which an operator specifies the desired outcome or end result, leaving the system or engine to determine the steps needed to achieve it. This approach emphasizes defining "what" the operator wants to accomplish, rather than detailing "how" to accomplish it, simplifying the implementation process and allowing the underlying system to handle the operational complexity.

[0139] According to some embodiments, a declarative engine integrated into the disk slicing system 10 enables an operator to define desired configurations, policies, or resource allocations in a high- level declarative format. Rather than requiring a detailed, step-by-step procedure for resource allocation, the operator specifies the desired outcome, and the engine automatically interprets and enforces these declarations to achieve the specified result.

[0140] For example, a key characteristic of a declarative engine in the context of cloud storage services is the ability to configure systems through high-level declarations. An operator specifies requirements or configurations using declarative syntax or language, such as defining storage capacity, access controls, replication policies, or other relevant parameters. In cloud storage and data management, declarative approaches are widely employed across various services and tools, streamlining the configuration process by focusing on desired outcomes rather than implementation details.

[0141] According to some embodiments, drive 202 may have a storage volume A which is close to full capacity, whereas drives 204, 206, 208, 210 and 212 which have storage volumes B, C, D, E, and

[0142] F are below capacity and may be used to store additional data. According to some embodiments, the disk slicing system 10 may utilize a declarative engine in order to direct resource allocation to ensure optimal storage disk performance.

[0143] Reference is now made to FIG. 7, which schematically illustrates a specific utilization of the disk slicing system 10, according to some embodiments. As shown, drive 500 comprises at least two volumes 502 and 504, wherein application stacks 506 and / or 508 are designated to process the data flow. According to some embodiments, in such an arrangement, there is a possible scenario in which application stack 508 will consume resources / IOPS from both several storage volumes. For example, if application stack 508 is not restricted it can use resources / IOPS from both volumes 502 and 504, hence disrupting their operation.

[0144] For example, drive 500 may be capable of executing 1.5 million IOPS, volume 502 may be restricted to use drive 500 for 0.5 million IOPS and volume 504 may be restricted to use drive 500 for 1 million IOPS, if application stack 508 further utilizes 0.7 million IOPS, the excesses IOPS will be reduced from the resources provided by volumes 502 & 504, presenting a troublesome scenario for the storage system. According to some embodiments of the invention, the disk slicing system prevents such a scenario, thereby ensuring the maintenance of the performance guarantee of the claimed invention.

[0145] According to some embodiments, the media slicing procedure depicted in FIG. 4 is also capable of managing and control the procedure depicted in FIG. 5.

[0146] According to some embodiments, an application component may be a modular and self- contained unit within a software application that performs a specific function or set of related functions. Application components are designed to be modular, reusable, and interchangeable, contributing to the overall structure and functionality of the software system. These components can be thought of as building blocks that, when combined, create a complete and functional application.

[0147] According to some embodiments, a designated limiting component / s such as Qdisc ingress rate limit and Qdisc root rate limit are configured to determine the bandwidth provided to a certain application component. According to some embodiments, a kernel core component of a Linux OS forming is designated to form a part of the disk slicing system 10. According to some embodiments, the various components forming disk slicing system 10 are configured to operate in a cloud computing SaaS environment providing software services.

Claims

CLAIMS1. A disk slicing system configured to analyze and optimize the performance of storage disks within a distributed network, comprising(i) at least one storage disk; and(ii) at least one benchmark engine; and(iii) at least one orchestrator configured to interact with the said storage disk and said benchmark engine, wherein the at least one orchestrator is configured to direct the at least one benchmark engine to configure a performance profile for said disk by determining an optimal ratio between read / write input / output operations per second (IOPS) and the total capacity of said disk and enforce said performance profile to said disk by undertaking a disk slicing operation thereon.

2. The system of claim 1, wherein the at least one benchmark engine is configured to operate at set time intervals, and wherein the performance profile is updated thereby at said intervals, and wherein the locking ratio relating to said updated performance profile is enforced by the orchestrator.

3. The system of claim 1, wherein the at least one benchmark engine is configured to procure a capacity-designated performance profile that prioritizes storage capacity over performance on the at least one disk.

4. The system of claim 1, wherein the at least one benchmark engine is further configured to procure a performance-designated performance profile that prioritizes the optimization of performance of the at least one disk.

5. The system of claim 1, wherein the at least one orchestrator is configured to create at least one catalog of the at least one disk provided by at least one cloud provider, the at least one catalog including performance profiles and metadata for each storage disk6. The system of claim 5, wherein the at least one catalog comprises at least two catalog entries for each disk provided by the cloud provider, one related to a first configuration that prioritizes capacity and a second related to a second configuration that prioritizes performance, wherein all of the at least two catalog entries are provided by the at least one benchmark engine.

7. The system of claim 1, wherein the at least one orchestrator further determines the pricing and availability of the at least one storage disk and stores this data in the catalog.

8. The system of claim 1, wherein the orchestrator is further configured to monitor pricing and availability of the at least one storage disk.

9. The system of claim 1, wherein analysis provided by the at least one benchmark engine is configured to consider numerous parameters of the at least one storage disk including capacity, type, operation system, and location.

10. The system of claim 1, wherein the at least one orchestrator is further configured to allocate storage disks from the cloud provider based on the performance profile, further incorporating parameters such as instance type, instance size, availability zone, and failure domain to address specific use case requirements or client needs.

11. The system of claim 1, wherein at least one planner service is configured to analyze data collected by the at least one orchestrator, including performance profiles and additional parameters provided to the at least one orchestrator by the at least one benchmark engine, and wherein said at least one planner service is designated to incorporate the analysis provided by the at least one planner service with user-specific requirements.

12. The system of claim 1, wherein the at least one orchestrator and / or at least one benchmark engine is implemented as a cloud-based service (SaaS)configured toprovide a composable storage system by slicing and composing storage disks through cloud computing operations.

13. The system of claim 12, wherein the cloud -based slicing and composing of at least one storage disk is configured to optimize the utilization of storage resources, according to a performance profile provided by the at least one benchmark engine.

14. The system of claim 13, wherein the optimization of storage resources is configured to minimize unused storage capacity.

15. The system of claim 1, wherein the at least one benchmark engine is configured to analyze and monitor the performance of the at least one storage disk in relation to heat generated during its operation.

16. The system of claim 1, wherein the at least one benchmark engine is configured to analyze and monitor the performance of the at least one storage disk in relation to internal processes such as memory management operations.

17. The system of claim 1, wherein the at least one storage disk is solid-state drive (SSD) based.

18. The system of claim 1, wherein the at least one storage disk is storage class memory (SCM) based.

19. The system of claim 1, wherein the at least one storage disk is random access memory (RAM) based.

20. The system of claim 1, wherein the storage disk is hard disk drive (HHD) based.

21. A method for analyzing and optimizing the performance of storage disks within a distributed network, comprising:(i) providing at least one storage disk;(ii) providing at least one benchmark engine;(iii) providing at least one orchestrator configured to interact with the said storage disk and said benchmark engine;(iv) directing, by the at least one orchestrator, the at least one benchmark engine to configure at least one performance profile for the said at least one storage disk, wherein the configuration comprises determining an optimal ratio between read / write input / output operations per second (IOPS) and the total capacity of the said at least one storage disk and enforcing the configured performance profile onto the said at least one storage disk by disk slicing;22. The method of claim 21, wherein the at least one performance profile is configured by the at least one benchmark engine to optimize the use of storage resources in accordance with the intended application.

23. The method of claim 21, wherein the disk slicing allocates storage resources dynamically based on the performance profile.

Images