Method of operating electronic system

Abstract

A method of operating an electronic system is disclosed. The method includes: identifying a physical function provided by an input / output device and obtaining resource amount information of the input / output device; determining a resource allocation rule for each of a plurality of allocation unit groups of an input / output device on the basis of the resource amount information; selecting one or more allocation unit groups among the plurality of allocation unit groups based on a required resource amount and an input / output attribute of the virtual machine and a resource allocation rule determined for each of the plurality of allocation unit groups; generating one or more distribution units for the selected one or more distribution unit groups; and mapping the one or more allocation units to a virtual machine.

Description

[0001] This application claims the benefit of priority from Korean Patent Application No. 10-2023-0118977 filed on September 7, 2023 in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0002] The present disclosure relates to a method of operating a memory controller and a method of operating an electronic system. Background Art

[0003] A storage device, such as a solid state drive (SSD), may include at least one nonvolatile memory for storing data. A host may store data in the nonvolatile memory using the storage device.

[0004] Multiple virtual machines can be run on a host. The storage device can abstract the physical storage space and provide the virtual storage device including the abstracted storage space to the virtual machine. The virtual machine can access the virtual storage device in the same way as accessing the physical storage device. Summary of the invention

[0005] Example embodiments of the present disclosure provide a method of reducing overhead in allocating resources of a storage device to multiple virtual storage devices.

[0006] Example embodiments of the present disclosure provide a method of providing stable performance for each of a plurality of virtual storage devices.

[0007] According to an example embodiment of the present disclosure, a method for operating an electronic system includes: identifying a physical function provided by an input / output device and obtaining resource quantity information of the input / output device; determining a resource allocation rule for each of a plurality of allocation unit groups of the input / output device based on the resource quantity information; selecting one or more allocation unit groups among the plurality of allocation unit groups based on the required resource quantity and input / output attributes of a virtual machine and the resource allocation rule determined for each of the plurality of allocation unit groups; generating one or more allocation units for the selected one or more allocation unit groups; and mapping the one or more allocation units to the virtual machine. According to an example embodiment of the present disclosure, a method for operating an electronic system includes: generating a plurality of allocation unit groups, each having a different resource allocation rule, in response to a resource configuration request; receiving a plurality of allocation unit generation requests, each allocation unit generation request respectively specifying one of the plurality of allocation unit groups, mapping a plurality of allocation units included in the plurality of allocation unit groups to a plurality of input / output queues provided by a storage controller, and allocating throughput to each of the plurality of allocation units based on the resource allocation rule.

[0008] According to an example embodiment of the present disclosure, a method for operating an electronic system includes: identifying multiple physical functions provided by an input / output device and obtaining resource quantity information of the input / output device; allocating storage capacity and quality of service (QoS) to each of the multiple physical functions; providing a resource configuration request to determine a resource allocation rule for each of the multiple physical functions based on the storage capacity and QoS allocated to each of the multiple physical functions; selecting a physical function for which an allocation unit is generated based on the resource quantity requested by a virtual machine and the resource allocation rule configured for each of the multiple physical functions; generating one or more allocation units of the selected physical function by providing an allocation unit generation request to the selected physical function; and mapping the one or more allocation units to the virtual machine. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other aspects, features and advantages of the present disclosure will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings.

[0010] Figure 1 is a diagram illustrating an electronic system according to an example embodiment of the present disclosure.

[0011] Figure 2 is a diagram illustrating a method of operating an electronic system according to an example embodiment of the present disclosure.

[0012] Figure 3 is a diagram illustrating a packet structure of a resource configuration request according to an exemplary embodiment of the present disclosure.

[0013] Figure 4 is a diagram illustrating a table representing resource allocation information for each allocation unit group according to an exemplary embodiment of the present disclosure.

[0014] Figure 5 is a diagram illustrating a mapping table of virtual machines and allocation units according to an exemplary embodiment of the present disclosure.

[0015] Figure 6 is a diagram illustrating a table indicating a resource allocation amount for each allocation unit group according to an exemplary embodiment of the present disclosure.

[0016] Figure 7 is a diagram illustrating a method of scheduling an input / output request of an electronic system according to an example embodiment of the present disclosure.

[0017] Figure 8 is a diagram illustrating a request scheduling method of a storage device according to an example embodiment of the present disclosure.

[0018] Fig. 9 is a flowchart illustrating a request scheduling method of a storage device according to an example embodiment of the present disclosure.

[0019] FIG. 10A to FIG. 10C is a diagram illustrating an electronic system according to an example embodiment of the present disclosure.

[0020] Fig.11 is a diagram illustrating an electronic system according to an example embodiment of the present disclosure.

[0021] Fig.12 is a diagram illustrating a server system to which an electronic system is applied according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] Hereinafter, embodiments of the present disclosure will be described as follows with reference to the accompanying drawings. Unless the context so dictates, although the drawings described herein may be referred to using language such as "one embodiment" or "a particular embodiment," these drawings and their corresponding descriptions are not intended to be mutually exclusive with other drawings or descriptions. Thus, particular aspects from a particular drawing may be the same as particular features in other drawings, and / or a particular drawing may be a different representation or a different portion of a particular exemplary embodiment.

[0023] Unless the context clearly indicates otherwise, as used herein, the singular forms are intended to include the plural forms as well.As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items and may be abbreviated as " / ".

[0024] Ordinal numbers such as "first", "second", "third", etc. may simply be used as labels for specific elements, steps, etc. to distinguish such elements, steps, etc. from each other. Terms that are not described using "first", "second", etc. in the specification may still be referred to as "first" or "second" in the claims. In addition, a term referenced with a specific ordinal number (e.g., "first" in a specific claim) may be described elsewhere with a different ordinal number (e.g., "second" in the specification or another claim).

[0025] Figure 1 is a diagram illustrating an electronic system according to example embodiments.

[0026] Reference Figure 1 , the electronic system 10 may include a host 100 , a storage device 200 , and a bus 50 connecting the host 100 to the storage device 200 .

[0027] The host 100 may be configured as a computing device such as a desktop computer, a laptop computer, a network server, or a mobile device. The host 100 may include an application processor, a microprocessor, a central processing unit (CPU), a processor core, a multi-core processor, a multiprocessor, an application specific integrated circuit (ASIC), and a field programmable gate array (FPGA).

[0028] The storage device 200 may store data in response to a request from the host 100. For example, the storage device 200 may include at least one of a solid state drive (SSD), an embedded memory, and a removable external memory.

[0029] The storage device 200 may include a storage controller 210 and a nonvolatile memory device (NVM) 220 . The nonvolatile memory device 220 may be configured as a storage medium storing data received from the host 100 , and may include, for example, a flash memory. The storage controller 210 may control the nonvolatile memory device 220 .

[0030] When the storage device 200 is configured as an SSD, the storage device 200 may be configured as a device compliant with the Non-Volatile Memory Express (NVMe) standard. When the storage device 200 is configured as a device compliant with the NVMe standard, the bus 50 may be configured as a Peripheral Component Interconnect Express (PCIe) bus.

[0031] The host 100 may provide a plurality of isolated operating environments called virtual machines VM. Virtualization may be a mechanism for integrating the workloads of a plurality of virtual machines VM on a single physical machine while maintaining the isolated workload of each of the plurality of virtual machines VM, and may be widely used in cloud servers, etc. The host 100 may provide a virtual machine VM, and may also provide other types of isolated operating environments (such as containers).

[0032] Reference Figure 1 The host 100 can generate a plurality of virtual machines VM1, VM2 and VM3 run by a host operating system (OS) by running a virtual machine manager VMM (110) (or software called a hypervisor). The virtual machine manager 110 can be configured as a component of the host operating system 120 and can be provided by an application running on the host operating system 120.

[0033] The virtual machine manager 110 may abstract physical resources including a processor, a memory, and an input / output device, and may provide the abstracted physical resources as virtual devices including a virtual processor, a virtual memory, and a virtual input / output device to virtual machines VM1, VM2, and VM3. Each of the virtual machines VM1, VM2, and VM3 may use the virtual device to run a client operating system. On the client operating system, one or more applications may be run.

[0034] The host 100 may provide a function that may allow the virtual machines VM1 , VM2 , and VM3 to directly access an input / output device such as the storage device 200 , rather than accessing the input / output device through software simulating the input / output device.

[0035] For example, a single root input / output virtualization (SR-IOV) architecture may support providing multiple virtual input / output devices for a physical input / output device, so that virtual machines VM1, VM2, and VM3 may share the physical input / output device. For example, an endpoint device supporting SR-IOV may support one or more physical functions PF corresponding to an input / output port, and each physical function may support multiple virtual functions VF. A physical function PF is a PCI (e.g., PCIe) function that can be discovered, managed, and manipulated in the same manner as any other PCI device. The physical function may be a PCI function that supports the SR-IOV capability defined by the SR-IOV specification. Each of the one or more physical functions PF may appear as a different physical device on the host 100. The host 100 may access the one or more physical functions PF independently and individually.

[0036] A virtual function VF is a virtualized instance of a physical function PF that shares physical resources with a physical function PF. In a single root input / output virtualization (SR-IOV) architecture, a physical function PF may have multiple virtual functions, and multiple virtual functions may each be assigned to a virtual machine to allow sharing of the physical PF. A virtual function may be dedicated (mapped) to a virtual machine VM to allow each VM to use the virtual function as if the virtual function were a hardware device. For example, a VM may issue an I / O command to a virtual function VF assigned to a VM, and the associated physical function PF that references the virtual function mapping may execute the I / O command on behalf of the virtual function VF, and return an I / O response to the corresponding (assigned) virtual function VF to complete the I / O command.

[0037] When the electronic system 10 is configured as a large-scale server such as a cloud server, the host 100 may provide thousands or more virtual machines. The storage device 200 may support a scalable IOV architecture (S-IOV) to provide virtualized resources to multiple virtual machines on a granular basis.

[0038] Compared with SR-IOV, S-IOV architecture can configure virtual devices more flexibly. In addition, among the tasks for virtual devices, tasks that are considered important in performance can be directly mapped to input / output devices, and tasks that are considered relatively unimportant in performance can be simulated by software in the host operating system 120.

[0039] In order to virtualize the resources of the storage device 200 with a larger granularity, an allocation unit called an allocable device interface (ADI) allocation unit (which may be referred to as an allocation unit ADI hereinafter) may be defined in the S-IOV architecture. The allocation unit ADI may be an isolated allocation unit and may refer to a group of resources that may be allocated, configured, and organized. For example, an allocation unit ADI for the storage controller 210 may be a group of input / output queues Q, each of which is associated with a namespace of the storage device. That is, each allocation unit ADI may be mapped to one or more input / output queues Q.

[0040] The allocation unit ADI may be managed in the physical function PF. Since the allocation unit ADI may be directly accessed by the virtual machine VM, the allocation unit ADI may be similar to the virtual function VF of the SR-IOV architecture.

[0041] However, in contrast to the SR-IOV architecture in which the virtual function VF has a request identifier (RID) that is different from the RID of the physical function PF, the allocation unit ADI may share the RID of the physical function PF. In order to enable requests of allocation units ADI that share a RID to be distinguished, a process address space identifier (PASID) added to the transaction layer packet may be assigned to each allocation unit ADI. Since the PASID can support 20-bit identifiers, the PASID can provide more identifiers than a RID that can support 16-bit identifiers. Therefore, the S-IOV architecture can support a greater number of allocation units ADI than the number of virtual functions that can be supported in the SR-IOV architecture.

[0042] The storage device 200 may be virtualized using multiple virtual devices VDEV. Multiple virtual devices VDEV may be exposed to software on virtual machines VM1, VM2, and VM3. For example, similar to the virtual functions VF supported in the SR-IOV architecture, multiple virtual devices VDEV may be mapped to corresponding virtual machines (such as virtual machines VM1, VM2, and VM3).

[0043] Multiple virtual devices VDEV can be configured using multiple allocation units ADI. For example, the host operating system 120 can map one or more of the multiple allocation units ADI generated on the physical function PF to the virtual device VDEV. In the following description, the mapping of the allocation unit ADI to the virtual device VDEV corresponding to the virtual machine VM can be referred to as mapping the allocation unit ADI to the virtual machine VM. In the figure, the manager 122 can map the allocation unit ADI to the virtual machine VM in the host operating system.

[0044] The hypervisor 122 may be a software component running in the host operating system 120, and according to an example embodiment, the hypervisor 122 may be integrated into the virtual machine manager 110. However, in an example embodiment, the hypervisor 122 is not limited to being integrated into the virtual machine manager 110, and the hypervisor 122 may be implemented as a virtual machine (such as a secure core).

[0045] The physical function PF can dynamically generate, manage, and delete allocation units ADI. When the host 100 provides more than thousands of virtual machines VM, the physical function PF can generate and manage more than thousands of allocation units ADI to map the allocation units ADI to multiple virtual machines VM. The physical function PF can manage each allocation unit ADI with various resource allocation amounts. For example, the physical function PF can manage multiple allocation units ADI that may have different storage capacities or different quality of service (QoS).

[0046] QoS may refer to the ability of the storage controller 210 to prioritize data flows from different allocation units ADI and ensure stable and consistent performance of data input / output operations for each device resource in the different allocation units ADI. For example, QoS indicators may include input / output operations per second (IOPS), response time, throughput, etc.

[0047] In order for the host operating system 120 to allocate resources to multiple allocation units ADI based on the amount of resources required by each of the multiple virtual machines VM, an efficient method of allocating resources to multiple allocation units ADI may be required. In addition, in order to provide the QoS required by the virtual machines VM, it may be desirable to effectively schedule requests for multiple allocation units ADI.

[0048] According to an example embodiment, the host 100 may generate a plurality of allocation unit groups in which different pieces of resource allocation information are defined. The resource allocation information may include storage capacity, QoS, etc. In order to allocate resources to the allocation units ADI, the host 100 may specify allocation unit groups in which the allocation units ADI may be respectively included. When allocating the units ADI, the resources may be allocated based on the resource allocation information defined in the allocation unit group. The host 100 may configure the virtual device VDEV using one or more allocation units ADI.

[0049] According to an exemplary embodiment, the host 100 can allocate resources to the allocation unit ADI by selecting a resource allocation type based on predetermined resource allocation information instead of specifying each resource allocation amount of the allocation unit ADI. In addition, since a plurality of allocation units ADI can be classified according to the resource allocation information, the virtual device VDEV can be easily configured by combining the allocation units ADI classified according to the resource amount required by the virtual machine VM. Therefore, the overhead for allocating resources to the allocation unit ADI and configuring the virtual device VDEV can be reduced.

[0050] The storage device 200 can effectively schedule the request for the multiple allocation units ADI from the host 100 by scheduling the request based on the allocation unit group to which the multiple allocation units ADI belong, and can provide stable performance for the multiple allocation units ADI to the virtual machine.

[0051] In the following description, reference will be made to Figures 2 to 9 The operation of the electronic system according to example embodiments is described in more detail.

[0052] Figure 2 is a diagram illustrating a method of operating an electronic system according to an example embodiment. Although embodiments of the method are not limited to Figure 1 electronic systems, but this method will refer to Figure 1 For example, although the description of the method may identify host 100, this is for ease of description and the method may be used in any other device except Figure 1 is executed on a host other than host 100.

[0053] exist Figure 2 , transactions between a virtual machine VM, a hypervisor 122, and a physical function PF included in an electronic system are shown. The virtual machine VM may be a reference Figure 1 One of the virtual machines VM1, VM2, and VM3 described above, and the hypervisor 122 and the physical function PF may correspond to the reference Figure 1 The manager 122 and the physical function PF are described.

[0054] In operation S11 , the manager 122 may identify a physical function PF provided by the storage device 200 by enumerating input / output devices connected to the host 100 .

[0055] The manager 122 may also obtain resource quantity information of the physical function PF. The resource quantity information may include the storage capacity of the storage device 200, the maximum number of allocation units ADI supported by the physical function PF, the maximum number of input / output queues, the maximum number of namespaces, the bandwidth of the storage device 200, the maximum quality of service (QoS), etc.

[0056] Operation S11 may be triggered when the storage device 200 providing the physical function PF is initialized.

[0057] In operation S12, the manager 122 may generate a plurality of allocation unit groups (ADI groups), and may determine a resource allocation rule for each of the plurality of allocation unit groups. As a first example, the manager 122 may determine the amount of resources to be allocated to “each allocation unit ADI to be generated in each of the plurality of allocation unit groups”. As a second example, the manager 122 may determine the amount of resources to be allocated to each allocation unit group and a resource allocation policy for each allocation unit group, and may allow the physical function PF to determine the amount of resources to be allocated to each allocation unit ADI according to the configured resource allocation policy.

[0058] In operation S13, the manager 122 may provide a resource configuration request (e.g., a resource setting command (CMD)) to the physical function PF to determine a resource allocation rule for a plurality of allocation unit groups. The resource configuration request may be a request for notifying the physical function PF of the amount of resources to be allocated for a plurality of allocation unit groups defined in the manager 122, and the resources owned (reserved) by the physical function PF may be actually allocated to the allocation unit ADI through an allocation unit generation command, wherein the physical function PF may perform the actual allocation in response to the allocation unit generation command.

[0059] In an example embodiment, the resource configuration request may be provided in the format of a set feature command. The set feature command may refer to a command for changing the configuration of the storage device 200, and the set feature command may be a management command.

[0060] In other embodiments, the resource configuration request is not limited to being provided in the format of the set feature command. For example, the resource configuration request may be provided by a command in various formats defined in a standard such as NVMe, and may be provided by a sideband signal sent via a dedicated line or pin allocated separately from a command line or pin.

[0061] In operation S14, the manager 122 may determine the amount of resources required by the virtual machine VM. The electronic system 10 may provide the resources allocated according to the policy to the user who uses the virtual machine VM. For example, the electronic system 10 may provide cloud storage with differentiated storage capacity and QoS according to the usage fee collected from the user. Therefore, the storage capacity and QoS required by each of the virtual machines VM1, VM2, and VM3 may be determined separately.

[0062] In an example embodiment, the manager 122 may also determine the input / output attributes required by the virtual machine VM. For example, the input / output attributes may include write-intensive attributes, read-intensive attributes, and mixed performance attributes.

[0063] In operation S15 , the manager 122 may request the physical function PF to generate one or more allocation units ADI based on the resource amount and input / output attributes determined in operation S14 (eg, provide an ADI generation CMD to the physical function PF).

[0064] For example, the manager 122 may select one or more allocation unit groups from among the multiple allocation unit groups based on the amount of resources and input / output attributes required by the virtual machine VM and the resource allocation rule of each allocation unit group, and may determine the number of allocation unit ADIs to be generated for the selected one or more allocation unit groups. In addition, the manager 122 may provide an identifier for specifying the allocation unit ADI and an allocation unit generation request for the selected one or more allocation unit groups to the physical function PF.

[0065] In operation S16, the physical function PF may generate one or more allocation units ADI in response to the allocation unit generation request, and may allocate resources to the generated allocation units ADI based on a resource allocation rule configured in an allocation unit group to which the generated allocation units ADI belong.

[0066] For example, the physical function PF may determine the storage capacity of a logical storage space (eg, namespace) provided by the storage device 200 in response to a resource configuration request, and may allocate the storage capacity to the allocation unit ADI by mapping the logical storage space to the allocation unit ADI.

[0067] The physical function PF may allocate QoS to the allocation unit ADI in response to the resource configuration request. Allocating QoS to the allocation unit ADI may include: allocating resources of sufficient size for ensuring QoS to the allocation unit ADI, and scheduling input / output requests corresponding to the allocation unit ADI based on QoS, thereby ensuring QoS.

[0068] In operation S17 , the hypervisor 122 may map the virtual machine VM and the one or more generated allocation units ADI.

[0069] In the following description, reference will be made to Figures 3 to 6 A method of managing resource allocation amounts of a plurality of allocation units ADI in the host 100 is described in more detail.

[0070] Figure 3 is a diagram illustrating a packet structure of a resource configuration request according to an example embodiment.

[0071] Reference Figure 3The packet 300 of the resource allocation command provided by the manager to the storage device 200 may include a header, an identifier of each allocation unit group, and a resource allocation rule for the allocation unit group. The resource allocation rule for the allocation unit group may include the storage capacity, write throughput, write latency, read throughput, and read latency of the allocation unit ADI to be generated in the allocation unit group. The write throughput, write latency, read throughput, and read latency may be included in the QoS.

[0072] Figure 3 An example of a packet including resource allocation information is shown, wherein the resource allocation information specifies the write throughput and the read throughput of the allocation group separately, but in other embodiments, the resource allocation information may specify a single throughput applied to write requests and read requests. Similarly, in some embodiments, the resource allocation information may specify the write latency and the read latency separately, or may also specify a single latency applied to write requests and read requests.

[0073] The amount of resources used for allocation, such as identifier range, storage capacity, write throughput, write latency, read throughput, and read latency, may be commonly applied to each allocation unit ADI included in the allocation unit group.

[0074] In an example embodiment, the resource allocation command may include resource allocation information for each of the plurality of allocation unit groups. That is, the manager may request to determine resource allocation information for each of the plurality of allocation unit groups by providing the resource allocation command to the physical function PF.

[0075] According to an example embodiment, instead of individually determining the amount of resources to be allocated to each allocation unit ADI included in an allocation unit group, the manager may determine resource allocation information for each allocation unit group, and may map the allocation unit ADI to the allocation unit group when the allocation unit ADI is generated, thereby determining the amount of resources to be allocated to the allocation unit ADI. The overhead of the host 100 and the bus 50 when allocating resources to a plurality of allocation units ADIs may be reduced.

[0076] When resources are uniformly allocated in the allocation units ADI included in the allocation unit group, the resources allocated to the virtual machine VM can be unitized. For example, according to the required resource amount and input / output attributes of the virtual machine VM, it can be easily determined how many allocation units ADI included in which allocation unit group can be allocated to the virtual machine VM. Therefore, the manager 122 can flexibly allocate resources to the virtual machine VM, and the overhead for determining the allocation unit ADI to be mapped to the virtual machine VM can be reduced.

[0077] The manager 122 may store resource allocation information for each allocation unit group in order to determine the amount of resources for allocation and input / output properties of the newly generated allocation units ADI.

[0078] Figure 4 is a diagram illustrating a table representing resource allocation information for each allocation unit group in a plurality of groups according to an example embodiment.

[0079] exist Figure 4 In the example of , the individual allocation units ADI included in the allocation unit group may have the same storage capacity and performance as each other, and the first table TABLE1 may include the storage capacity and performance of each individual allocation unit ADI. Figure 4 , the first table TABLE1 may include allocation unit group identifiers (ADIG IDs), individual storage capacities, individual read throughputs, individual read latencies, individual write throughputs, and individual write latencies.

[0080] The first table TABLE1 may be stored in a memory of the host 100 and may be referenced by the manager 122. For example, the manager 122 may refer to the first table TABLE1 to generate one or more allocation units ADI based on the amount of resources and input / output attributes required by the virtual machine VM. Specifically, the manager 122 may classify the allocation units ADI by defining allocation unit groups. In addition, based on the amount of resources and input / output attributes required by the virtual machine VM, by determining the allocation unit group to which the newly generated allocation unit ADI belongs and determining the number of the newly generated allocation units ADI, the resources of the storage device 200 may be easily shared with the virtual machine VM.

[0081] For example, the amount of resources (such as storage capacity and QoS) required by each virtual machine VM may be different, and each virtual machine VM may have various input / output attributes (such as read-intensive attributes, write-intensive attributes, mixed attributes, etc.).

[0082] Different storage capacity and QoS may be defined for each allocation unit group, and the storage capacity and QoS may define the input / output attributes and the amount of resources provided by the allocation unit ADI. For example, when the write throughput of the allocation unit ADI is higher than the read throughput, the write-intensive attribute may be included, when the read throughput is higher than the write throughput, the read-intensive attribute may be included, and when the write throughput is the same as the read throughput, the mixed attribute may be included. The manager 122 may select at least one of the allocation unit groups having various input / output attributes based on the input / output attributes required by the virtual machine VM, and may determine the number of allocation units ADI to be newly generated for the selected allocation unit group based on the amount of resources required by the virtual machine VM.

[0083] According to an example embodiment, since a plurality of allocation units ADI can be classified into several allocation unit groups, the manager 122 can easily allocate the allocation units ADI to provide the required resource amount and input / output attributes to the virtual machine VM. Therefore, the overhead required for resource allocation can be reduced.

[0084] The host 100 may also store a mapping table between the virtual machines and the allocation units ADI.

[0085] Figure 5 is a diagram illustrating a mapping table of virtual machines and allocation units according to an example embodiment.

[0086] Reference Figure 5 The second table TABLE2 may include an identifier VM ID of each virtual machine VM and an allocation unit identifier ADI ID of each allocation unit ADI mapped to the corresponding virtual machine VM. Figure 5 , each virtual machine VM can be mapped to one or more allocation units ADI.

[0087] The second table TABLE2 may be stored in the memory of the host 100 and may be referenced by the hypervisor 122. For example, when the hypervisor 122 newly maps the allocation unit ADI to the virtual machine VM, the hypervisor 122 may identify the allocation unit ADI not mapped to other virtual machines VM by referring to the second table TABLE2.

[0088] exist Figure 4 In the example of , the manager 122 defines the resource allocation amount of the individual allocation unit ADI for each allocation unit group, but the embodiment is not limited thereto. For example, the manager 122 may determine the total resource allocation amount for each allocation unit group, and the storage device 200 may determine the resource allocation amount of each allocation unit ADI based on the total resource allocation amount for each allocation unit group, and may allocate resources to the individual allocation unit ADI.

[0089] Figure 6 is a diagram illustrating a mapping table indicating a total resource allocation amount for each allocation unit group according to an example embodiment.

[0090] Reference Figure 6 , the third table TABLE3 may include the total resource allocation amount of each allocation unit ADI group and the allocation unit group identifier (ADIG ID). For example, the third table TABLE3 may include the total storage capacity and total throughput of the allocation unit group. In an example embodiment, the third table TABLE3 may further define the input / output properties of the allocation unit group by further defining the ratio between the read throughput and the write throughput (e.g., the ratio of the read throughput and the write throughput) and / or the ratio between the storage capacity and the throughput of each allocation unit ADI.

[0091] According to an example embodiment, the manager 122 may provide a resource configuration request including a total storage capacity and a total throughput of an allocation unit group to the physical function PF, and may optionally provide a resource configuration request including a ratio of a read throughput and a write throughput to the physical function PF. The physical function PF may allocate resources to individual allocation units ADI included in the allocation unit group based on the total storage capacity, the total throughput, and one or more policies of the allocation unit group.

[0092] As a first example of a policy, the physical function PF may determine and allocate the same storage capacity and throughput to each allocation unit ADI within the range of the total storage capacity and total throughput of the allocation unit group. In the example where the total storage capacity of the first allocation unit group (ADI group ID=1) is 6GB and the total throughput is 300MB / s, the same storage capacity of 1GB and throughput of 50MB / s may be allocated to the allocation units ADI included in the first allocation unit group (ADI group ID=1). Based on the total storage capacity and total throughput defined in the allocation unit group and the storage capacity and throughput defined in the individual allocation units ADI of the allocation unit group, the maximum number of individual allocation units ADI included in the allocation unit group may be determined. When the maximum number of individual allocation units ADI is about to be exceeded (such as because a new generation request for generating a new individual allocation unit ADI is received), the physical function PF may reject the generation request.

[0093] As a second example of a strategy, the physical function PF may allocate storage capacity and throughput to each allocation unit ADI such that the ratio between storage capacity and throughput of each allocation unit ADI may be constant within the range of the total storage capacity and total throughput of the allocation unit group.

[0094] In the second example of the policy, the storage capacity of the allocation unit ADI can be defined separately. That is, the allocation unit generation request may include the storage capacity to be allocated to the allocation unit ADI and the identifier of the allocation unit ADI and the identifier of the allocation unit group. For example, in the example where the total storage capacity of the second allocation unit group (ADI group ID=2) is 20GB and the total throughput is 800MB / s, when the physical function PF receives a resource configuration request from the manager 122 to generate a first allocation unit ADI belonging to the second allocation unit group and having a storage capacity of 1GB, the physical function PF may allocate the storage capacity of 1GB and the throughput of 40MB / s (for example, 20 / 800=1 / 40) to the first allocation unit ADI. In addition, when the physical function PF receives a resource configuration request from the manager 122 to generate a second allocation unit ADI having a storage capacity of 2GB, the physical function PF may allocate the storage capacity of 2GB and the throughput of 80MB / s (for example, 20 / 800=2 / 80) to the second allocation unit ADI.

[0095] When a ratio between the read throughput and the write throughput is defined for the allocation unit group, the physical function PF may distribute the throughput allocated to the allocation unit ADI into the read throughput and the write throughput of the allocation unit ADI.

[0096] The storage capacity allocated to the allocation unit ADI provided by the storage device 200 may be obtained by determining the size of the namespace mapped to the allocation unit ADI. In addition, the throughput allocated to the allocation unit ADI provided by the storage device 200 may be obtained by allocating one or more input / output queues Q of the storage controller 210 to each allocation unit ADI based on the throughput of the allocation unit ADI and scheduling input / output requests for the allocation unit ADI.

[0097] In the following description, reference will be made to Figure 7 and Figure 8 A method of scheduling input / output requests of individual allocation units ADI based on their throughput is described in more detail.

[0098] Figure 7 is a diagram illustrating a method of scheduling input / output requests of an electronic system according to example embodiments.

[0099] Figure 7 An electronic system (such as, for example, Figure 1 Although the method is described with reference to Figure 1The electronic system 10 is described, but the embodiment is not limited thereto. The host 100 may include a plurality of virtual machines VM and a manager 122. The storage controller 210 may provide a physical function PF, and the physical function PF may provide a plurality of allocation units ADI. The manager 122 may request the physical function PF to generate an allocation unit ADI, and map the allocation unit ADI to the virtual machine VM.

[0100] As reference Figures 3 to 6 As described above, the physical function PF may provide a plurality of allocation unit groups ADIG1, ​​ADIG2, and ADIG3 having different resource allocation amounts and different input / output attributes in response to a resource configuration request from the manager 122. In addition, the physical function PF may generate an allocation unit ADI included in one of the plurality of allocation unit groups ADIG1, ​​ADIG2, and ADIG3 in response to an allocation unit generation request from the manager 122.

[0101] One or more input / output queues Q may be allocated to each allocation unit ADI. Furthermore, one or more allocation units ADI may be allocated to each of a plurality of virtual machines VM.

[0102] When the physical function PF provides a plurality of allocation units ADI, a plurality of input / output requests may be queued in a plurality of input / output queues Q allocated to the plurality of allocation units ADI. When the plurality of input / output requests are not effectively scheduled in the storage controller 210, the input / output requests may not be processed in a timely manner, and it may be difficult to guarantee the QoS allocated to the allocation unit ADI.

[0103] According to an example embodiment, the storage controller 210 may use a multi-polling method to schedule input / output requests queued in a plurality of input / output queues Q. Specifically, the storage controller 210 may provide request processing opportunities to a plurality of allocation unit groups ADIG1, ​​ADIG2, and ADIG3 in a polling method, and the request processing opportunity provided to one allocation unit group may be provided to a plurality of allocation units ADI included in one allocation unit group in a polling method.

[0104] The request processing opportunity of one allocation unit ADI may be provided to the input / output queue Q mapped to the one allocation unit ADI, and the input / output request queued to the input / output queue Q having the request processing opportunity may be processed. When one allocation unit ADI has a plurality of input / output queues Q, the request processing opportunity provided to one allocation unit ADI may be provided to the plurality of input / output queues Q in a polling method. Here, the polling method may include a simple polling method RR and a weighted polling method WRR.

[0105] In the following description, reference is made to Figures 8 to 9A request scheduling method of a storage controller according to a multi-polling method is described in more detail.

[0106] Figure 8 is a diagram illustrating a request scheduling method of a storage device according to example embodiments.

[0107] Figure 8 A request scheduling method according to a multi-polling method is shown with reference to an example, in which the physical function PF has a first allocation unit group ADIG1 and a second allocation unit group ADIG2, the first allocation unit group ADIG1 has a first allocation unit ADI1 and a second allocation unit ADI2, the second allocation unit group ADIG2 has a third allocation unit ADI3 and a fourth allocation unit ADI4, and the first to fourth allocation units ADI1, ADI2, ADI3, ADI4 each have an input-output queue Q.

[0108] A plurality of requests A, B, C, D, L, M, N, P, R, S, X, Y, and Z may be queued to input / output queues Q of first to fourth allocation units ADI1, ADI2, ADI3, and ADI4. The storage controller 210 may perform request scheduling to determine an order in which the plurality of requests are processed according to the multi-polling method.

[0109] exist Figure 8 In the example in , using the weighted round robin WRR method, a request processing opportunity may be provided to one of the first allocation unit group ADIG1 and the second allocation unit group ADIG2. The weights of the first allocation unit group ADIG1 and the second allocation unit group ADIG2 may be "2" and "1", respectively. During a period in which both the first allocation unit group ADIG1 and the second allocation unit group ADIG2 are provided with at least one request processing opportunity, the first allocation unit group ADIG1 may have two request processing opportunities, and the second allocation unit group ADIG2 may have one request processing opportunity.

[0110] In a first example, the first allocation unit group ADIG1 and the second allocation unit group ADIG2 may have a weight determined according to a ratio of QoS assigned to the first allocation unit group ADIG1 and the second allocation unit group ADIG2. In a second example, the weight may be determined according to a ratio of throughput between the first allocation unit group ADIG1 and the second allocation unit group ADIG2. In a third example, in order to determine the weight, in addition to the throughput ratio, delay may be further considered. In some examples, one or more of the QoS ratio, the throughput ratio, and the delay may be used to determine the weight.

[0111] A simple round-robin RR method may be used to provide a request processing opportunity to one of the first allocation unit ADI1 and the second allocation unit ADI2, and a simple round-robin RR method may be used to provide a request processing opportunity between the third allocation unit ADI3 and the fourth allocation unit ADI4. However, when the throughputs between the first allocation unit ADI1 and the second allocation unit ADI2 and the third allocation unit ADI3 and the fourth allocation unit ADI4 are different, a weighted round-robin WRR method may be used to provide a request processing opportunity.

[0112] In the first cycle Cycle1 according to the multi-polling method, the first allocation unit group ADIG1 may have two request processing opportunities, and the second allocation unit group ADIG2 may have one request processing opportunity. The two request processing opportunities of the first allocation unit group ADIG1 may be provided to the first allocation unit ADI1 and the second allocation unit ADI2 in sequence. In addition, one request processing opportunity of the second allocation unit group ADIG2 may be provided to the third allocation unit ADI3. For example, in the first cycle Cycle1, requests may be processed in the order of ALP.

[0113] In the second cycle Cycle2, two request processing opportunities of the first allocation unit group ADIG1 can be provided to the first allocation unit ADI1 and the second allocation unit ADI2 in sequence. In addition, one request processing opportunity of the second allocation unit group ADIG2 can be preferentially provided to the fourth allocation unit ADI4. For example, in the second cycle Cycle2, requests can be processed in the order of BMX.

[0114] In order to process the requests queued in the plurality of input / output queues Q without delay according to the multi-polling method, the order of the request processing opportunities among the plurality of input / output queues Q and also the size of the data that can be input / output for the request processing opportunities may also be determined. For example, the size of the data that can be input / output may be controlled by a credit-based method.

[0115] Fig. 9 is a flowchart illustrating a request scheduling method of a storage device according to an example embodiment.

[0116] In operation S21, the storage controller 210 may determine the size of the time window. The time window may be a credit-based method and may refer to a period during which credits are newly allocated when requests queued in the input / output queue Q are scheduled.

[0117] In operation S22, the storage controller 210 may determine the throughput of each allocation unit group. For example, the storage controller 210 may determine the throughput of each allocation unit group based on the throughput of currently generated allocation units ADI of each allocation unit group.

[0118] In operation S23 , the storage controller 210 may allocate credits for each allocation unit group per time window.

[0119] A credit may refer to an opportunity to process an input / output request for a determined amount of data. The storage controller 210 may allocate multiple credits to each allocation unit group based on the size of the allocation unit group's throughput and the time window. For example, when the allocation unit group's throughput is 300MB / s and the time window size is 10ms, a number of credits corresponding to 3MB of data may be allocated to each allocation unit group.

[0120] In operation S24 , the storage controller 210 may process a request for each allocation unit group within the allocated credit range.

[0121] As previously described, the storage controller 210 may schedule multiple requests using a multi-polling method. When processing a request, the storage controller 210 may deduct the credit of the allocation unit group associated with the request based on the size of the input / output data associated with the request. In one embodiment, when processing a request queued in an input / output queue, the storage controller 210 may deduct a determined number of credits from the allocated credits for the allocation unit group associated with the input / output queue based on the size of the input / output data corresponding to the processed request. In addition, even if a request processing opportunity is provided to an input / output queue, the storage controller 210 may skip request processing for the input / output queue when the credits of the allocation unit group associated with the input / output queue are exhausted.

[0122] According to an example embodiment, the storage controller 210 may use a multi-polling method to schedule requests queued in multiple input / output queues Q, and may use a credit-based control method to control data throughput so that the QoS defined in each of the allocation unit groups ADIG1, ​​ADIG2 and ADIG3 and the QoS assigned to the individual allocation units ADI can be guaranteed.

[0123] As reference Figures 7 to 9 As described above, the memory controller 210 may queue the input / output requests in the input / output queue Q without distinguishing between the read request and the write request, and the read request and the write request may be scheduled without distinguishing from each other. However, example embodiments thereof are not limited thereto.

[0124] As described above, the read throughput and the write throughput may be configured in the allocation unit ADI. According to an example embodiment, in order to ensure the read throughput and the write throughput of each allocation unit ADI, the input / output queue Q mapped to the allocation unit ADI may include a read request queue and a write request queue. In addition, according to the read throughput and the write throughput of the allocation unit ADI, by changing the number of read request queues and the number of write request queues included in the input / output queue Q, or providing request processing opportunities to the read request queue and the write request queue by a weighted round-robin WRR method, the read throughput and the write throughput of each allocation unit ADI may be guaranteed.

[0125] exist Figures 1 to 9 In the example of the storage device 200 providing a single physical function PF, it is shown. However, the exemplary embodiment thereof is not limited thereto. In the following description, reference will be made to FIG. 10A to FIG. 11 A method of generating and managing a plurality of allocation units when the storage device 200 provides a plurality of physical functions is described.

[0126] FIG. 10A to FIG. 10C is a diagram illustrating an electronic system according to example embodiments.

[0127] Reference Fig. 10A , the electronic system 20 may include a host 400 and a storage device 500. The host 400 and the storage device 500 may be similar to those described in reference Figure 1 The host 100 and the storage device 200 described above are similar to those described above. However, the host 400 and the storage device 500 may support multiple physical functions PF1, PF2, and PF3. The host operating system 420, the storage controller 510, and the non-volatile memory device (NVM) 520 may be similar to those described above. Figure 1 The host operating system 120, storage controller 210, and NVM 220 are described.

[0128] The physical functions PF1, PF2, and PF3 may be distinguished from virtual functions, which are lightweight PCIe functions that share one or more resources with physical functions. Even when the host 100 does not support a single root input / output virtualization (SR-IOV) architecture, the physical functions PF1, PF2, and PF3 may appear as actual PCIe devices accessible from the host 400. In other words, the storage device 500 may implement virtualization by providing the physical functions PF1, PF2, and PF3 without relying on software conversion performed by the virtual machine manager 410.

[0129] The plurality of physical functions PF1 , PF2 , and PF3 may be identical to one another, and each of the plurality of physical functions PF1 , PF2 , and PF3 may process an input / output request and also a management request such as a resource configuration request.

[0130] The manager 424 of the host 400 may identify a plurality of physical functions PF1, PF2, and PF3 provided by the storage device 500, and may obtain resource amount information of the storage device 500. In addition, the manager 424 may determine a storage capacity and a QoS to be allocated to each of the plurality of physical functions PF1, PF2, and PF3, and may control the storage device 500 to allocate the determined storage capacity and QoS to each of the plurality of physical functions PF1, PF2, and PF3.

[0131] As reference Figure 1 As described above, each resource of the plurality of physical functions PF1, PF2, PF3 may be subdivided by a plurality of allocation units ADI. Each of the plurality of allocation units ADI may be mapped to at least one input / output queue Q.

[0132] According to an example embodiment, the manager 424 may provide a resource configuration request defining a resource allocation rule of an allocation unit group to the plurality of physical functions PF1, PF2, and PF3 through the physical function drivers 421, 422, and 423 based on the storage capacity and QoS allocated to the plurality of physical functions PF1, PF2, and PF3. Fig. 10A In the example, a plurality of allocation units ADI included in a physical function may configure an allocation unit group. For example, an allocation unit ADI included in a first physical function PF1 may configure a first allocation unit group ADIG1, ​​an allocation unit ADI included in a second physical function PF2 may configure a second allocation unit group ADIG2, and an allocation unit ADI included in a third physical function PF3 may configure a third allocation unit group ADIG3.

[0133] The manager 424 may select a physical function in which an allocation unit ADI may be generated, and may determine the number of allocation units ADI to be allocated in the physical function based on the amount of resources required by the virtual machine VM and the resource allocation rule configured for each of the plurality of physical functions PF1, PF2, and PF3. The manager 424 may provide an allocation unit generation request to a physical function driver corresponding to the physical function having the allocation unit group.

[0134] In an example embodiment, the hypervisor 424 may run in a secure core. The secure core may be a virtual machine that can provide enhanced security by running in a different area from other virtual machines VM. The hypervisor 424 running on the secure core may control the amount of resources to be allocated to the plurality of physical functions PF1, PF2, PF3 and the plurality of allocation units ADI, thereby preventing malicious users from hijacking resources of the storage device 500.

[0135] Fig. 10AAn example of a configuration allocation unit group of one or more allocation units ADI included in a physical function is shown, but the embodiment is not limited thereto. For example, a physical function may have two or more allocation unit groups, two or more physical functions may have an allocation unit group, or a physical function and an allocation unit group may have a many-to-many mapping relationship.

[0136] Reference Fig. 10B and Fig. 10C , the electronic system 20 may include a reference Fig. 10A The host 400 and the storage device 500 described above may support a plurality of physical functions PF1, PF2, and PF3. Each of the plurality of physical functions PF1, PF2, and PF3 may include one or more allocation units ADI.

[0137] exist Fig. 10B In the example of FIG. 1 , the allocation unit ADI included in the first physical function PF1 and the second physical function PF2 may configure the allocation unit group ADIG1. In addition, the allocation unit ADI included in the third physical function PF3 may configure two allocation unit groups ADIG2 and ADIG3.

[0138] For example, the manager 424 may provide a resource configuration request defining a resource allocation rule for the first allocation unit group ADIG1 to the first physical function PF1 and the second physical function PF2 through the physical function drivers 421 and 422. In addition, the manager 424 may provide a resource configuration request defining a resource allocation rule for the second allocation unit group ADIG2 and the third allocation unit group ADIG3 to the third physical function PF3 through the physical function driver 423.

[0139] Manager 424 may select an allocation unit group in which allocation unit ADI may be included based on the amount of resources required by the virtual machine VM and the resource allocation rule configured for each of the plurality of allocation unit groups ADIG1, ​​ADIG2, and ADIG3, and may determine the number of allocation units ADI to be generated in the allocation unit group. Manager 424 may provide an allocation unit generation request to a physical function driver corresponding to a physical function having the allocation unit group.

[0140] exist Fig. 10C In the example in , the physical function and the allocation unit group may have a many-to-many mapping relationship. For example, the allocation unit ADI included in the first to third physical functions PF1, PF2, and PF3 may configure the first allocation unit ADIG1, ​​and the allocation unit ADI included in the second physical function PF2 and the third physical function PF3 may configure the second allocation unit ADIG2.

[0141] The manager 424 can provide resource configuration requests defining resource allocation rules of the first allocation unit group ADIG1 to the first physical function to the third physical function PF1, PF2 and PF3 through the physical function drivers 421, 422 and 423, and can provide resource configuration requests defining resource allocation rules of the second allocation unit group ADIG2 to the second physical function PF2 and the third physical function PF3 through the physical function drivers 422 and 423.

[0142] Embodiments of the storage device 500 are not limited to supporting FIG. 10A to FIG. 10C The physical functions PF1, PF2 and PF3 are equivalent to each other, and multiple physical functions may have a parent-child relationship.

[0143] Fig.11 is a diagram illustrating an electronic system according to example embodiments.

[0144] Reference Fig.11 , the electronic system 30 may include a host 600 and a storage device 700. FIG. 10A to FIG. 10C As described above, the host 600 and the storage device 700 may support a plurality of physical functions PF0, PF1, and PF2.

[0145] A plurality of physical functions PF0, PF1, and PF2 may have a parent-child relationship. For example, the first physical function PF0 may be configured as a parent physical function, and the second physical function PF1 and the third physical function PF2 may be configured as child physical functions. The child physical function may process input / output requests, and the parent physical function may process management requests to manage the child physical functions.

[0146] Similar to reference Figure 1 In the described example, the hypervisor 624 of the host 600 may run on the host operating system 620. The hypervisor 624 may provide a resource configuration request or an allocation unit generation request to the first physical function PF0 as a parent physical function. The first physical function PF0 may control the second physical function PF1 and the third physical function PF2 as child physical functions in response to the resource configuration request or the allocation unit generation request. The PF drivers 621 to 623, the VMM 610, the storage controller 710, and the NVM 720 may be similar to those described in reference to FIG. FIG. 10A to FIG. 10C PF drivers 421 to 423, storage controller 510, and NVM 520 are described.

[0147] According to an example embodiment, the second physical function PF1 and the third physical function PF2 may include one or more allocation units ADI. In an example embodiment, a sub-physical function may have one or more allocation unit groups. In addition, the allocation unit ADI included in two or more sub-physical functions may configure an allocation unit group, and the sub-physical function and the allocation unit group may have a many-to-many mapping relationship.

[0148] According to reference FIG. 10A to FIG. 10C and Fig.11 The described exemplary embodiment can classify a plurality of allocation units ADI for each of a plurality of physical functions, thereby improving the convenience of resource management of the plurality of allocation units ADI and resource allocation of the plurality of virtual machines VM.

[0149] Fig.12 is a diagram illustrating a server system to which an electronic system is applied according to an example embodiment.

[0150] Reference Fig.12 , the data center 1000 may be implemented as a facility that collects various types of data and provides services (and may also be referred to as a data storage center). The data center 1000 may be configured as a system for operating a search engine and a database, or may be configured as a computing system used by a company (such as a bank) or a government agency. The data center 1000 may include application servers 1100 to 1100n (e.g., application server 1 to application server n) and storage servers 1200 to 1200m (e.g., storage server 1 to storage server m). In example embodiments, the number of application servers 1100 to 1100n and the number of storage servers 1200 to 1200m may be changed, and the number of application servers 1100 to 1100n and the number of storage servers 1200 to 1200m may be different.

[0151] The application server 1100 or the storage server 1200 may include at least one of the processors 1110 and 1210 and the memories 1120 and 1220. For example, the processor 1210 may control the overall operation of the storage server 1200, may access the memory 1220, and may execute instructions and / or data loaded into the memory 1220. The memory 1220 may be implemented as a double data rate synchronous DRAM (DDR SDRAM), a high bandwidth memory (HBM), a hybrid memory cube (HMC), a dual in-line memory module (DIMM), an Optane DIMM, and / or a non-volatile DIMM (NVMDIMM). In an example embodiment, the number of processors 1210 and the number of memories 1220 included in the storage server 1200 may be changed. In an example embodiment, the processor 1210 and the memory 1220 may provide a processor-memory pair. In an example embodiment, the number of processors 1210 and the amount of memory 1220 may be different. The processor 1210 may include a single-core processor or a multi-core processor. The above description of the storage server 1200 may be similarly applied to the application server 1100. In an example embodiment, the application server 1100 may not include the storage device 1150. The storage server 1200 may include at least one storage device 1250. In an example embodiment, the number of storage devices 1250 included in the storage server 1200 may be changed.

[0152] The application servers 1100 to 1100n and the storage servers 1200 to 1200m can communicate with each other through the network 1300. The network 1300 can be implemented using a fiber channel (FC) or an Ethernet. In this case, FC can be a medium for relatively high-speed data transmission, and an optical switch providing high performance / high availability can be used. According to the access method of the network 1300, the storage servers 1200 to 1200m can be provided as a file storage device, a block storage device, or an object storage device.

[0153] In one example embodiment, the network 1300 may be configured as a storage-only network such as a storage area network (SAN). For example, the SAN may be configured as an FC-SAN using an FC network and may be implemented according to the FC protocol (FCP). As another example, the SAN may be configured as an IP-SAN using a TCP / IP network and may be implemented according to the iSCSI (SCSI over TCP / IP or Internet SCSI) protocol. In another example embodiment, the network 1300 may be configured as a general network such as a TCP / IP network. For example, the network 1300 may be implemented according to protocols such as FC over Ethernet (FCoE), network attached storage (NAS), and NVMe over network fabric (NVMe-oF).

[0154] Hereinafter, the application server 1100 and the storage server 1200 will be described. The description of the application server 1100 is also applicable to other application servers 1100n, and the description of the storage server 1200 is also applicable to other storage servers 1200m.

[0155] The application server 1100 may store data requested by a user or client to one of the storage servers 1200 to 1200m through the network 1300. In addition, the application server 1100 may obtain data readout requested by a user or client from one of the storage servers 1200 to 1200m through the network 1300. For example, the application server 1100 may be implemented as a network server or a database management system (DBMS).

[0156] The application server 1100 may access a memory 1120n or a storage device 1150n included in another application server 1100n through the network 1300, or may access a memory 1220 to 1220m or a storage device 1250 to 1250m included in the storage servers 1200 to 1200m through the network 1300. Therefore, the application server 1100 may perform various operations on data stored in the application servers 1100 to 1100n and / or the storage servers 1200 to 1200m. For example, the application server 1100 may execute a command to move or copy data between the application servers 1100 to 1100n and / or the storage servers 1200 to 1200m. In this case, the data may be moved from the storage devices 1250 to 1250m of the storage servers 1200 to 1200m through the storage 1220 to 1220m of the storage servers 1200 to 1200m, or may be directly moved to the storage 1120 to 1120n of the application servers 1100 to 1100n. For security or privacy, the data moved through the network 1300 may be encrypted.

[0157] For example, in the storage server 1200, the interface (I / F) 1254 may provide a physical connection between the processor 1210 and the controller (CTRL) 1251 and a physical connection between the NIC (Network Interconnect) 1240 and the controller 1251. For example, the interface 1254 may be implemented as a direct attached storage (DAS) method that directly connects the storage device 1250 using a dedicated cable. In addition, for example, the interface 1254 may be implemented as various interface methods (such as Advanced Technology Attachment (ATA), Serial ATA (SATA), External SATA (e-SATA), Small Computer Small Interface (SCSI), Serial Attached SCSI (SAS), PCI Peripheral Component Interconnect (SAS), PCIe (PCI Express), NVM Express (NVMe), IEEE 1394, Universal Serial Bus (USB), Secure Digital (SD) Card, Multimedia Card (MMC), Embedded Multimedia Card (eMMC), Universal Flash Storage (UFS), Embedded Universal Flash Storage (eUFS), and / or Compact Flash (CF) Card Interface).

[0158] The storage server 1200 may further include a switch 1230 and a NIC 1240. The switch 1230 may selectively connect the processor 1210 to the storage device 1250 under the control of the processor 1210, or selectively connect the NIC 1240 to the storage device 1250. The switches 1130 to 1130n and the NICs 1140 to 1140n included in the application servers 1100 to 1100n may be configured similarly to the switches 1230 to 1230m and the NICs 1240 to 1240m included in the storage servers 1200 to 1200m.

[0159] In an example embodiment, the NIC 1240 may include a network interface card, a network adapter, etc. The NIC 1240 may be connected to the network 1300 through a wired interface, a wireless interface, a Bluetooth interface, an optical interface, etc. The NIC 1240 may include an internal memory, a digital signal processor (DSP), a host bus interface, etc., and may be connected to the processor 1210 and / or the switch 1230 through the host bus interface. The host bus interface may be implemented as one of the examples of the interface 1254 described above. In an example embodiment, the NIC 1240 may be integrated with at least one of the processor 1210, the switch 1230, and the storage device 1250.

[0160] In the storage servers 1200 to 1200m or the application servers 1100 to 1100n, the processor may program or read the data by sending a command to the storage devices 1150 to 1150n and 1250 to 1250m or the memories 1120 to 1120n and 1220 to 1220m. In this case, the data may be data after error correction by an error correction code (ECC) engine. The data may be data that has been processed by a data bus inversion (DBI) or a data mask (DM), and may include cyclic redundancy code (CRC) information. For security or privacy, the data may be encrypted.

[0161] In response to a read command received from a processor, the storage devices 1150 to 1150n and 1250 to 1250m may send control signals and command / address signals to the NAND (e.g., NAND flash memory devices or NAND flash memories) 1252 to 1252m. Therefore, when data is read out from the NAND flash memory devices 1252 to 1252m, a read enable (RE) signal may be input as a data output control signal and may be used to output data to the DQ bus. A data strobe (DQS) may be generated using the RE signal. The command and address signals may be latched in the page buffer according to the rising edge or falling edge of the write enable (WE) signal.

[0162] The controller 1251 may generally control the operation of the storage device 1250. In an example embodiment, the controller 1251 may include a static random access memory (SRAM). The controller 1251 may write data to the NAND flash memory 1252 in response to a write command, or read data from the NAND flash memory 1252 in response to a read command. For example, the write and / or read command may be provided from the processor 1210 in the storage server 1200, the processor 1210m in another storage server 1200m, or the processors 1110 to 1110n in the application servers 1100 to 1100n. The DRAM 1253 may temporarily store (buffer) data to be written to the NAND flash memory 1252 or data read from the NAND flash memory 1252. In addition, the DRAM 1253 may store metadata. Here, the metadata may be generated by the controller 1251 to manage user data or the NAND flash memory 1252. The storage device 1250 may include a secure element (SE) for security or privacy. In one embodiment, the controller 1251 m , NAND 1252 m , DRAM 1253 m , and interface 1254 m included in the memory device 1250 m may be configured similarly to the controller 1251 , NAND 1252 , DRAM 1253 , and interface 1254 included in the memory device 1250 .

[0163] Reference Figures 1 to 11The electronic systems 10, 20, and 30 described in the exemplary embodiments may be applied to Fig.12 The application server 1100 or the storage server 1200 in the storage server 1200. For example, in the storage server 1200, the processor 1210 can run multiple virtual machines on the host operating system. In addition, the storage device 1250 can provide multiple virtual storage devices for multiple virtual machines.

[0164] According to an example embodiment, the storage device 1250 may provide one or more physical functions, and the resources owned by the physical functions may be subdivided into a plurality of allocation units. The processor 1210 may classify the plurality of allocation units by defining resource allocation information for a plurality of allocation unit groups and providing a resource configuration request notifying the resource allocation information to the storage device 1250. The processor 1210 may control the storage device 1250 to generate one or more allocation units included in one or more allocation unit groups based on the amount of resources required by the virtual machine, and may provide storage resources to the virtual machine by mapping the one or more allocation units to a virtual storage device corresponding to the virtual machine.

[0165] According to the exemplary embodiment, the convenience of resource management of multiple allocation units and resource allocation of multiple virtual machines can be improved, so that overhead can be reduced. In addition, requests corresponding to multiple allocation units can be scheduled based on a credit-based multi-polling method, and stable performance can be provided to the virtual storage device.

[0166] According to the aforementioned example embodiments, by defining resource allocation information for an allocation unit group and mapping individual allocation units to the allocation unit group, resources can be allocated to individual allocation units based on the defined resource allocation information, and by configuring a virtual storage device with one or more individual allocation units, the overhead for configuring the virtual storage device can be reduced.

[0167] Furthermore, by scheduling requests corresponding to individual allocation units based on allocation unit groups and resource allocation amounts of individual allocation units included in the allocation unit groups, stable performance can be provided to a virtual storage device configured with one or more individual allocation units.

[0168] While example embodiments have been shown and described above, it will be apparent to those skilled in the art that modifications and variations may be made without departing from the scope of the present disclosure as defined by the appended claims.

Claims

1. A method of operating an electronic system, the method comprising: identifying physical functions provided by the input / output device and obtaining resource quantity information of the input / output device; determining a resource allocation rule for each of a plurality of allocation unit groups of an input / output device based on the resource amount information; selecting one or more allocation unit groups among the plurality of allocation unit groups based on the required resource amount and input / output attributes of the virtual machine and the resource allocation rule determined for each of the plurality of allocation unit groups; generating one or more allocation units for the selected one or more allocation unit groups; as well as The one or more allocation units are mapped to a virtual machine.

2. The method according to claim 1, in, The input / output device includes a storage device, and The step of determining resource allocation rules includes: in response to one or more resource configuration requests, respectively determining storage capacity and service quality of allocation units to be generated from the plurality of allocation unit groups.

3. The method according to claim 2, wherein: Quality of service includes write throughput, write latency, read throughput, and read latency.

4. The method according to claim 1, in, The input / output device includes a storage device, and The step of determining the resource allocation rule includes: determining the total storage capacity and the total throughput of each of the plurality of allocation unit groups in response to one or more resource configuration requests.

5. The method according to claim 4, wherein: The step of generating the one or more allocation units for the selected one or more allocation unit groups includes: in response to an allocation unit generation request specifying an allocation unit identifier and the selected one or more allocation unit groups, generating an allocation unit with a determined storage capacity and a determined throughput within the range of a total storage capacity and a total throughput of the selected one or more allocation unit groups.

6. The method according to claim 4, wherein: The step of generating the one or more allocation units for the selected one or more allocation unit groups includes: in response to an allocation unit generation request specifying an allocation unit identifier, a storage capacity of the allocation unit, and the selected one or more allocation unit groups, generating an allocation unit having a specified storage capacity and a throughput determined in proportion to the specified storage capacity within a range of a total storage capacity and a total throughput of the selected one or more allocation unit groups.

7. The method according to claim 4, wherein: The step of determining the resource allocation rule includes determining a ratio between a read throughput and a write throughput of each of the plurality of allocation unit groups in response to the one or more resource configuration requests.

8. The method according to claim 4, in, The input / output attribute is one of a read-intensive attribute, a write-intensive attribute, and a hybrid attribute, and The step of selecting the one or more allocation unit groups among the plurality of allocation unit groups comprises: selecting the one or more allocation unit groups based on a ratio between a read throughput and a write throughput and an input / output property of each of the plurality of allocation unit groups.

9. The method according to at least one of claims 1 to 8, in, The physical functions correspond to the input / output ports, and The one or more allocation units are configured as an allocable device interface supported by an extensible input / output virtualization architecture.

10. The method according to at least one of claims 1 to 8, wherein: The request for each of the one or more allocation units generated in the physical function is identified by a process address space identifier.

11. A method of operating an electronic system, the method comprising: In response to a resource configuration request, generating a plurality of allocation unit groups each having a different resource allocation rule; as well as receiving a plurality of allocation unit generation requests, each allocation unit generation request specifying one of the plurality of allocation unit groups; generating, in response to the plurality of allocation unit generation requests, a plurality of allocation units included in the plurality of allocation unit groups; mapping the plurality of allocation units included in the plurality of allocation unit groups to a plurality of input / output queues provided by a storage controller; as well as Throughput is allocated to each of the plurality of allocation units based on a resource allocation rule.

12. The method according to claim 11, further comprising: receiving a plurality of input / output requests and queuing each of the plurality of input / output requests to one of the plurality of input / output queues based on an identifier of each of the plurality of input / output requests; as well as providing a request processing opportunity to one of the plurality of input / output queues by a multi-polling method based on the plurality of allocation unit groups and the allocation units corresponding to each of the plurality of input / output queues; as well as Processes input / output requests queued in the input / output queue with request processing opportunities.

13. The method according to claim 12, in, The step of providing a request processing opportunity to one of the plurality of input / output queues by a multi-polling method comprises: providing request processing opportunities to the plurality of allocation unit groups by a first polling method, providing the request processing opportunity provided to one of the plurality of allocation unit groups to a plurality of allocation units included in the one allocation unit group by a second polling method, and A request processing opportunity provided to one allocation unit among the plurality of allocation units is provided to an input / output queue mapped to the one allocation unit.

14. The method according to claim 13, wherein: Each of the first polling method and the second polling method is one of a simple polling method and a weighted polling method.

15. The method according to claim 13, further comprising: A weight of the first polling method for each of the plurality of allocation unit groups is determined according to a ratio of the throughputs of each of the plurality of allocation unit groups.

16. The method according to claim 12, wherein: Each of the plurality of input / output queues includes a read request queue and a write request queue.

17. The method according to claim 11, further comprising: Determine the size of the periodically repeating time window; determining a throughput of each of the plurality of allocation unit groups based on the throughputs allocated to the plurality of allocation units; allocating a plurality of credits for each of the plurality of allocation unit groups in each time window based on a size of the time window and a throughput of each of the plurality of allocation unit groups; as well as Requests queued in an input / output queue are processed, and a determined number of credits are deducted from allocated credits for an allocation unit group associated with the input / output queue according to a size of input / output data corresponding to the processed request.

18. The method according to claim 17, wherein: The method further includes skipping request processing for an input / output queue when credits of an allocation unit group associated with the input / output queue are exhausted when the input / output queue has a request processing opportunity.

19. The method according to claim 11, further comprising: A namespace having a storage capacity determined based on the resource allocation rule is allocated to each of the plurality of allocation units.

20. A method of operating an electronic system, the method comprising: identifying a plurality of physical functions provided by an input / output device and obtaining resource quantity information of the input / output device; allocating storage capacity and quality of service to each of the plurality of physical functions based in part on the resource quantity information; providing a resource configuration request to determine a resource allocation rule for each of the plurality of physical functions based on a storage capacity and a quality of service allocated to each of the plurality of physical functions; selecting a physical function in which to generate an allocation unit based on an amount of resources requested by the virtual machine and a resource allocation rule configured for each of the plurality of physical functions; generating one or more allocation units for a selected physical function by providing an allocation unit generation request to the selected physical function; as well as The one or more allocation units are mapped to a virtual machine.

Images