Disk read / write resource allocation method, device, equipment, and storage medium

Abstract

The embodiments of the present application provide a method, apparatus, device, and computer-readable storage medium for allocating disk read and write resources, and relate to the field of cloud technology. The method comprises: determining the partition identifier of the target disk partition corresponding to the host based on a first mapping relationship between the host identifier and the host and the virtual device, a second mapping relationship between the virtual device and a preset namespace, and a third mapping relationship between the preset namespace and the disk partition of the disk; allocating the target disk partition corresponding to the partition identifier to the host, and responding to the read and write requests in the read and write request queue through the target disk partition. Through the present application, it is possible to achieve dynamic division of resources allocated to the read and write request queue, thereby achieving read and write load balancing for each preset namespace.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and are related to, but not limited to, a method, apparatus, device, and computer-readable storage medium for allocating disk read and write resources. Background Art

[0002] With the increasing popularity of flash memory technology, the Non-Volatile Memory Express (NVMe) host controller interface specification has become a key channel for supporting high-speed access between flash memory devices and hosts. Currently, the application of NVMe solid-state drives (SSDs) is mainly based on the demand for high-speed exclusive use, achieving extreme performance increases through specific hosts.

[0003] However, as the capacity of NVMe SSDs increases, the failure range and utilization rate of a single disk can no longer meet the requirements. It is necessary to consider capacity splitting to achieve resource reuse and sharing. However, the relevant technology does not have a constraint queue management and allocation mechanism for NVMe SSDs, and it is impossible to effectively split the capacity of NVMe SSDs to achieve resource reuse and sharing, and therefore it is impossible to achieve load balancing for NVMe SSDs. Summary of the Invention

[0004] The present invention provides a method, apparatus, device, and computer-readable storage medium for allocating disk read and write resources, relating to the field of cloud technology. Based on a first mapping relationship between a host and a virtual device, a second mapping relationship between a virtual device and a preset namespace, and a third mapping relationship between a preset namespace and a disk partition, a target disk partition corresponding to the host is determined. Resources of the target disk partition are then allocated to the host's read and write request queue, thereby dynamically allocating disk resources and achieving read and write load balancing across the preset namespaces.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] The present invention provides a method for allocating disk read and write resources, including:

[0007] receiving a read / write request queue sent by a host, wherein the read / write request queue includes a host identifier of the host;

[0008] Determining a device identifier of a target virtual device corresponding to the host according to the host identifier and a first mapping relationship between the host and the virtual device;

[0009] Determining a space identifier of a target namespace corresponding to the host according to the device identifier and a second mapping relationship between the virtual device and the preset namespace;

[0010] Determining a partition identifier of a target disk partition corresponding to the host according to the space identifier and a third mapping relationship between the preset namespace and the disk partition of the disk;

[0011] The target disk partition corresponding to the partition identifier is allocated to the host, and the read and write requests in the read and write request queue are responded to through the target disk partition.

[0012] An embodiment of the present application provides a device for allocating disk read and write resources, including:

[0013] A receiving module, configured to receive a read / write request queue sent by a host, wherein the read / write request queue includes a host identifier of the host;

[0014] A first determining module is configured to determine a device identifier of a target virtual device corresponding to the host according to the host identifier and a first mapping relationship between the host and the virtual device;

[0015] A second determining module is configured to determine a space identifier of a target namespace corresponding to the host according to the device identifier and a second mapping relationship between the virtual device and the preset namespace;

[0016] a third determining module, configured to determine a partition identifier of a target disk partition corresponding to the host according to the space identifier and a third mapping relationship between the preset namespace and the disk partition of the disk;

[0017] The allocation module is configured to allocate the target disk partition corresponding to the partition identifier to the host, and respond to the read and write requests in the read and write request queue through the target disk partition.

[0018] An embodiment of the present application provides a page display device, including:

[0019] The memory is used to store executable instructions; the processor is used to implement the above-mentioned method for allocating disk read and write resources when executing the executable instructions stored in the memory.

[0020] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the above-mentioned method for allocating disk read and write resources.

[0021] The embodiment of the present application has the following beneficial effects: for a read / write request queue received from a host, the target disk partition corresponding to the host is determined based on the host identifier of the host, in sequence based on the first mapping relationship between the host and the virtual device, the second mapping relationship between the virtual device and the preset namespace, and the third mapping relationship between the preset namespace and the disk partition of the disk, thereby allocating the resources of the target disk partition to the read / write request queue of the host, and responding to the read / write requests in the read / write request queue through the target disk partition. In this way, by dynamically binding the preset namespace to the disk partition and the virtual device respectively, the isolation and dynamic division of disk resources is achieved, and in addition, the mapping between the host's read / write request queue and the virtual device is combined to achieve dynamic division of the resources allocated to the read / write request queue, thereby achieving read / write load balancing for each preset namespace. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is an overall framework diagram for disk read and write resource allocation in related technologies;

[0023] Figure 2A This is a schematic diagram of the structure of the cloud platform provided in the embodiment of the present application;

[0024] Figure 2B This is a schematic diagram of the deployment relationship between the IaaS layer, PaaS layer, and SaaS layer provided in an embodiment of the present application;

[0025] Figure 3 This is a schematic diagram of the structure of the master node provided in an embodiment of the present application;

[0026] Figure 4 This is an optional flowchart of a method for allocating disk read and write resources provided in an embodiment of the present application;

[0027] Figure 5 This is an optional flowchart of a method for allocating disk read and write resources provided in an embodiment of the present application;

[0028] Figure 6 This is an optional flowchart of a method for allocating disk read and write resources provided in an embodiment of the present application;

[0029] Figure 7 This is an optional flowchart of a method for allocating disk read and write resources provided in an embodiment of the present application;

[0030] Figure 8 This is an optional flowchart of a method for allocating disk read and write resources provided in an embodiment of the present application;

[0031] Figure 9 This is a system framework diagram provided by an embodiment of the present application;

[0032] Figure 10 This is a structural diagram of an implementation scheme of a resource isolation module provided in an embodiment of the present application;

[0033] Figure 11 It is the NVMe PRP resource mapping diagram provided by the embodiment of the present application;

[0034] Figure 12 This is a queue scheduling diagram provided by an embodiment of the present application;

[0035] Figure 13 This is a test framework diagram provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limiting this application. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0037] In the following description, reference is made to "some embodiments," which describe a subset of all possible embodiments. However, it will be understood that "some embodiments" may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by those skilled in the art to which the embodiments of this application pertain. The terms used in the embodiments of this application are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0038] In order to better understand the method for allocating disk read and write resources provided in the embodiments of the present application, the method for allocating disk read and write resources in the related art is first described. The following description is made using an NVMe SSD as an example of a disk to be allocated resources:

[0039] As NVMe SSD capacity increases, the failure range and utilization of a single drive are no longer sufficient. This necessitates capacity splitting to achieve resource reuse and sharing, leading to the emergence of multiple namespaces within the NVMe specification. However, namespace technology is currently defined only for the device itself within the NVMe specification, and namespaces do not constrain queue management or allocation mechanisms, only defining rules.

[0040] There are currently no examples of technical solutions similar to this application in the industry. NVMe SSD manufacturers have launched similar solutions based on the NVMe specification on the device side, but they only have simple resource isolation and virtual device (VF, Virtual Function) enabling capabilities of high-speed serial computer expansion bus standard (PCIe, Peripheral Component Interconnect express) controllers. In addition, the current technology is limited by the device itself and there is no further evolutionary technology. Figure 1 This is the overall framework diagram for disk read and write resource allocation in related technologies, such as Figure 1 As shown, the processing flow of the technical solution in the related art is as follows: First, the primary controller (Primary Controller) 11 clearly defines the resource size it supports, and the primary controller 11 manages resource allocation; then, the resources of the specific size are mounted under the secondary controller (Secondary Controller) 12. The primary controller 11 includes private resources (Private Resources) 111 and flexible resources (Flexible Resources) 112. The flexible resources 112 are allocated based on a flexible allocation mechanism. All the flexible resources 112 together form a resource pool (Resource Pool) 13, which includes multiple unallocated flexible resources 112. Each secondary controller 12 can be allocated a flexible resource 112 from the resource pool 13.

[0041] The methods in the related art have at least the following problems: although the resource pool is defined, there are no constraints on allocation and binding rules, making actual application difficult; the methods in the related art do not define the allocation mechanism and pooling concept of NVMe queues, and do not have load balancing capabilities; for different application scenarios, they cannot meet the balanced many-to-many allocation requirements of NVMe devices, and the resource and queue utilization rate is insufficient; they are more focused on implementation under NVMe SSDs, and do not consider customized software and hardware combination solutions, and lack flexibility and scalability.

[0042] Based on at least one of the above-mentioned problems existing in the related art, an embodiment of the present application provides a concept of NVM namespace + queue resource pooling. Through a combination of software and hardware, it is designed to implement dynamic mapping with the physical device (PF, Physical Function) or VF of the PCIe controller, while ensuring effective isolation of NVMe resources, realizing the dynamic expansion and load balancing capabilities of resources.

[0043] The embodiment of the present application provides a method for allocating disk read and write resources. First, a read and write request queue sent by a host is received, wherein the read and write request queue includes a host identifier of the host; then, the device identifier of the target virtual device corresponding to the host is determined according to the host identifier and the first mapping relationship between the host and the virtual device; the space identifier of the target namespace corresponding to the host is determined according to the device identifier and the second mapping relationship between the virtual device and the preset namespace; the partition identifier of the target disk partition corresponding to the host is determined according to the space identifier and the third mapping relationship between the preset namespace and the disk partition of the disk; finally, the target disk partition corresponding to the partition identifier is allocated to the host, and the read and write requests in the read and write request queue are responded to through the target disk partition. In this way, the isolation and dynamic division of disk resources are achieved by dynamically binding the preset namespace to the disk partition and the virtual device respectively, and the mapping between the host's read and write request queue and the virtual device is combined to achieve dynamic division of resources allocated to the read and write request queue, thereby achieving read and write load balancing of each preset namespace.

[0044] The following describes an exemplary application of a disk read / write resource allocation device in an embodiment of the present application. In one implementation, the disk read / write resource allocation method provided in an embodiment of the present application relates to the field of cloud technology and can be implemented through cloud technology. For example, the disk read / write resource allocation device in an embodiment of the present application can be a node in a cloud platform. Figure 2A This is a schematic diagram of the structure of the cloud platform provided in the embodiment of the present application. Figure 2A As shown, the cloud platform 20 includes a master node 21 and multiple working nodes 22, wherein the master node 21 can be used as a device for allocating disk read and write resources in an embodiment of the present application, that is, the master node 21 serves as a master control node for multiple nodes in the cloud platform 20, and is used to control the multiple working nodes 22. In some embodiments, the working node 22 can be a host that sends read and write requests. The master node 21 allocates resources to the read and write request queues sent by the working node 22, virtualizes the physical resources 23 corresponding to the cloud platform (wherein the physical resources 23 can be a resource pool formed by multiple SSDs), and then allocates the actual physical resources corresponding to the virtualized resources to each read and write request queue of the host through the mapping relationship between the virtual device VF corresponding to the virtualized resources and the host, thereby responding to the read and write requests of the read and write request queues.

[0045] In an embodiment of the present application, the master node 21, as an allocation device for disk read and write resources, can at least further implement the following functions: receive a read and write request queue sent by each working node 22, wherein the read and write request queue includes the host identifier of the working node 22; determine the device identifier of the target virtual device corresponding to the working node 22 based on the host identifier and a first mapping relationship between the host and the virtual device; determine the space identifier of the target namespace corresponding to the working node 22 based on the device identifier and a second mapping relationship between the virtual device and a preset namespace; determine the partition identifier of the target disk partition corresponding to the working node 22 based on the space identifier and a third mapping relationship between the preset namespace and the disk partition of the disk; and assign the target disk partition corresponding to the partition identifier to the working node 22, so as to respond to the read and write request queue through the target disk partition.

[0046] It should be noted that cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to enable data computing, storage, processing, and sharing. Cloud technology is a general term for network, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form a resource pool for on-demand, flexible, and convenient use. Cloud computing technology will become a crucial support. Backend services for technical network systems, such as video websites, image websites, and more portals, require extensive computing and storage resources. With the rapid development and application of the internet industry, every item will likely have its own unique identifier and will need to be transmitted to backend systems for logical processing. Data of varying levels will be processed separately, and data from all industries will require a strong system backend, which can only be achieved through cloud computing.

[0047] As a provider of basic capabilities for cloud computing, a cloud computing resource pool platform will be established, referred to as a cloud platform, generally referred to as Infrastructure as a Service (IaaS), and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices. According to the logical function division, the Platform as a Service (PaaS) layer can be deployed on the IaaS layer, and the Software as a Service (SaaS) layer can be deployed on the Paa S layer, or the SaaS layer can be deployed directly on the IaaS layer. The PaaS layer is a platform for software operation, such as databases, web containers, etc. The SaaS layer is a variety of business software, such as web portals, SMS mass senders, etc. Among them, the deployment relationship between the IaaS layer 203, the PaaS layer 202, and the SaaS layer 201 is as follows. Figure 2B As shown, generally speaking, the SaaS layer 201 and the PaaS layer 202 are upper layers relative to the IaaS layer 203 .

[0048] Figure 3 is a schematic diagram of the structure of the master node 21 provided in an embodiment of the present application. Figure 3 The master node 21 shown includes: at least one processor 310, a memory 350, at least one network interface 320 and a user interface 330. The various components in the master node 21 are coupled together via a bus system 340. It is understood that the bus system 340 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 340 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 340 is not described in detail. Figure 3 Various buses are labeled as bus system 340 .

[0049] The processor 310 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0050] The user interface 330 includes one or more output devices 331 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 330 also includes one or more input devices 332, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0051] The memory 350 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, and the like. The memory 350 may optionally include one or more storage devices physically located away from the processor 310. The memory 350 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 350 described in the embodiments of the present application is intended to include any suitable type of memory. In some embodiments, the memory 350 is capable of storing data to support various operations, examples of which include programs, modules, and data structures, or subsets or supersets thereof, as exemplified below.

[0052] Operating system 351, including system programs for processing various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, which are used to implement various basic services and process hardware-based tasks;

[0053] A network communication module 352 for reaching other computing devices via one or more (wired or wireless) network interfaces 320 , exemplary network interfaces 320 including Bluetooth, WiFi, and USB;

[0054] The input processing module 353 is configured to detect one or more user inputs or interactions from one of the one or more input devices 332 and to translate the detected inputs or interactions.

[0055] In some embodiments, the apparatus provided in the embodiments of the present application may be implemented in software. Figure 3 A disk read / write resource allocation device 354 stored in the memory 350 is shown. The disk read / write resource allocation device 354 may be a disk read / write resource allocation device in the master node 21. The device may be software in the form of a program or plug-in, and includes the following software modules: a receiving module 3541, a first determination module 3542, a second determination module 3543, a third determination module 3544, and an allocation module 3545. These modules are logical and can be arbitrarily combined or further separated according to the functions implemented. The functions of each module will be described below.

[0056] In other embodiments, the apparatus provided in the embodiments of the present application may be implemented in hardware. As an example, the apparatus provided in the embodiments of the present application may be a processor in the form of a hardware decoding processor, which is programmed to execute the method for allocating disk read and write resources provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor may be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0057] The following will describe the method for allocating disk read and write resources provided by the embodiment of the present application in conjunction with the exemplary application and implementation of the master node 21 provided by the embodiment of the present application. Figure 4 , Figure 4 This is an optional flow chart of the method for allocating disk read and write resources provided in the embodiment of the present application, which will be combined with Figure 4 The steps shown are explained.

[0058] Step S401: receiving a read / write request queue sent by a host.

[0059] Here, the read / write request queue includes the host identifier of the host. In an embodiment of the present application, the read / write request queue can be any one of a submission queue (SQ) or a completion queue (CQ). The read / write request queue includes at least one read / write request. The read / write request queue is formed by writing the read / write request sent by the host to a preset SQ queue or a preset CQ queue. Each host can correspond to at least one SQ queue or CQ queue.

[0060] Step S402: Determine the device identifier of the target virtual device corresponding to the host according to the host identifier and the first mapping relationship between the host and the virtual device.

[0061] The first mapping relationship is between each host in the host cluster and a virtual device. Each virtual machine and application on each host is assigned at least one virtual device. Each virtual device corresponds to a virtual disk, which is the virtual storage space created by virtualizing a physical disk. In other words, a virtual disk is a virtual storage space created by virtualizing a physical disk using virtualization technology. Virtual storage space is allocated to the host's read and write request queue using the virtual disk corresponding to the virtual device.

[0062] In the embodiments of the present application, since each host may have a first mapping relationship with one or more virtual devices (ie, virtual disks), in some embodiments, one or more target virtual devices may be determined based on the host identification.

[0063] Step S403: Determine the space identifier of the target namespace corresponding to the host according to the device identifier and the second mapping relationship between the virtual device and the preset namespace.

[0064] Here we take the example of a target virtual device. After determining a target virtual device, the target namespace corresponding to the target virtual device is obtained according to the device identifier of the target virtual device and the second mapping relationship. This namespace is the target namespace corresponding to the host.

[0065] In the embodiment of the present application, the second mapping relationship refers to a mapping relationship between virtual devices and preset namespaces, and each virtual device corresponds to a namespace.

[0066] Step S404: Determine the partition identifier of the target disk partition corresponding to the host according to the space identifier and a third mapping relationship between the preset namespace and the disk partition of the disk.

[0067] Here, since the target namespace corresponding to the virtual device can be determined based on the second mapping relationship, the target disk partition can be determined based on the third mapping relationship between the target namespace and the disk partition. It should be noted that in the embodiment of the present application, multiple disks in the disk array can be partitioned, each disk corresponding to at least one disk partition, and for each disk in the disk array, the target namespace can correspond to a disk partition. In other words, for each preset namespace, a disk partition can correspond to each disk in the disk array. Then, after determining the target namespace, multiple disk partitions corresponding to the disk array can be determined.

[0068] Step S405: Allocate the target disk partition corresponding to the partition identifier to the host, and respond to the read / write request in the read / write request queue through the target disk partition.

[0069] Here, after the target disk partition is determined, the target disk partition is allocated to the host, and the read and write request response to the read and write request queue of the host is achieved through the target disk partition.

[0070] The method for allocating disk read and write resources provided in an embodiment of the present application determines, for a read and write request queue received from a host, the target disk partition corresponding to the host based on the host identifier of the host, in sequence based on the first mapping relationship between the host and the virtual device, the second mapping relationship between the virtual device and the preset namespace, and the third mapping relationship between the preset namespace and the disk partition of the disk, thereby allocating the resources of the target disk partition to the read and write request queue of the host, so as to achieve the response to the read and write requests in the read and write request queue through the target disk partition. In this way, the isolation and dynamic division of disk resources are achieved by dynamically binding the preset namespace to the disk partition and the virtual device respectively, and the dynamic division of resources allocated to the read and write request queue is achieved by combining the mapping between the host's read and write request queue and the virtual device, thereby achieving the read and write load balancing of each preset namespace.

[0071] In some embodiments, a system for allocating disk read / write resources includes a master node and a host cluster. The host cluster includes at least one host, which can be a virtual machine or a physical machine. At least one application runs on the host, and the application sends read / write requests to the master node, requesting the master node to allocate resources for the host's read / write requests. The master node can be a node obtained by combining software and hardware. A computer program product runs on the master node, and when executed, the computer program product can implement the disk read / write resource allocation method provided in the embodiments of the present application.

[0072] The following is an example of a system that allocates disk read and write resources and includes a master node and host H1. Figure 5 This is an optional flow chart of the method for allocating disk read and write resources provided in the embodiment of the present application, such as Figure 5 As shown, the method includes the following steps:

[0073] Step S501 : The host H1 obtains at least one read / write request while running an application program.

[0074] Step S502: Send at least one read / write request to the master node.

[0075] In step S503, the master node writes at least one read / write request to a preset submission queue or completion queue of host H1, forming a read / write request queue. The read / write request queue includes the host identifier H11 of host H1. The submission queue and completion queue can be an area in disk memory. In terms of data structure principles, the submission queue and completion queue can be a ring buffer with a head and a tail. Read / write requests from host H1 can be written sequentially from the head to the tail of the ring buffer queue.

[0076] Step S504 : The master node determines the device identifier of the target virtual device corresponding to the host H1 according to the host identifier H11 and the first mapping relationship between the host and the virtual device.

[0077] Step S505 : Determine the space identifier of the target namespace corresponding to the host H1 according to the device identifier and the second mapping relationship between the virtual device and the preset namespace.

[0078] Step S506 : Determine the partition identifier of the target disk partition corresponding to the host H1 according to the space identifier and the third mapping relationship between the preset namespace and the disk partition.

[0079] Step S507: Allocate the target disk partition corresponding to the partition identifier to the host H1, so as to respond to the read and write requests in the read and write request queue of the host H1 through the target disk partition.

[0080] It should be noted that steps S504 to S507 are the same as the above-mentioned steps S402 to S405, and will not be repeated in this embodiment of the application.

[0081] In the method for allocating disk read and write resources provided in the embodiments of the present application, the master node can be a combination of software and a hardware carrier corresponding to the method of the embodiments of the present application. Through the interaction between the host and the master node, resource allocation to the host's read and write request queue is achieved, and the combination of software and hardware can be customized, with flexibility and scalability. At the same time, by dynamically binding preset namespaces to disk partitions and virtual devices, the isolation and dynamic division of disk resources are achieved. In addition, the mapping between the host's read and write request queues and virtual devices is combined to achieve dynamic division of resources allocated to the read and write request queues, thereby achieving read and write load balancing for each preset namespace.

[0082] In some embodiments, a disk partition in a disk array can be allocated to each host in a host cluster before a read / write request queue is received from the host. In this way, when a read / write request is received from the host, the read / write request can be directly responded to within the allocated disk partition to achieve data reading or writing. Figure 4 , Figure 6 This is an optional flow chart of the method for allocating disk read and write resources provided in the embodiment of the present application, such as Figure 6 As shown, before step S401, the method further includes the following steps:

[0083] Step S601: Using at least one of the preset namespaces, resource isolation processing is performed on the disk resources of each disk in the pooled disk array to obtain multiple isolated disk partitions, wherein each disk partition has the third mapping relationship with a preset namespace.

[0084] Here, before the host sends a read or write request, the resources in the disk array are pooled, isolated, and mapped in advance, that is, the resources are allocated in advance. This ensures that when a read or write request is subsequently received from the host, the read or write request is responded to according to the pre-allocated resources. This realizes the pre-allocation of resources and ensures that the read or write request of the host can be responded to in a timely and effective manner.

[0085] In an embodiment of the present application, when performing resource isolation processing, one preset namespace may correspond to one disk partition, so that isolation of disk resources can be achieved through multiple preset namespaces.

[0086] In some embodiments, step S601 may be implemented by the following steps:

[0087] In step S6011, each preset namespace is used to perform resource isolation processing on the continuous disk resources corresponding to the continuous addresses in each disk to obtain a plurality of isolated disk partitions.

[0088] Here, the disk resources of each disk partition correspond to continuous addresses, that is, when the disk resources are isolated through each preset namespace, the disk resources corresponding to a continuous address are determined as a disk partition, and the capacity of each disk partition can be the same as or different from the capacity of other disk partitions.

[0089] In the embodiments of the present application, using a preset namespace to isolate disk resources can be understood as mapping the preset namespace to any disk partition of the disk resource. Therefore, each disk in the disk array can have a disk partition that is mapped to the same preset namespace.

[0090] In an embodiment of the present application, disk resources in a disk array are divided through a preset namespace to form multiple disk partitions. In this way, a resource pool can be formed through the disk partitions of multiple disks. This resource pool includes a large number of disk partitions, each disk partition corresponds to a storage space of a certain capacity, and the sum of the resource capacity of the entire resource pool is the sum of the resource capacity of all disks in the disk array. By dividing the resources of the disks in the disk array, resource sharing is achieved, that is, the resources in the resource pool can be used by applications on any host.

[0091] Step S602 : performing resource mapping processing on the disk partition and the preset namespace having the third mapping relationship and a virtual device in the virtual device set to obtain a second mapping relationship between the virtual device, the disk partition and the preset namespace.

[0092] Each virtual machine on a host and each application on the host will be allocated at least one virtual device. Therefore, for multiple hosts in the current system, a virtual device cluster can correspond to each other. Each virtual device corresponds to a disk partition in the resource pool obtained after the division in step S601, and the read and write requests of the virtual device are responded to through the disk partition.

[0093] For example, the disk SSD includes two SSD disk partitions, namely: SSD1 and SSD2, the preset namespaces include NS1 and NS2, and the virtual device cluster includes four virtual devices: VF1, VF2, VF3 and VF4. Then the second mapping relationship and the third mapping relationship of the embodiment of the present application can be: VF1 corresponds to SSD1 and NS1, VF2 corresponds to SSD1 and NS2, VF3 corresponds to SSD2 and NS1, and VF4 corresponds to SSD2 and NS2. That is, for VF1 corresponding to SSD1 and NS1, then the virtual device VF1 has a second mapping relationship with the disk partition SSD1 and the preset namespace NS1, and the disk partition SSD1 has a third mapping relationship with the preset namespace NS1; for VF4 corresponding to SSD2 and NS2, then the virtual device VF4 has a second mapping relationship with the disk partition SSD2 and the preset namespace NS2, and the disk partition SSD2 has a third mapping relationship with the preset namespace NS2.

[0094] Step S603 : pre-set a first mapping relationship between each host in the host cluster and at least one virtual device in the virtual device set.

[0095] Here, the first mapping relationship is a mapping relationship between a host and a virtual device. The first mapping relationship may be preset by the system, that is, a specific virtual device is allocated to each host in advance.

[0096] Please continue to refer to Figure 6 In some embodiments, a third mapping relationship exists between each preset namespace and a disk partition in each of the multiple disks in the disk array; therefore, when determining the target disk partition, a disk partition can be determined from the corresponding multiple disk partitions in the multiple disks. Therefore, after step S404, the method may further include the following steps:

[0097] Step S604 : For any disk partition corresponding to each preset namespace, a first priority of the disk partition is determined according to the read / write response speed or capacity of the disk partition.

[0098] Here, the larger the capacity of the disk partition is, the higher the first priority of the disk partition is; the higher the read and write response speed of the disk partition is, the higher the first priority of the disk partition is.

[0099] Step S605: Obtain the second priority of the read / write request queue sent by the host.

[0100] Here, the second priority of the read / write request queue can be determined based on the importance and processing urgency of the read / write request queue. When sending the read / write request queue, the host can add a priority tag or importance level tag to each read / write request queue to clarify the second priority of the read / write request queue.

[0101] Step S606 : Match a disk from the multiple disks according to the second priority and the first priority of each disk partition, and determine the disk partition in the matched disk as the target disk partition.

[0102] Here, since multiple target disk partitions can be matched across multiple disks based on the host identifier, a suitable target disk partition can be selected to respond to the read / write request based on the second priority of the read / write request queue. For example, if the read / write request queue is for read / write requests with relatively high latency requirements or is relatively important, and therefore has a relatively high second priority, the target disk partition with a relatively high first priority can be selected from the multiple target disk partitions as the target disk partition that ultimately responds to the read / write request queue.

[0103] Please continue to refer to Figure 6 In some embodiments, before step S401, the method further includes the following steps: Step S607, adding an initialization tag to the host, where the initialization tag is used to identify the target capacity of the target disk partition corresponding to the host's read and write request queue.

[0104] Here, the target capacity corresponds to the storage capacity demand of the read / write request queue. If the storage capacity demand of the read / write request queue is high, the initialization tag may be a tag corresponding to a higher target capacity, for example, the tag may be set to "High"; if the storage capacity demand of the read / write request queue is low, the initialization tag may be a tag corresponding to a lower target capacity, for example, the tag may be set to "Low".

[0105] Correspondingly, after step S404, the method may further include the following steps:

[0106] Step S608: Match one or more disks from the multiple disks according to the target capacity.

[0107] Here, if the initialization tag is "Low", a disk with a lower capacity can be matched among multiple disks; if the initialization tag is "High", a disk with a higher capacity can be matched among multiple disks, or multiple disks can be matched, and the host's read and write requests can be responded to simultaneously through the corresponding multiple disk partitions in the multiple disks.

[0108] Step S609: determining the disk partition in the matched one or more disks as the target disk partition.

[0109] based on Figure 4 , Figure 7 This is an optional flow chart of the method for allocating disk read and write resources provided in the embodiment of the present application, such as Figure 7 As shown, step S404 can be implemented by the following steps:

[0110] Step S701: Determine the address of the target namespace corresponding to the page offset and space identifier of the target virtual device.

[0111] Here, each namespace has a specific address, and the page offset of the target virtual device may be an address offset value.

[0112] Step S702 : determining the offset value corresponding to the page offset as the offset of the address of the target virtual device relative to the address of the target namespace.

[0113] Here, the offset value corresponding to the page offset is the address offset value. In this application, the address offset value is determined as the offset of the address of the target virtual device relative to the address of the target namespace.

[0114] Step S703: Determine the offset address relative to the target namespace, which is offset by the offset amount, as the address of the target disk partition corresponding to the host.

[0115] Here, the address of the target namespace is used as the starting address, and the address offset from the starting address by the offset is determined as the offset address. The disk partition corresponding to the offset address is the target disk partition. It should be noted that since each disk partition corresponds to multiple consecutive addresses, after determining any address of a disk partition, the target disk partition can be determined based on that address.

[0116] Step S704: Determine the partition identifier of the target disk partition according to the address of the target disk partition.

[0117] Here, the partition identifier of the target disk partition can be the address of the target disk partition, or any identifier that can distinguish the target disk partition, such as the name, address area (i.e., the first address and the last address of the disk partition), the identifier of the namespace uniquely corresponding to the target disk partition, etc.

[0118] based on Figure 4 , Figure 8 This is an optional flow chart of the method for allocating disk read and write resources provided in the embodiment of the present application, such as Figure 8 As shown, before step S404, the method further includes the following steps:

[0119] Step S801: Determine the resource allocation weight corresponding to the current read and write request queue of the host.

[0120] In this embodiment of the present application, the following three methods are provided for determining resource allocation weights:

[0121] Method 1: Step S8011: After adjusting the host's resource allocation weights, determine the host's current read / write request volume and the number of pending read / write requests in the target virtual device. Step S8012: Determine the resource allocation weight corresponding to the host's current read / write request queue based on the current read / write request volume and the pending read / write request volume.

[0122] Method 2: Step S8013: Determine a first average value of the delay time for responding to read / write requests in a historical period and a second average value of the delay time for responding to read / write requests in a current period. Step S8014: If the second average value is greater than or equal to the first average value, increase the resource allocation weight corresponding to the host's current read / write request queue. Step S8015: If the second average value is less than the first average value, decrease the resource allocation weight corresponding to the host's current read / write request queue.

[0123] Method 3: Step S8016: Determine the latency sensitivity type of the host's read / write request queue. Step S8017: If the read / write request queue is a tail latency sensitive queue, increase the resource allocation weight corresponding to the host's current read / write request queue. The increased resource allocation weight is the maximum weight among all read / write request queues in the current disk read / write resource allocation system.

[0124] Correspondingly, step S404 can be implemented by the following steps:

[0125] Step S802: Based on the space identifier and the resource allocation weight, one or more disks in the disk array are identified as target disks. A higher resource allocation weight corresponds to more disk resources allocated, while a lower resource allocation weight corresponds to fewer disk resources allocated. Alternatively, a higher resource allocation weight corresponds to a greater number of target disks allocated, while a lower resource allocation weight corresponds to a smaller number of target disks allocated.

[0126] Step S803: Among the matched target disks, determine the disk partition corresponding to the third mapping relationship as the target disk partition.

[0127] The disk read and write resource allocation method provided in the embodiment of the present application configures the resource allocation weight corresponding to the host's current read and write request queue, and sets different resource allocation weights for different hosts. Therefore, reasonable disk resources can be allocated to the corresponding host according to the resource allocation weight, thereby realizing reasonable allocation of resources. Moreover, the resource allocation weight can be dynamically adjusted according to the current read and write request volume, thereby realizing dynamic allocation of resources, thereby improving the system's rapid and effective response to the read and write request queue of each host.

[0128] Below, an exemplary application of the embodiment of the present application in a practical application scenario will be described.

[0129] An embodiment of the present application provides a method for allocating disk read and write resources. By dynamically binding NameSpace (i.e., namespace, corresponding to the above-mentioned preset namespace) with PCIe VF / PF, the isolation and dynamic division of N VMe device resources are achieved. In addition, exclusive memory is used to dynamically divide NVMe queue resources (i.e., resources occupied by read and write request queues) to achieve read and write load balancing between each NameSpace.

[0130] The method of the embodiment of the present application can be applied to storage and high-IO bare metal scenarios, achieving NVMe resource offloading and greatly enhancing the user's convenience in using NVMe. At the same time, it can ensure that users only need to configure the priority of each application to provide a stable and controllable service bandwidth and latency experience.

[0131] In a typical application scenario of the embodiment of the present application, such as in a high-performance IO instance, the method of the embodiment of the present application can be used for back-end storage, and dynamic adjustment of IO resources is achieved through dynamic mapping of NVMe queue resources and PF / VF; the scheduling logic of the IO queue is introduced into the NVMe mapping framework to improve the queue resource utilization; it overcomes the current lack of effective NVMeIO resource sharing and virtualization solutions in the industry, and provides a set of NVMe resource pooling solutions; it solves the problems of low utilization and uncontrollable performance caused by the necessity of occupying a large amount of CPU resources when using NVMe resources, and the inability to effectively implement NVMe resource pooling and dynamic scheduling.

[0132] Figure 9 This is a system framework diagram provided by the embodiment of the present application, such as Figure 9 As shown, the system of the embodiment of the present application includes, from bottom to top, a disk array 91, a hardware layer (HW, Hardware) 92, i.e., a master node, which is deployed with the hardware resource management solution of the embodiment of the present application, an intermediate software layer running between the basic physical server and the operating system, a virtual machine monitor (HyperVisor) 93 and a user layer 94 that allows multiple operating systems and applications to share hardware.

[0133] The disk array 91 includes multiple NVMe SSD disks 911;

[0134] In the hardware layer 92, it includes an NVMe root complex device 921, an NVMe virtualization manager (NVMe Virtualization Manager) 922, a physical device (PF) 923 of the PCIe controller, and a virtual device (VF) 924 of at least one PCIe controller; between the hardware layer 92 and the virtual machine monitor 93, communication between the hardware layer 92 and the virtual machine monitor 93 is achieved through a read-write memory management unit (IOMMU, IO Memory Management Unit).

[0135] The virtual machine monitor 93 includes: a PF driver 931, a data I / O controller for a virtual device based on the PCI bus (VFIO-PCI) 932, a data I / O controller for a VF device (mediated-VFIO) 933, an I / O control manager (AdminIOCTL) 934, an NVMe PF 935, data I / O controllers for at least one VF device (VFIO) 936, and a muser scheduler 937. It should be noted that the PF driver 931, VFIO-PCI 932, and mediated-VFIO 933 are three parallel methods for interacting with the hardware layer 92. Any one of these methods can be used to interact with the hardware layer 92, and different methods can be used for different hosts. The physical device 923 of the PCIe controller interacts with the PF driver 931 in the virtual machine monitor 93 through I / O control (IOCTL). The virtual device 924 of the PCIe controller interacts with the VFIO-PCI 932 for I / O data exchange.

[0136] In the user layer 94, there are multiple hosts 941 (the hosts here include not only hosts but also clients), each of which runs an application (APP), and each of which also includes a standard NVMe (Standard NVMe) 942 and an NVMe character (NVMe Character) 943.

[0137] The key to the above-mentioned system in the embodiment of the present application is to deploy a hardware resource management solution in the underlying hardware (HW) and combine it with the internal software scheduling mechanism to realize dynamic scheduling of NVMe SSD resources. For the NVMe SSD disk array under the HW hardware, the solution of the embodiment of the present application is that the hardware resource management solution in the HW is embodied as a management port for the NVMe SSD, which is responsible for the initialization, resource configuration, and allocation of queues and NameSpaces of the NVM e SSD to ensure normal access to the NVMe SSD; within the hardware resource management solution in the HW, the applied memory resources (exclusive resource design can also be done) can be allocated to the corresponding queue (that is, the read and write request queue sent by the host), and unified management and scheduling can be achieved in the same memory segment; for the management of the virtual machine monitor 93 and the user layer 94 above the hardware resource management solution in the HW, the simulation and reporting of the NVMe controller (NVMe Controller) is realized (that is, the hardware layer 92 is simulated as an NVMe controller), and through a specific binding mechanism, the PF / VF is uniformly bound to the NameSpace and queue resources, and a virtual NVMe controller is provided to the upper-level virtual machine monitor 93 or the user layer 94 application for direct use.

[0138] The method for allocating disk read and write resources in the embodiment of the present application mainly includes the following parts:

[0139] Part 1: NVMe queue resource isolation module.

[0140] Figure 10 This is a structural diagram of the implementation scheme of the resource isolation module provided in the embodiment of the present application, such as Figure 10 As shown, the bottom layer is the NVMe resource 1001 to be allocated, above the NVMe resource 1001 is the hardware layer 1002, and above the hardware layer 1002 is the host 1003.

[0141] Among them, in the hardware layer 1002, the virtual machine monitor (VMM) of the hardware layer corresponds to the client physical address (GPA) and the host physical address (HPA), and the GPA and HPA correspond to the dynamic random access memory (DRAM). Among them, the GPA is the address sent down by the upper-layer application, and the GPA needs to be converted into HPA through the preset extended page table (EPT) to correspond to the HPA. The HPA corresponds to the actual address in the DRAM (i.e., HW DRAM). The hardware layer 1002 also includes a direct memory access (DMA) buffer (DMA Buffer) 1004. The DMA Buffer 1004 is located in a physical queue pool (Physical Queue Pool) 1005. The physical queue pool 1005 includes multiple physical submission queues (PSQs) and multiple physical completion queues (PCQs).

[0142] In the host 1003, each host has at least one virtual machine (VM), for example Figure 10 As shown in the figure, VM1 and VMx each have a client application (Guest APP), which generates a virtual submission queue (VSQ, Virtual Submission Queue) and a virtual completion queue (VCQ, Virtual Completion Queue). Each virtual machine can be assigned one or more VFs (such as Figure 10In the example, VF1 and VFx in the virtual machine are communicated with each other through IOMMU.

[0143] In the embodiment of the present application, for different NVMe virtualization requests and resources, exclusive application for memory and DMA buffer (DMA Buffer) 1004 is implemented through hardware to ensure resource independence and pooling. What is presented to the upper layer is the PCIe function (PCIe Function), because when accessing the PCIe Function, the client virtual address (GVA, GuestVirtual Address) is corresponding, thereby achieving isolation of the access address. NVMe resources in the memory pool are made corresponding to the DMA Buffer of the PF / VF to be bound, and address conversion is achieved through the IOMMU mechanism of VFIO and the EPT table customized and maintained by the hardware. The IO service quality (QoS, Quality of Service) configures the corresponding mapping method, and abstracts the hardware resources related to the hardware queue into a circular queue method similar to the NVMe submission queue / completion queue (SQ / CQ, Submission Queue / Completion Queue), ensuring that the upper interface remains unchanged.

[0144] Internal NVMe SSD resources are isolated by logical block address (LBA) access through NameSpaces, and I / O access to each NVMe NameSpace is isolated by binding it to NVMe Sets and NVMe Dies. This ensures that when multiple NameSpaces read and write concurrently, there will be no performance conflicts or resource preemption.

[0145] In the internal exclusive memory, the NVMe CQ / SQ corresponding to each NameSpace is allocated separately, and each queue pair corresponds to only one NameSpace, achieving performance and memory isolation during the IO delivery process.

[0146] After the VF exposes the bound resources to upper-layer applications, user-side applications can only access virtual memory addresses. At the same time, the corresponding address offsets on the application-side GVA and GPA are not the offsets on the actual N VMe SSD device, but require re-address mapping by the mapping layer (that is, the mapping layer in the HW that performs VF and Name Space mapping processing), thereby ensuring memory access isolation from the application side.

[0147] Part 2: Resource mapping module.

[0148] Figure 11It is a resource mapping diagram of the NVMe physical request page (PRP) provided in the embodiment of the present application, such as Figure 11 As shown, the hardware layer 111 includes a mapping layer 1111, which has mapping relationships between the public memory 1112 and the SSD, NS and VF. It should be noted that, Figure 11 The mapping is achieved based on the page base offset (Page Base Offset (PerController)) of each NVMe controller (i.e., the master node corresponding to each hardware layer 111), the page base offset (Page Base Offset (PerNamespace)) of each Namespace, and the command offset (Command Offset).

[0149] The resource mapping here mainly includes the correspondence and mapping between NameSpace (i.e. NS in the figure) and PCIe function (PCIe Function). The mapping relationship is established during initialization and mapped in real time during the IO process. The detailed steps are as follows:

[0150] 1) Initially, based on user configuration requirements, a specific size of NS and read / write request queues are bound to the VF. The corresponding relationship is that NSy of device SSDx corresponds to VFz simulated by the upper layer.

[0151] 2) The host sends a standard NVMe PRP request to VFz. At this time, the default NS identifier (NSID) is 1, which facilitates quick query of the mapping relationship;

[0152] 3) The NVMe resource pooling layer obtains the PRP request and converts the correspondence between VFz and SSD xNSy into the corresponding page offset in the resource mapping module;

[0153] 4) The NVMe resource pooling layer sends the modified PRP request to the corresponding actual NVMe SSDx, keeping the original command offset unchanged;

[0154] 5) After the actual NVMe SSD processing is completed at the lower end, it directly returns to the CQ and command ID corresponding to the issued command SQ. This part does not involve address conversion.

[0155] Part III: Queue Scheduling Strategy.

[0156] Figure 12 This is a queue scheduling diagram provided by an embodiment of the present application, such as Figure 12As shown, in the embodiment of the present application, after the NVMe queue resource pool is realized by combining software and hardware, a dynamic mapping and scheduling method with a virtual queue VQ and a backend physical queue PQ is currently adopted, such as Figure 12 As shown, different applications or client operating systems (OS, Operating System) correspond to different VQs. By adding different tags 121 (Tags) to various applications, access identification of each VQ interval is achieved, and shared occupation of the backend PQ is achieved. This strategy avoids the situation where physical queue resources are idle or software virtual resources wait for a long time to the greatest extent while ensuring access isolation. In the hardware layer 122, it includes an NVMe host manager 1221, independent memory 1222 and multiple PSQs (such as PSQ1, PSQ2 and PSQ3 in the figure) and multiple PCQs (such as PCQ1 and PCQ2 in the figure). In the host 123, it can include host APP, Guest APP and container APP. Among them, in the queue mapping, different filling colors in each PSQ and each PCQ in the figure correspond to different APPs, that is, each PSQ and each PCQ allocates different proportions of queue memory space to each APP.

[0157] In an embodiment of the present application, a weight distribution indicator based on the real-time load of the software virtual queue is designed: for loads that are sensitive to task completion time (deadline), virtual queue scheduling needs to occur as little as possible to prevent dynamic resource adjustments caused by competition and scheduling of other types of loads, so its weight is set to the maximum; for loads that are sensitive to throughput and delay, it is possible to consider using short-term load prediction and feedback correction methods to set weights and dynamically correct them during operation.

[0158] Specifically, for the indicators configured for different hosts, the following different strategies can be adopted:

[0159] 1) Throughput-sensitive load weight: The weight is calculated based on the corresponding load after real-time weight adjustment and the load waiting in the virtual software queue.

[0160] 2) Average latency-sensitive load weight: This is calculated based on the average latency of the most recent cycles and compared with the actual average latency of the current cycle. If the average latency is lower than the current average latency, it indicates that the performance of the virtual queue is being affected by other virtual queues. In this case, the weight should be increased to maintain the original average latency. If the average latency is higher than the current average latency, it indicates that the current weight is sufficient to maintain a good average latency. In this case, the weight of the virtual queue can be reduced to increase the probability of other queues obtaining physical queue arbitration.

[0161] 3) Tail Latency-Sensitive Load Weight: This configuration targets scenarios with high I / O completion time requirements. With this configuration, the system prioritizes queues with tail latency-sensitive loads. Physical queues currently processing I / O instructions are no longer added to the scheduling system, ensuring exclusive use of the physical queues. During weight updates, the weights of virtual queues with tail latency-sensitive I / O tasks within a cycle are also set to the maximum value.

[0162] Currently, three priority levels are defined for applications: throughput-sensitive (default is Low), latency-sensitive (default is Normal), and deadline-sensitive (default is High). The number of physical queues that can be allocated is set based on the virtual queue weights of the three types of applications. The settings are divided into the following steps:

[0163] A. Allocate physical queues for virtual queues of deadline-sensitive workloads. Since the virtual queues corresponding to these types of applications are set to the maximum weight by default when weights are updated, these virtual queues can obtain physical queues by default during initialization and are not affected by the virtual queue allocation mechanism for other types of workloads.

[0164] B. During operation, polling is performed according to a specific period, and the number of physical queue mappings N1 that can be obtained by the application corresponding to the virtual queue of the average delay-sensitive load in the period is calculated;

[0165] C. Allocate a corresponding number of physical queues to virtual queues that execute average delay-sensitive workloads based on pre-set weights. To ensure high-priority applications, ensure that the number of physical queue mappings allocated to average delay-sensitive workloads does not exceed N2.

[0166] D. Finally, physical queues are allocated to virtual queues of relatively low-priority throughput-sensitive loads based on weight values. This ensures that the loads do not starve and that all virtual queues have corresponding remaining physical queues mapped to them.

[0167] Part 4: Testing process.

[0168] Figure 13 This is a test framework diagram provided by the embodiment of the present application, such as Figure 13 As shown, the embodiment of this application mainly implements the rescheduling and mapping of NVMe IO resources, so the corresponding test framework mainly focuses on IO read and write testing of the re-managed resources. Key dimensions that require attention include: multi-host applications, tenant isolation, IO scheduling, and performance assurance.

[0169] At the beginning of the test, a corresponding virtual machine or I / O application is created for each simulated device, and the corresponding I / O testing software is run on the virtual machine or host. Different tags can be added to different virtual machines / applications. Four scenarios can be configured for this test: no QoS, bandwidth-sensitive QoS, latency-sensitive QoS, and long-tail-sensitive QoS. During the I / O testing, the IOPS, latency, and 99.99% QoS of each virtual machine / application are primarily observed. The corresponding performance results are compared with those of a virtual I / O application without any scheduling or resource isolation.

[0170] For load models, we selected the three load models shown in Table 1, which are respectively for high-bandwidth scenarios and database latency-sensitive scenarios:

[0171] Table 1 Three load models

[0172] load Block size iodepth I / O characteristics Multi-threaded intensive workloads 512K / 1024K 32 Random Read Highly concurrent random SQL 4K 2-16 Mixed random read and write High-performance database 4K 2-12 Mixed random read and write

[0173] In Table 1 above, multi-threaded intensive workloads are large-packet workloads with no I / O traffic restrictions, focusing more on throughput. High-concurrency random SQL workloads, on the other hand, are more sensitive to average latency. High-performance database workloads simulate high-end database solutions and are more sensitive to deadlines. The throughput of the latter two applications primarily targets small, random database workloads and are relatively low compared to the first.

[0174] In this embodiment, by configuring the tags corresponding to three VMs with high, medium, and low bandwidth priorities, and running the same multi-threaded intensive workload, the high-priority instance achieved a 10% bandwidth optimization compared to the medium (Normal) level, while the low-priority instance experienced a 15% bandwidth degradation compared to the medium (Normal) level, without experiencing any performance drop-off. Because the current software model simulation approach results in relatively low memory access and conversion efficiency, this test result is consistent with expectations.

[0175] The method provided in the embodiments of this application is a combined hardware and software system that implements pooling and dynamic mapping of NVMe SSD resources. It also integrates IO QoS scheduling policies to achieve dynamic adjustment of NVMe IO resources and offload NVMe virtualization capabilities. It also proposes an NVMe resource management method that can achieve resource pooling and dynamic scheduling while accessing NVMe resources, and implements resource isolation and dynamic mapping through PF / VF. Typical application scenarios include most NVMe storage resource access and bare metal virtualization scenarios. It addresses how to effectively manage NVMe resources using a set of standard hardware, thereby solving the problems of NVMe resource sharing isolation and IO QoS.

[0176] The embodiments of the present application circumvent the problem of the inability to effectively isolate in traditional NVMe storage systems, and instead design a set of IO virtualization frameworks to achieve resource isolation from the perspective of PCIe Function. In typical application scenarios such as NVMe virtualization scenarios, this method implements dynamic calling and allocation of IO and storage resources corresponding to NVMe devices on the basis of resource isolation, thereby effectively improving the complexity of system use and enhancing controllability. The embodiments of the present application overcome the problem that traditional NVMe IO systems cannot achieve complete resource isolation while achieving resource sharing. The present invention does not require the planning of corresponding exclusive physical machines for applications in advance when deploying NVMe storage systems.

[0177] Another significant beneficial effect is that after achieving unified management of NVMe devices, the embodiments of the present application implement a dynamic binding layer between software virtual queues and hardware queues within the system, which can effectively add a resource scheduling mechanism and implement scheduling strategies with different priorities through different tags of each host, further ensuring IO smoothness. Because this part does not require high hardware configuration, the embodiments of the present application are easier to implement in real application scenarios than other existing methods. For example, the existing FPGA or IP solutions in the industry currently support a limited scale of NVMe, are expensive, and do not have the ability to be implemented in large quantities.

[0178] The embodiments of the present application take advantage of the premise that NVMe IO cannot be effectively isolated after sharing, and integrate independent resources to manage NVMe devices to eliminate interference effects; through internal resource scheduling and dynamic resource binding, the advantages of QoS stability and unified management are achieved.

[0179] Finally, the embodiment of the present application can achieve more refined priority arbitration for different target hosts. For delay-sensitive services, it can ensure rapid IO response by configuring deadlines, ensuring that various performance indicators and long-tail indicators are significantly better than other existing reference systems.

[0180] The embodiment of the present application realizes the effective management of NVMe SSD resources through a set of software and hardware combination solutions. However, the embodiment of the present application does not specifically limit the IO transmission layer corresponding to the NVMe SSD, and a network module can be added to realize the interconnection support and clustered management of remote devices. Based on this type of scenario, the interconnection of NVMe resources can be realized under the premise of large-grained (at least each NVMe Set) blocks (Chunk) through distributed storage technology, but this part does not have the ability of virtualized Offloading and NV Me access isolation, and more needs to rely on upper-layer QoS policy guarantees.

[0181] The main innovations of the embodiments of the present application include: proposing a method combining software and hardware, utilizing the unified management features of N VMe queue resources to achieve resource pooling and access isolation, and increasing resource dynamic scheduling and mapping capabilities.

[0182] Specifically, an independent memory is used to achieve unified management of back-end NVMe devices and present a unified management node for NV Me SSDs; within the system, queue resources are dispersed and redistributed to achieve resource scheduling; a standard NVMe controller is presented to the upper layer, and resource access isolation and dynamic mapping of NameSpace are achieved from the PCIe Function level.

[0183] More specifically, first, during the initialization phase, a mapping relationship between PCIe Function and Name Space is established within the system, and default NVMe Queue resources are allocated to each NameSpace based on default rules. During the runtime phase, the system processes PRP requests sent by the Host and implements memory access and request issuance to the actual NVMe SSD device based on the mapping relationship. At the same time, the number of virtual software queues and physical hardware queues corresponding to each NameSpace is dynamically adjusted through the tags attached to each application, providing a scheduling and arbitration mechanism based on priority, time slice, congestion time, and other angles to ensure that long-term congestion or starvation does not occur. Finally, when accessing the Host application, the PCIe Function is used in conjunction with the IOMMU to achieve complete isolation of the PCIe resources corresponding to each NameSpace, ensuring isolation and security in virtualized access and multi-tenant scenarios.

[0184] During IO QoS scheduling, the priority tag information of the target host's IO request is first used to weight and regularize the corresponding arbitration value to obtain the target host's priority scheduling level. Then, during operation, the mapping relationship between the virtual software queue and the physical queue is dynamically adjusted based on the occupancy load and application status of the virtual software queue to ensure that other hosts do not have to wait for a long time while high-priority tasks are being carried out.

[0185] The following continues to describe the exemplary structure of the disk read / write resource allocation device 354 provided in the embodiment of the present application implemented as a software module. In some embodiments, such as Figure 3 As shown, the software module stored in the disk read / write resource allocation device 354 of the memory 350 may be the disk read / write resource allocation device in the master node 21, including:

[0186] The receiving module 3541 is configured to receive a read / write request queue sent by a host, wherein the read / write request queue includes a host identifier of the host;

[0187] The first determination module 3542 is used to determine the device identifier of the target virtual device corresponding to the host based on the host identifier and the first mapping relationship between the host and the virtual device; the second determination module 3543 is used to determine the space identifier of the target namespace corresponding to the host based on the device identifier and the second mapping relationship between the virtual device and the preset namespace; the third determination module 3544 is used to determine the partition identifier of the target disk partition corresponding to the host based on the space identifier and the third mapping relationship between the preset namespace and the disk partition of the disk; the allocation module 3545 is used to allocate the target disk partition corresponding to the partition identifier to the host, and respond to the read and write requests in the read and write request queue through the target disk partition.

[0188] In some embodiments, the receiving module is further configured to: receive at least one read / write request sent by any host in the host cluster; and write the at least one read / write request into a preset submission queue or completion queue to form the read / write request queue.

[0189] In some embodiments, the device also includes: a resource isolation processing module, which is used to use at least one of the preset namespaces to perform resource isolation processing on the disk resources of each of the disks in the pooled disk array before receiving the read and write request queue sent by the host, and obtain multiple isolated disk partitions, wherein each disk partition has the third mapping relationship with one of the preset namespaces; a resource mapping processing module, which is used to perform resource mapping processing on the disk partition and the preset namespace having the third mapping relationship and a virtual device in the virtual device set, so as to obtain the second mapping relationship between the virtual device and the disk partition and the preset namespace.

[0190] In some embodiments, the apparatus further includes: a setting module configured to pre-set the first mapping relationship between each of the hosts in the host cluster and at least one virtual device in the virtual device set.

[0191] In some embodiments, the resource isolation processing module is also used to: use each of the preset namespaces to perform resource isolation processing on the continuous disk resources corresponding to the continuous addresses in each of the disks, and obtain multiple isolated disk partitions; wherein the disk resources of each of the disk partitions correspond to continuous addresses, and the capacity of each of the disk partitions is the same as or different from the capacity of other disk partitions.

[0192] In some embodiments, among the multiple disks in the disk array, there is the third mapping relationship between each of the preset namespaces and a disk partition in each of the disks; the device also includes: a first priority determination module, used to determine the first priority of the disk partition of any disk corresponding to each preset namespace according to the read and write response speed of the disk partition or the capacity of the disk partition; a priority acquisition module, used to obtain the second priority of the read and write request queue sent by the host; a first matching module, used to match a disk from the multiple disks according to the second priority and the first priority of each of the disk partitions, and determine the disk partition in the matched disk as the target disk partition.

[0193] In some embodiments, among the multiple disks in the disk array, there is the third mapping relationship between each of the preset namespaces and a disk partition in each of the disks; the device also includes: an adding module for adding an initialization tag for the host, the initialization tag being used to identify the target capacity of the target disk partition corresponding to the read and write request queue of the host; a second matching module for matching one or more disks from the multiple disks according to the target capacity; and a target disk partition determination module for determining the disk partition from the matched one or more disks as the target disk partition.

[0194] In some embodiments, the third determination module is also used to: determine the page offset of the target virtual device and the address of the target namespace corresponding to the space identifier; determine the offset value corresponding to the page offset as the offset of the address of the target virtual device relative to the address of the target namespace; determine the offset address after the offset relative to the address of the target namespace as the address of the target disk partition corresponding to the host; and determine the partition identifier of the target disk partition based on the address of the target disk partition.

[0195] In some embodiments, the device also includes: a weight determination module, used to determine the resource allocation weight corresponding to the current read and write request queue of the host; the third determination module is also used to: determine one or more disks as target disks among multiple disks in the disk array based on the space identifier and the resource allocation weight; and among the matched target disks, determine the disk partition corresponding to the third mapping relationship as the target disk partition.

[0196] In some embodiments, the weight determination module is also used to: after adjusting the resource allocation weight of the host, determine the current read and write request volume of the host and the pending read and write request volume in the target virtual device; based on the current read and write request volume and the pending read and write request volume, determine the resource allocation weight corresponding to the current read and write request queue of the host.

[0197] In some embodiments, the weight determination module is also used to: determine a first average of the delay time for responding to the read and write requests in a historical period, and a second average of the delay time for responding to the read and write requests in a current period; if the second average is greater than or equal to the first average, increase the resource allocation weight corresponding to the current read and write request queue of the host; if the second average is less than the first average, reduce the resource allocation weight corresponding to the current read and write request queue of the host.

[0198] In some embodiments, the weight determination module is also used to: determine the delay sensitivity type of the host's read and write request queue; if the read and write request queue is a tail delay-sensitive queue, increase the resource allocation weight corresponding to the host's current read and write request queue, and the increased resource allocation weight is the maximum weight among all read and write request queues in the current disk read and write resource allocation system.

[0199] It should be noted that the description of the device embodiment of the present application is similar to the description of the method embodiment described above, and has similar beneficial effects as the method embodiment, so it will not be repeated. For technical details not disclosed in the device embodiment, please refer to the description of the method embodiment of the present application for understanding.

[0200] The present invention provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method described above in the present invention.

[0201] The embodiment of the present application provides a storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the method provided by the embodiment of the present application, for example, Figure 4 The method shown.

[0202] In some embodiments, the storage medium can be a computer-readable storage medium, such as a ferroelectric random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); it can also be various devices including one or any combination of the above memories.

[0203] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0204] By way of example, executable instructions may, but need not necessarily, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinating files (e.g., files storing one or more modules, subroutines, or code portions). By way of example, executable instructions may be deployed for execution on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0205] The above description is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, and improvements made within the spirit and scope of the present application are included in the scope of protection of the present application.

Claims

1. A method for allocating disk read and write resources, characterized in that: include: Receiving a read / write request queue sent by a host, wherein the read / write request queue includes a host identifier of the host and at least one read / write request; Determining a device identifier of a target virtual device corresponding to the host according to the host identifier and a first mapping relationship between the host and the virtual device; Determining a space identifier of a target namespace corresponding to the host according to the device identifier and a second mapping relationship between the virtual device and the preset namespace; Determining a partition identifier of a target disk partition corresponding to the host according to the space identifier and a third mapping relationship between the preset namespace and the disk partition of the disk; The target disk partition corresponding to the partition identifier is allocated to the host, and the read and write requests in the read and write request queue are responded to through the target disk partition.

2. The method according to claim 1, characterized in that The read and write request queue sent by the receiving host includes: Receive at least one read or write request from any host in the host cluster; The at least one read / write request is written into a preset submission queue or completion queue to form the read / write request queue.

3. The method according to claim 1, characterized in that Each virtual device corresponds to a virtual disk, which is a virtual storage space obtained by processing a physical disk using virtualization technology.

4. The method according to claim 1, wherein The method further comprises: Before receiving the read and write request queue sent by the host, at least one of the preset namespaces is used to perform resource isolation processing on the disk resources of each of the disks in the pooled disk array to obtain a plurality of isolated disk partitions, wherein each disk partition has the third mapping relationship with one of the preset namespaces; The disk partition and the preset namespace having the third mapping relationship are subjected to resource mapping processing with a virtual device in a virtual device set to obtain the second mapping relationship between the virtual device and the disk partition and the preset namespace.

5. The method according to claim 4, characterized in that For each preset namespace, each disk in the disk array corresponds to a disk partition; after determining the target namespace corresponding to the host, it is determined that there are multiple disk partitions corresponding to the disk array.

6. The method according to claim 4, characterized in that The method further comprises: The first mapping relationship between each host in the host cluster and at least one virtual device in the virtual device set is preset.

7. The method according to claim 4, characterized in that The method of using at least one of the preset namespaces to perform resource isolation processing on the disk resources of each disk in the pooled disk array to obtain a plurality of isolated disk partitions includes: Using each of the preset namespaces, performing resource isolation processing on the continuous disk resources corresponding to the continuous addresses in each of the disks to obtain a plurality of isolated disk partitions; The disk resources of each disk partition correspond to continuous addresses, and the capacity of each disk partition is the same as or different from the capacity of other disk partitions.

8. The method according to claim 4, characterized in that Among the multiple disks in the disk array, each of the preset namespaces has the third mapping relationship with a disk partition on each of the disks; the method further includes: For any disk partition corresponding to each preset namespace, determining a first priority of the disk partition according to a read / write response speed of the disk partition or a capacity of the disk partition; Obtaining a second priority level of the read / write request queue sent by the host; According to the second priority and the first priority of each disk partition, a disk is matched from the multiple disks, and the disk partition in the matched disk is determined as the target disk partition.

9. The method according to claim 4, characterized in that Among the multiple disks in the disk array, each of the preset namespaces has the third mapping relationship with a disk partition on each of the disks; the method further includes: Adding an initialization tag to the host, where the initialization tag is used to identify a target capacity of the target disk partition corresponding to the read and write request queue of the host; According to the target capacity, one or more disks are matched among the multiple disks; The disk partition in the matched one or more disks is determined as the target disk partition.

10. The method according to any one of claims 1 to 9, characterized in that The determining, based on the space identifier and a third mapping relationship between the preset namespace and the disk partition of the disk, the partition identifier of the target disk partition corresponding to the host includes: Determine a page offset of the target virtual device and an address of the target namespace corresponding to the space identifier; Determine the offset value corresponding to the page offset as the offset of the address of the target virtual device relative to the address of the target namespace; Determine the offset address after the offset amount relative to the address of the target namespace as the address of the target disk partition corresponding to the host; Determine a partition identifier of the target disk partition according to the address of the target disk partition.

11. The method according to any one of claims 1 to 9, characterized in that The method further includes: determining a resource allocation weight corresponding to a current read / write request queue of the host; The determining, based on the space identifier and a third mapping relationship between the preset namespace and the disk partition of the disk, the partition identifier of the target disk partition corresponding to the host includes: According to the space identifier and the resource allocation weight, determining one or more disks as target disks from a plurality of disks in the disk array; Among the matched target disks, the disk partition corresponding to the third mapping relationship is determined as the target disk partition.

12. The method according to claim 11, characterized in that Determining the resource allocation weight corresponding to the current read and write request queue of the host includes: After adjusting the resource allocation weight of the host, determining the current read and write request amount of the host and the pending read and write request amount in the target virtual device; The resource allocation weight corresponding to the current read / write request queue of the host is determined according to the current read / write request amount and the pending read / write request amount.

13. The method according to claim 11, characterized in that Determining the resource allocation weight corresponding to the current read and write request queue of the host includes: Determine a first average value of delay times for responding to the read / write requests in a historical period and a second average value of delay times for responding to the read / write requests in a current period; If the second mean is greater than or equal to the first mean, increase the resource allocation weight corresponding to the current read and write request queue of the host; If the second mean is smaller than the first mean, the resource allocation weight corresponding to the current read and write request queue of the host is reduced.

14. The method according to claim 11, characterized in that Determining the resource allocation weight corresponding to the current read and write request queue of the host includes: Determining a delay-sensitive type of a read / write request queue of the host; If the read / write request queue is a tail delay-sensitive queue, the resource allocation weight corresponding to the current read / write request queue of the host is increased, and the increased resource allocation weight is the maximum weight of all read / write request queues in the current disk read / write resource allocation system.

15. A device for allocating disk read and write resources, characterized in that: include: A receiving module, configured to receive a read / write request queue sent by a host, wherein the read / write request queue includes a host identifier of the host and at least one read / write request; A first determining module is configured to determine a device identifier of a target virtual device corresponding to the host according to the host identifier and a first mapping relationship between the host and the virtual device; A second determining module is configured to determine a space identifier of a target namespace corresponding to the host according to the device identifier and a second mapping relationship between the virtual device and the preset namespace; a third determining module, configured to determine a partition identifier of a target disk partition corresponding to the host according to the space identifier and a third mapping relationship between the preset namespace and the disk partition of the disk; The allocation module is configured to allocate the target disk partition corresponding to the partition identifier to the host, and respond to the read and write requests in the read and write request queue through the target disk partition.

16. A device for allocating disk read and write resources, characterized in that: include: a memory for storing executable instructions; The processor is configured to implement the disk read and write resource allocation method according to any one of claims 1 to 14 when executing the executable instructions stored in the memory.

17. A computer-readable storage medium, characterized in that Executable instructions are stored, which are used to cause a processor to execute the executable instructions to implement the method for allocating disk read and write resources as described in any one of claims 1 to 14.

18. A computer program product comprising executable instructions or a computer program, characterized in that When the executable instructions or computer program are executed by a processor, the method for allocating disk read and write resources according to any one of claims 1 to 14 is implemented.

Images