A disk access method and device, electronic equipment and storage medium
By creating a dynamic input/output command queue in the RAID controller, performance issues when using a mix of SATA and NVMe drives are resolved based on the performance characteristics of different hard drive media, improving overall system performance and efficiency while ensuring the high-speed advantages of NVMe drives.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANDONG YUNHAI GUOCHUANG CLOUD COMPUTING EQUIP IND INNOVATION CENT CO LTD
- Filing Date
- 2024-09-12
- Publication Date
- 2026-06-23
Smart Images

Figure CN118778910B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computers, and in particular to a disk access method and apparatus, electronic device, and storage medium. Background Technology
[0002] As data center workloads increase, the amount of data a single server needs to process also grows. This makes it impossible for traditional single physical hard drives to meet the system's demands in terms of reliability, performance, and capacity. To address this, RAID (Redundant Array of Independent Disks) technology emerged. It improves data reliability and performance by combining multiple independent hard drives into a single logical storage unit. Different RAID levels, such as RAID 0 and RAID 1, can be optimized for different application scenarios based on their data protection strategies and data distribution methods.
[0003] In related technologies, a RAID controller is a dedicated hardware device, typically installed on a computer motherboard or connected to an expansion slot on the motherboard. A RAID controller has its own processor, memory, and interfaces to manage multiple hard drives and implement RAID functionality. In particular, RAID controllers based on the NVMe (Non-Volatile Memory Express) protocol communicate with the host through a command queue located in the host's memory. Specifically, read and write commands issued by the host are placed into the SQ (Submission Queue). The RAID controller then reads the relevant commands from the SQ queue via PCIe (Peripheral Component Interconnect Express) TLP (Transaction Layer Packet) messages and executes them. After execution, the results are returned to the CQ (Completion Queue) in the host's memory via PCIe TLP messages to complete an I / O interaction.
[0004] However, while RAID controllers greatly improve data processing efficiency, performance issues often arise when connecting hard drives with different properties, such as using slower SATA (Serial Advanced Technology Attachment) drives and faster NVMe drives simultaneously. For example, SATA drives may fill the host memory's SQ queue during I / O (Input / Output) operations, preventing the host from issuing I / O commands to NVMe drives. This leads to a decrease in overall system performance and also affects the performance of NVMe drives, preventing them from realizing their high-speed advantages. Summary of the Invention
[0005] In view of the above problems, a disk access method, apparatus, electronic device, and storage medium are proposed to overcome or at least partially solve the above problems, including:
[0006] A disk access method, applied on a host side, the method comprising:
[0007] The processor information of the host is obtained, and an identification command is sent to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller; wherein, the disk array controller is used to respond to the identification command issued by the host and send the input / output queue attribute information supported by the disk array controller to the host.
[0008] Based on the processor information and the input / output queue attribute information, an input / output command queue is created;
[0009] In response to input / output commands issued by the upper-layer application on the host side, a read / write command is generated and the input / output command queue corresponding to the input / output command is determined;
[0010] The read / write command is sent to the disk array controller via the input / output command queue corresponding to the input / output command; wherein, the disk array controller is also used to receive the read / write command and perform disk access according to the read / write command.
[0011] Optionally, creating an input / output command queue based on the processor information and the input / output queue attribute information includes:
[0012] Based on the processor information and the input / output queue attribute information, determine the processor quantity information and the input / output queue attribute type information;
[0013] The input / output command queue is created based on the processor quantity information and the input / output queue attribute type information.
[0014] Optionally, before generating read / write commands and determining the input / output command queue corresponding to the input / output commands in response to input / output commands issued by the upper-layer application on the host side, the method further includes:
[0015] Based on the processor information, the input / output queue attribute information, and the input / output command queue, a queue mapping table is created.
[0016] Optionally, determining the input / output command queue corresponding to the input / output command includes:
[0017] Based on the mapping table between the input / output commands and the queues, determine the input / output command queue corresponding to the input / output command.
[0018] Optionally, the method further includes:
[0019] The system receives an asynchronous event notification sent by the disk array controller and determines, based on the asynchronous event notification, whether an input / output queue attribute change event has occurred on the disk array controller. The disk array controller is also configured to send the asynchronous event notification to the host in response to an asynchronous event occurring on the disk array controller.
[0020] If the input / output queue attribute change event occurs at the disk array controller, the step of sending an identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller is executed again.
[0021] Optionally, after creating the queue mapping table, the method further includes:
[0022] The system receives attribute information of the namespace in the disk array sent by the disk array controller; wherein the disk array controller is also used to send the attribute information of the namespace in the disk array to the host.
[0023] Optionally, determining the input / output command queue corresponding to the input / output command based on the input / output command and queue mapping table includes:
[0024] Determine the attribute information of the namespace requested by the input / output command, and determine the input / output command queue corresponding to the input / output command based on the attribute information of the namespace requested by the input / output command and the queue mapping table.
[0025] Optionally, before obtaining the processor information of the host, the method further includes:
[0026] Obtain the queue depth information and single command size information supported by the disk array controller.
[0027] Optionally, creating an input / output command queue based on the processor information and the input / output queue attribute information includes:
[0028] The input / output command queue is created based on the processor information, the input / output queue attribute information, the queue depth information, and the single command size information.
[0029] Optionally, after creating the input / output command queue, the method further includes:
[0030] The first address information of the input / output command queue is sent to the disk array controller; wherein, the disk array controller is also used to receive the first address information sent by the host and save the first address information into the disk array controller.
[0031] A disk access method, applied to a disk array controller, the method comprising:
[0032] In response to an identification command issued by the host, the system sends input / output queue attribute information supported by the disk array controller to the host; wherein, the host is used to obtain the processor information of the host and send the identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller; and an input / output command queue is created based on the processor information and the input / output queue attribute information.
[0033] The system receives read / write commands and performs disk access based on the read / write commands; wherein, the host is further configured to respond to input / output commands issued by the upper-layer application on the host, generate read / write commands and determine the input / output command queue corresponding to the input / output commands; and send the read / write commands to the disk array controller through the input / output command queue corresponding to the input / output commands.
[0034] Optionally, the method further includes:
[0035] In response to an asynchronous event occurring at the disk array controller, an asynchronous event notification is sent to the host. The host is further configured to receive the asynchronous event notification from the disk array controller and, based on the asynchronous event notification, determine whether an input / output queue attribute change event has occurred at the disk array controller. If the input / output queue attribute change event has occurred at the disk array controller, the step of sending an identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller is executed again.
[0036] Optionally, before receiving the read / write command and performing disk access according to the read / write command, the method further includes:
[0037] The host sends attribute information of the namespace in the disk array to the host, wherein the host is further configured to create a queue mapping table based on the processor information, the input / output queue attribute information, and the input / output command queue; receive attribute information of the namespace in the disk array sent by the disk array controller; determine the attribute information of the namespace requested by the input / output command, and determine the input / output command queue corresponding to the input / output command based on the attribute information of the namespace requested by the input / output command and the queue mapping table.
[0038] Optionally, before sending the attribute information of the namespace in the disk array to the host, the method further includes:
[0039] The namespace is created in the disk array, and the attribute information corresponding to the namespace is recorded.
[0040] Optionally, after sending the input / output queue attribute information supported by the disk array controller to the host, the method further includes:
[0041] The host receives the starting address information sent by the host and saves the starting address information to the disk array controller; wherein, the host is also used to send the starting address information of the input / output command queue to the disk array controller.
[0042] A disk access device, applied on a host side, the device comprising:
[0043] The identification command sending module is used to obtain the processor information of the host and send an identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller; wherein, the disk array controller is used to respond to the identification command issued by the host and send the input / output queue attribute information supported by the disk array controller to the host.
[0044] The command queue creation module is used to create an input / output command queue based on the processor information and the input / output queue attribute information.
[0045] The command queue determination module is used to respond to input / output commands issued by the upper-layer application on the host side, generate read / write commands, and determine the input / output command queue corresponding to the input / output commands;
[0046] The read / write command sending module is used to send the read / write command to the disk array controller through the input / output command queue corresponding to the input / output command; wherein, the disk array controller is also used to receive the read / write command and perform disk access according to the read / write command.
[0047] A disk access device, applied to a disk array controller, the device comprising:
[0048] An attribute information sending module is used to respond to an identification command issued by the host and send the input / output queue attribute information supported by the disk array controller to the host; wherein, the host is used to obtain the processor information of the host and send the identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller; and create an input / output command queue based on the processor information and the input / output queue attribute information;
[0049] The read / write command receiving module is used to receive read / write commands and perform disk access according to the read / write commands; wherein, the host is also used to respond to input / output commands issued by the upper-layer application of the host, generate read / write commands and determine the input / output command queue corresponding to the input / output commands; and send the read / write commands to the disk array controller through the input / output command queue corresponding to the input / output commands.
[0050] An electronic device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the disk access method as described above.
[0051] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the disk access method described above.
[0052] A computer program product includes a computer program that, when executed by a processor, implements the disk access method as described above.
[0053] The embodiments of the present invention have the following advantages:
[0054] In this embodiment of the invention, by acquiring the host's processor information and sending an identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller, and creating an input / output command queue based on the processor information and the input / output queue attribute information, read / write commands are generated in response to input / output commands issued by the upper-layer application on the host, and the corresponding input / output command queue is determined. Then, the read / write commands are sent to the disk array controller through the corresponding input / output command queue for disk access. This method achieves the creation of command queues based on the performance characteristics of different hard disk media and dynamic selection of command queues after the upper-layer application issues input / output commands, improving the overall system response speed and data processing efficiency; avoiding performance degradation of high-speed NVMe disks due to queue congestion; optimizing the performance coordination between different storage devices, ensuring that high-speed storage devices such as NVMe disks can fully utilize their performance advantages; and reducing the waste of system resources and the negative impact of low-speed SATA disks on overall system performance. Attached Figure Description
[0055] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0056] Figure 1 This is a schematic diagram of a RAID controller structure that uses both SATA and NVMe disks in the related technologies provided by this invention;
[0057] Figure 2 This is a schematic diagram of the command queue mechanism in the related technology provided by the present invention;
[0058] Figure 3 This is a flowchart of the steps of a disk access method provided in some embodiments of the present invention;
[0059] Figure 4 This is a schematic diagram of the RAID controller structure for hybrid use of SATA and NVMe disks provided in some embodiments of the present invention;
[0060] Figure 5 This is a flowchart of the steps of a disk access method provided in some embodiments of the present invention;
[0061] Figure 6 This is a schematic diagram of the RAID controller queue initialization process provided in some embodiments of the present invention;
[0062] Figure 7This is a schematic diagram of the asynchronous reporting process for dynamic adjustment of RAID controller queue attributes provided in some embodiments of the present invention;
[0063] Figure 8 This is a schematic diagram of the interaction process between the RAID controller namespace and the host provided in some embodiments of the present invention;
[0064] Figure 9 This is a schematic diagram of the structure of a disk access device provided in some embodiments of the present invention;
[0065] Figure 10 This is a schematic diagram of the structure of a disk access device provided in some embodiments of the present invention. Detailed Implementation
[0066] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0067] As data center workloads increase, the amount of data a single server needs to process also grows. This means that traditional single physical hard drives can no longer meet the system's demands in terms of reliability, performance, and capacity. With the continuous development of storage technology, RAID technology has become an important means of solving this problem. RAID technology improves data reliability and performance by combining multiple independent hard drives into a single logical storage unit. Different RAID levels (such as RAID 0, RAID 1, RAID 5, RAID 6, RAID 10, etc.) have their own advantages and applicable scenarios based on different data protection strategies and data distribution methods, and are widely used in enterprise-level servers, storage systems, and data center environments.
[0068] A RAID controller is a dedicated hardware device, typically installed on the computer motherboard or connected to an expansion slot on the motherboard. A RAID controller has its own processor, memory, and interfaces, used to manage multiple hard drives and implement RAID functionality. NVMe-based RAID controllers communicate with the host through a command queue located in the host's memory. Specifically, read and write commands issued by the host are placed into the SQ queue, and then the RAID controller reads the relevant commands from the SQ queue via PCIe Tlp messages and executes them. After execution, the results are placed back into the CQ queue in the host's memory via PCIe Tlp messages, completing one I / O interaction.
[0069] However, when a RAID controller connects multiple hard drives with different properties (such as different media types), some performance issues may arise. For example, due to the slower speed of SATA drives, I / O commands may fill the host memory's SQ queue, preventing the host from issuing I / O commands to NVMe drives. Alternatively, within the RAID controller, slow I / O from SATA drives may consume excessive system resources, even exhausting them, which can also affect the performance of NVMe drives, ultimately dragging their performance down to match that of SATA drives.
[0070] Despite these performance issues, most enterprises still cannot completely replace SATA drives with NVMe drives due to the performance and price differences between NVMe and SATA drives. The usual practice is to use RAID arrays composed of NVMe drives for scenarios requiring high-speed I / O response, and RAID arrays composed of SATA drives for scenarios requiring large-capacity data storage. Therefore, RAID controllers that use a combination of SATA and NVMe drives are widely used in related technologies.
[0071] like Figure 1 The diagram illustrates the structure of a RAID controller that uses both SATA and NVMe drives in related technologies. The RAID controller communicates with the host via a PCIe bus and connects to SATA drives (HDDs, mechanical hard drives) and NVMe drives (SSDs, solid-state drives) using different media. Internally, the RAID controller manages the NVMe controller, RAID groups, and namespaces. During initialization, command queues (SQs / CQs) are created in the host memory, and each CPU is bound to a specific SQ / CQ. The internal logic groups the connected hard drives into two RAID arrays based on their media, creating a certain number of namespaces on each RAID array. These namespaces are then mapped to the host as block devices for I / O read / write operations. Figure 1 As can be seen from the content, multiple hard drives of different media form different RAID arrays and namespaces, but they all interact through the same command queue in the host memory. Since the performance of SATA drives is much lower than that of NVMe drives, the command queue may be excessively occupied or even filled by SATA drive I / O. This could lead to command response delays or even timeouts on the NVMe drive, dragging its performance down to match that of SATA. The root cause of the above problems is:
[0072] 1. The NVMe protocol is designed for SSDs that use Flash as the storage medium, pursuing high performance (i.e., high bandwidth and high IOPS (Input / Output Operations Per Second)). SATA disks are significantly inferior to NVMe disks in performance, with a difference of up to four orders of magnitude in IOPS performance.
[0073] 2. Due to the design of the command queue mechanism in the host NVMe driver, although the mechanism supports up to 64K queues, the NVMe driver binds the CPU to the command queue one-to-one, allowing I / O requests from namespaces composed of different media hard drives to be submitted to any I / O queue; for example... Figure 2 As shown, the host has 2 CPUs, each bound to an I / O command queue. `head` and `tail` represent the starting position and the position of the element being processed in the current command queue, respectively. `max_depth` represents the maximum depth of the command queue, i.e., the maximum number of elements it can support. However, in practical applications, the following situations often occur:
[0074] Scenario 1: The host upper-layer application issues IO through the CPU (1), and the current IO command queue (1) contains a mixture of NVMe disk IO and SATA disk IO. Although the elements in the current IO command queue (1) are not full (tail is less than depth), when the host issues high-concurrency NVMe disk IO, the number of free elements in the IO command queue (1) cannot meet the concurrent requirements of NVMe disk IO, resulting in a situation of NVMe disk IO concurrent back pressure. It is necessary to wait for the IO command queue (1) to have free elements before continuing to issue NVMe disk IO.
[0075] Scenario 2: The host upper layer application sends IO through the CPU (2). The current IO command queue (2) is full of SATA IO (tail equals depth). When the host sends NVMe disk IO, since there are no free elements in the IO command queue (2), the NVMe disk IO will be blocked. It is necessary to wait for the IO command queue (2) to have free elements before the NVMe disk IO can be sent again.
[0076] As can be seen from the above, when the physical disks connected to the RAID controller use a mix of NVMe and SATA, the performance defects of the host I / O will inevitably reduce the actual performance of the NVMe disks.
[0077] In this embodiment of the invention, based on the core technical concept of creating command queues according to the performance characteristics of different hard disk media and dynamically selecting command queues after input / output commands are issued by the upper-layer application, the disk access method in related technologies has been improved. The invention will be described in detail below with reference to the accompanying drawings:
[0078] Reference Figure 3 The diagram illustrates a flowchart of a disk access method according to some embodiments of the present invention, applied to a host device, and specifically includes the following steps:
[0079] Step 301: Obtain the processor information of the host and send an identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller; wherein, the disk array controller is used to respond to the identification command issued by the host and send the input / output queue attribute information supported by the disk array controller to the host.
[0080] To facilitate understanding of the technical solutions of this invention, the relevant technical terms involved in this invention are explained below:
[0081] RAID stands for Redundant Array of Independent Disks, which is a redundant array composed of many independent disks. It combines many independent disks into a large-capacity disk group, and improves the overall disk system performance by utilizing the additive effect of data provided by individual disks.
[0082] NVMe stands for Non-Volatile Memory Express, a logical device interface specification. Similar to AHCI (Serial ATA Advanced Host Controller Interface), it's a bus transmission protocol specification based on a device logical interface (equivalent to the application layer in communication protocols) used to access non-volatile memory media attached via the PCI Express (PCIe) bus (such as solid-state drives using flash memory), although theoretically, a PCIe bus protocol is not necessarily required.
[0083] SATA stands for Serial Advanced Technology Attachment, a new hard drive interface standard proposed by companies such as Intel, IBM, Maxtor, and Seagate. Because it uses a serial connection, hard drives using the SATA interface are also called serial hard drives.
[0084] PCIe: Short for Peripheral Component Interconnect Express, it is a high-speed serial bus standard used to connect computer motherboards and external devices. It replaces the traditional PCI (Peripheral Component Interconnect) bus and provides higher bandwidth and lower latency.
[0085] IO: Short for Input / Output.
[0086] IOPS: Short for Input / Output Per Second, it is one of the main indicators for measuring the performance of storage media. IOPS refers to the number of read and write requests that the system can handle per second.
[0087] SQ stands for Submission Queue, a queue defined in the NVMe protocol used by the host to store submission commands. It is typically created in the host memory and the address of the queue is communicated to the device through NVMe-related registers mapped to the PCIe Bar space, thus enabling command interaction between the host and the device.
[0088] CQ stands for Completion Queue, a queue defined in the NVMe protocol used by the device to store command completion results. It is typically created in the host memory and the address of the queue is communicated to the device through NVMe-related registers mapped to the PCIe Bar space, thus enabling command interaction between the host and the device.
[0089] Namespace: Namespace is defined in the NVMe protocol and is used to represent a collection of a certain number of logical blocks, equivalent to the concept of a logical volume in storage.
[0090] AER stands for Asynchronous Event Request, which is an asynchronous event request command defined in the NVMe protocol. It is a mechanism used by hosts to register asynchronous event notifications with storage devices.
[0091] AEN stands for Asynchronous Event Notice, an asynchronous event response command defined by the NVMe protocol. It is used to notify the host of the occurrence of asynchronous events when they happen inside the storage device, and the host performs corresponding processing according to different event types.
[0092] In a specific implementation, the disk array controller can be a RAID controller, i.e., a Redundant Array of Independent Disks (RAID) controller. The input / output queue attribute information can be determined by the hard disk media connected to the disk array controller. Based on this, to facilitate the implementation of the disk access method provided in this embodiment, a two-dimensional array can be defined:
[0093] int map[NUM_CPUS][NUM_QUEUES_ATTRS];
[0094] Specifically, this two-dimensional array can be used to store the mapping relationship between CPU, input / output queue attributes, and input / output command queues. NUM_CPUS can represent the number of host CPUs, NUM_QUEUES_ATTRS can represent IO queue attributes (for example, they can be divided into NVMe attributes, SATA attributes, etc. according to different media). The one-dimensional index can represent cpuId (CPU identifier), with a value range of 0-NUM_CPUS-1. The two-dimensional index can represent attrId (attribute identifier), with a value range of 0-NUM_QUEUES_ATTRS-1. The value corresponding to the array map can represent the index of the input / output command queue under a certain CPU and a certain queue attribute.
[0095] Furthermore, information related to RAID controller support for multiple IO attributes can be added to the NVMe protocol's Identify Controller data structure (CNS 01h), as shown in Table 1 below:
[0096] Table 1: Examples of Added Content in the Identifier Controller Data Structure
[0097]
[0098] Furthermore, you can add information related to the IO attributes of the Namespace to the NVMe protocol's Identify Namespace data structure (CNS 00h), as shown in Table 2 below:
[0099] Table 2: Examples of Identifying Added Content in Namespace Data Structures
[0100]
[0101] Furthermore, asynchronous event notifications of changes in IO queue attributes can be added to the AER (Asynchronous Event Request), as shown in Table 3 below:
[0102] Table 3: Examples of adding asynchronous event notification information to asynchronous event request commands
[0103]
[0104] Based on the above, the host can obtain the number of processors on the host and send an identify command (CNS 01h) to the RAID controller to obtain controller information. Specifically, it can obtain information about the device's ability to support multiple I / O queue attributes from the 110th byte of the Identify Controller datastructure. It can determine whether the device supports multiple I / O queue attributes based on Bit0 and obtain the types and quantities of multiple I / O queue attributes supported by the device based on Bits 1 to 7. This information is used to create input / output command queues in subsequent steps.
[0105] In some embodiments of the present invention, before obtaining the processor information of the host, the method further includes:
[0106] Obtain the queue depth information and single command size information supported by the disk array controller.
[0107] In practical applications, in addition to processor information and the input / output queue attribute information supported by the disk array controller, the queue depth information and single command size information supported by the disk array controller can also be obtained. These can be used together in the corresponding steps of creating input / output command queues, improving the practicality of the created input / output command queues and ensuring efficient configuration and performance maximization of the command queues.
[0108] Specifically, after the host completes the allocation of the device Bar space during the initialization phase, it can read the queue depth (i.e., queue depth information) and the size of a single command (i.e., single command size information) from the configuration space capability register of the RAID controller device.
[0109] Step 302: Create an input / output command queue based on the processor information and the input / output queue attribute information;
[0110] In practical implementation, the host can create [(number of CPUs) * (number of IO queue attribute types)] IO command queues in memory based on the number of CPUs on the host, the number of IO queue attribute types (i.e., input / output queue attribute types), the queue depth information supported by the disk array controller, and the single command size information. For example, when the host has 8 CPUs and 2 IO queue attribute types (NVMe and SATA), 16 IO command queues can be created in memory based on the number of CPUs on the host, the number of IO queue attribute types, the queue depth information supported by the disk array controller, and the single command size information.
[0111] In some embodiments of the present invention, creating an input / output command queue based on the processor information and the input / output queue attribute information includes:
[0112] Based on the processor information and the input / output queue attribute information, determine the processor quantity information and the input / output queue attribute type information;
[0113] The input / output command queue is created based on the processor quantity information and the input / output queue attribute type information.
[0114] In practical applications, the host can create [(number of CPUs) * (number of IO queue attribute types)] IO command queues in memory based on the number of CPUs on the host, the number of IO queue attribute types (i.e., input / output queue attribute types), the queue depth information supported by the disk array controller, and the size of a single command. This ensures the integrity and rationality of the mapping relationship and enhances the system's configuration flexibility and scalability.
[0115] In some embodiments of the present invention, after creating the input / output command queue, the method further includes:
[0116] The first address information of the input / output command queue is sent to the disk array controller; wherein, the disk array controller is also used to receive the first address information sent by the host and save the first address information into the disk array controller.
[0117] In practical applications, after the host successfully creates the IO command queue, it can write the starting address of the IO command queue into the control register of the RAID controller device's configuration space. The RAID controller device can then receive the starting address of the IO command queue written by the host through an interrupt and store it internally. Furthermore, the RAID controller device can initialize two registers of the IO command queue to 0. These two registers represent the indices of elements in the queue and are called head and tail, respectively. Head is used to record the position index where the device retrieves a command from the IO command queue, and tail is used to record the position index where the host puts a command into the RAID controller device's IO command queue (the host can update the tail value after adding an element to the IO command queue, and the RAID controller device can update the head value after retrieving an element from the IO command queue).
[0118] In some embodiments of the present invention, creating an input / output command queue based on the processor information and the input / output queue attribute information includes:
[0119] The input / output command queue is created based on the processor information, the input / output queue attribute information, the queue depth information, and the single command size information.
[0120] In practical implementation, the host can create [(number of CPUs) * (number of IO queue attribute types)] IO command queues in memory based on the number of CPUs on the host, the number of IO queue attribute types (i.e., input / output queue attribute types), the queue depth information supported by the disk array controller, and the single command size information. For example, when the host has 8 CPUs and 2 IO queue attribute types (NVMe and SATA), 16 IO command queues can be created in memory based on the number of CPUs on the host, the number of IO queue attribute types, the queue depth information supported by the disk array controller, and the single command size information.
[0121] Step 303: In response to the input / output command issued by the upper-layer application on the host side, generate a read / write command and determine the input / output command queue corresponding to the input / output command;
[0122] In specific implementations, such as Figure 4As shown, during RAID controller initialization, the host driver can obtain RAID controller information via the `identify` command (i.e., obtain the RAID controller's multi-IO queue attribute support capability and the number of attribute types based on byte 110 of the Controller data structure (CNS 01h) (Bit 0 represents the multi-IO queue attribute support capability (1 indicates support, 0 indicates no support), Bits 1 to 7 represent the number of IO queue attribute types)). During IO queue binding, a two-dimensional array can be defined based on the number of CPUs and the number of queue attributes to store the mapping relationship between CPUs, IO queue attributes, and IO command queue indices. Figure 4 As can be seen from the content, the host has 8 CPUs and 2 types of IO queue attributes (NVMe and SATA). Therefore, we can define an int map[8][2] to store the queue mapping relationship. Assuming that queue indices 1, 3, 5, ..., 15 correspond to the SATA disk IO command queues of 8 CPUs, and queue indices 2, 4, 6, ..., 16 correspond to the NVMe disk IO command queues of 8 CPUs, we can obtain a two-dimensional array of mapping relationships as follows:
[0123] map[8][2]={{1,2},{3,4},{5,6},{7,8},{9,10},{11,12},{13,14},{15,16}}.
[0124] Therefore, after the host successfully creates the IO queue, it can request a queue mapping table (map) based on the number of CPUs and the number of queue attributes. It then binds the cpuID, queue attribute ID, and queue index one-to-one, generating a multi-dimensional array `int map[i][j]` representing the binding relationship (where i represents cpuId, j represents queue attribute ID, and the value of `map[i][j]` represents the mapped input / output command queue ID). This allows the host to directly determine the corresponding input / output command queue based on the input / output commands issued by the upper-layer application and the mapping table, thus enabling disk access.
[0125] In some embodiments of the present invention, before generating read / write commands and determining the input / output command queue corresponding to the input / output commands in response to input / output commands issued by the upper-layer application on the host side, the method further includes:
[0126] Based on the processor information, the input / output queue attribute information, and the input / output command queue, a queue mapping table is created.
[0127] In practical applications, after the host successfully creates an IO queue, it can request a queue mapping table (map) based on the number of CPUs and the number of queue attributes. It then binds the cpuID, queue attribute ID, and queue index one-to-one, generating a multi-dimensional array intmap[i][j] (where i represents cpuId, j represents queue attribute ID, and the value of map[i][j] represents the mapped input / output command queue ID). This allows the host to determine the corresponding input / output command queue based on the input / output commands issued by the upper-layer application and the mapping table, thus enabling disk access.
[0128] In some embodiments of the present invention, determining the input / output command queue corresponding to the input / output command includes:
[0129] Based on the mapping table between the input / output commands and the queues, determine the input / output command queue corresponding to the input / output command.
[0130] In practical applications, such as Figure 4 As shown, during RAID controller initialization, the host driver can obtain RAID controller information via the `identify` command (i.e., obtain the RAID controller's multi-IO queue attribute support capability and the number of attribute types based on byte 110 of the Controller data structure (CNS 01h) (Bit 0 represents the multi-IO queue attribute support capability (1 indicates support, 0 indicates no support), Bits 1 to 7 represent the number of IO queue attribute types)). During IO queue binding, a two-dimensional array can be defined based on the number of CPUs and the number of queue attributes to store the mapping relationship between CPUs, IO queue attributes, and IO command queue indices. Figure 4 As can be seen from the content, the host has 8 CPUs and 2 types of IO queue attributes (NVMe and SATA). Therefore, we can define an int map[8][2] to store the queue mapping relationship. Assuming that queue indices 1, 3, 5, ..., 15 correspond to the SATA disk IO command queues of 8 CPUs, and queue indices 2, 4, 6, ..., 16 correspond to the NVMe disk IO queues of 8 CPUs, we can obtain a two-dimensional array of mapping relationships as follows:
[0131] map[8][2]={{1,2},{3,4},{5,6},{7,8},{9,10},{11,12},{13,14},{15,16}}.
[0132] Therefore, after the host successfully creates the IO queue, it can request a queue mapping table (map) based on the number of CPUs and the number of queue attributes. It then binds the cpuID, queue attribute ID, and queue index one-to-one, generating a multi-dimensional array `int map[i][j]` representing the binding relationship (where i represents cpuId, j represents queue attribute ID, and the value of `map[i][j]` represents the mapped input / output command queue ID). This allows the host to directly determine the corresponding input / output command queue based on the input / output commands issued by the upper-layer application and the mapping table, enabling disk access. This helps to accurately direct input / output commands to the appropriate command queue, improving processing efficiency and system response speed.
[0133] In some embodiments of the present invention, after creating the queue mapping table, the method further includes:
[0134] The system receives attribute information of the namespace in the disk array sent by the disk array controller; wherein the disk array controller is also used to send the attribute information of the namespace in the disk array to the host.
[0135] In practical applications, multiple SATA disks connected to the RAID controller can be combined into a RAID array, denoted as RAID#0. Three different namespaces can be created on RAID#0, denoted as NS#0, NS#1, and NS#2, and the IO attribute corresponding to each namespace can be marked as 1 (SATA). Multiple NVMe disks can then be combined into another RAID array, denoted as RAID#1. Three different namespaces can be created on RAID#1, denoted as NS#3, NS#4, and NS#5, and the IO attribute corresponding to each namespace can be marked as 2 (NVMe). This allows NS#0, NS#1, and NS#2 to be indirectly bound to the odd-numbered index queues on each CPU, and NS#3, NS#4, and NS#5 to the even-numbered index queues on each CPU, based on the IO attribute type. When the host performs IO read / write operations on NS#0, NS#1, and NS#2, it can directly interact with commands through the corresponding SATA attribute queues on the CPU. When the host performs IO read / write operations on NS#3, NS#4, and NS#5, it can directly interact with commands through the corresponding NVMe attribute queues on the CPU.
[0136] Based on this, after creating the queue mapping table, the disk array controller can send the attribute information (including IO queue attributes) of the marked namespaces in the disk array to the host. This allows the controller to determine the corresponding input / output command queue based on the attribute information of the namespace accessed by the input / output command request and the queue mapping table. This isolates SATA and NVMe on the host IO path, ensuring their respective performance and thus achieving overall IO acceleration for the RAID controller.
[0137] In some embodiments of the present invention, determining the input / output command queue corresponding to the input / output command according to the input / output command mapping table includes:
[0138] Determine the attribute information of the namespace requested by the input / output command, and determine the input / output command queue corresponding to the input / output command based on the attribute information of the namespace requested by the input / output command and the queue mapping table.
[0139] In the specific implementation, after receiving and saving the Namespace attribute information (including IO queue attributes) sent by the disk array controller, the host can generate read and write commands upon receiving an IO request for that Namespace (i.e., input / output commands issued by the upper-layer application on the host). The host then routes the IO command to the specified IO command queue based on the IO queue attribute information of that Namespace and the queue mapping table to complete an IO interaction, thereby further optimizing disk read and write performance and reducing latency.
[0140] Specifically, the overall process of the host receiving and saving the Namespace attribute information sent by the disk array controller can include:
[0141] After the host-side initialization is complete and the command queue mapping table is generated, the RAID controller can create a RAID array, create namespaces on the array, mark the queue attributes corresponding to the namespaces, generate asynchronous notifications of namespace attribute changes, and send them to the host. Upon receiving the AEN (Namespace Entry), the host can initiate a rescan namespace request to the RAID controller (i.e., send an Identify Active Namespace ID list request). The RAID controller can assemble an identify command response, sending the list of active namespace IDs in the system to the host (the RAID controller generates a CQE (Completion Queue Entry) for the Identify Active Namespace ID list). The host can then update its locally stored namespace list and send an Identify Namespace data structure request to the RAID controller. The RAID controller can then assemble an identify command response, sending detailed namespace information (including IO queue attributes) to the host (the RAID controller generates an Identify Namespace data structure request). The host can then update the Namespace attribute information (including IO queue attributes) stored on the local end, wait for IO requests for that Namespace, and route them to the specified IO command queue according to the IO queue attributes to complete an IO interaction.
[0142] Step 304: The read / write command is sent to the disk array controller via the input / output command queue corresponding to the input / output command; wherein, the disk array controller is also used to receive the read / write command and perform disk access according to the read / write command.
[0143] In practical applications, after the host side binds the CPU and IO queue attributes to the IO command queue mapping relationship, it can wait for the upper-layer application on the host side to send input and output commands, generate corresponding read and write commands, and route the read and write commands to the specified IO command queue through the binding relationship. Based on the IO command queue, the read and write commands are sent to the disk array controller for disk access processing.
[0144] In some embodiments of the present invention, the method further includes:
[0145] The system receives an asynchronous event notification sent by the disk array controller and determines, based on the asynchronous event notification, whether an input / output queue attribute change event has occurred on the disk array controller. The disk array controller is also configured to send the asynchronous event notification to the host in response to an asynchronous event occurring on the disk array controller.
[0146] If the input / output queue attribute change event occurs at the disk array controller, the step of sending an identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller is executed again.
[0147] In practical implementation, when the RAID controller initializes on the host side, it can automatically send an AER (Asynchronous Event Request) command to register asynchronous notifications generated by the RAID controller for a certain event. Specifically, when the IO queue attributes change internally in the RAID controller, it can first check whether there is a blocking wait for an AER command. If there is a blocking wait for an AER command, the RAID controller can assemble the response CQE (called Asynchronous Event Notice) for the AER command based on the event type and event information, and interrupt the notification to the host to retrieve the CQE from the CQ to complete the AEN processing. If there is no blocking wait for an AER command, the event generated can be directly saved in the internal event queue of the RAID controller, and AEN can be sent after the AER command is issued.
[0148] Furthermore, after receiving AEN, the host can obtain the event type and event information through Dword 0. If the event type is Notice and the event information is QueueAttribute Changed, it can send the identify command (CNS 01h) to the RAID controller again to obtain the RAID controller information. The device's ability to support multiple IO queue attributes can be obtained from the 110th byte of the RAID controller information Identify Controller data structure. That is, Bit0 is used to obtain whether the RAID controller device supports multiple IO queue attributes, and Bits 1 to 7 are used to obtain the number of IO queue attribute types of the RAID controller device.
[0149] Furthermore, the host can update the queue mapping table (map) generated in the previous stage based on the newly acquired queue attribute information and the number of CPUs. After the mapping table is updated, it can continue to wait for new IO commands to be issued, and then route them to the specified IO command queue for processing through the new binding relationship.
[0150] Reference Figure 5 The diagram illustrates a flowchart of a disk access method according to some embodiments of the present invention, applied to a disk array controller, and specifically includes the following steps:
[0151] Step 501: In response to the identification command issued by the host, send the input / output queue attribute information supported by the disk array controller to the host; wherein, the host is used to obtain the processor information of the host and send the identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller; and create an input / output command queue based on the processor information and the input / output queue attribute information.
[0152] In the specific implementation, the host can obtain the number of processors on the host and can send an identify command (CNS 01h) to the RAID controller to obtain controller information. That is, it can obtain the device's ability to support multiple IO queue attributes from the 110th byte of the Identify Controller datastructure. It can determine whether the device supports multiple IO queue attributes based on Bit0, and obtain the types and number of multiple IO queue attributes supported by the device based on Bit1~Bit7.
[0153] Furthermore, the host can create [(number of CPUs) * (number of IO queue attribute types)] IO command queues in memory based on the host's CPU count, the number of IO queue attribute types (i.e., input / output queue attribute types), the queue depth supported by the disk array controller, and the single command size. For example, when the host has 8 CPUs and 2 IO queue attribute types (NVMe and SATA), 16 IO command queues can be created in memory based on the host's CPU count, IO queue attribute types, the queue depth supported by the disk array controller, and the single command size.
[0154] In some embodiments of the present invention, after sending the input / output queue attribute information supported by the disk array controller to the host, the method further includes:
[0155] The host receives the starting address information sent by the host and saves the starting address information to the disk array controller; wherein, the host is also used to send the starting address information of the input / output command queue to the disk array controller.
[0156] In practical applications, after the host successfully creates the IO command queue, it can write the starting address of the IO command queue into the control register of the RAID controller device's configuration space. The RAID controller device can then receive the starting address of the IO command queue written by the host through an interrupt and store it internally. Furthermore, the RAID controller device can initialize two registers of the IO command queue to 0. These two registers represent the indices of elements in the queue and are called head and tail, respectively. Head is used to record the position index where the device retrieves a command from the IO command queue, and tail is used to record the position index where the host puts a command into the RAID controller device's IO command queue (the host can update the tail value after adding an element to the IO command queue, and the RAID controller device can update the head value after retrieving an element from the IO command queue).
[0157] Step 502: Receive read / write commands and perform disk access according to the read / write commands; wherein, the host is further configured to respond to input / output commands issued by the upper-layer application on the host, generate read / write commands and determine the input / output command queue corresponding to the input / output commands; and send the read / write commands to the disk array controller through the input / output command queue corresponding to the input / output commands.
[0158] In specific implementations, such as Figure 4 As shown, during RAID controller initialization, the host driver can obtain RAID controller information via the `identify` command (i.e., obtain the RAID controller's multi-IO queue attribute support capability and the number of attribute types based on byte 110 of the Controller data structure (CNS 01h) (Bit 0 represents the multi-IO queue attribute support capability (1 indicates support, 0 indicates no support), Bits 1 to 7 represent the number of IO queue attribute types)). During IO queue binding, a two-dimensional array can be defined based on the number of CPUs and the number of queue attributes to store the mapping relationship between CPUs, IO queue attributes, and IO command queue indices. Figure 4As can be seen from the content, the host has 8 CPUs and 2 types of IO queue attributes (NVMe and SATA). Therefore, we can define an int map[8][2] to store the queue mapping relationship. Assuming that queue indices 1, 3, 5, ..., 15 correspond to the SATA disk IO command queues of 8 CPUs, and queue indices 2, 4, 6, ..., 16 correspond to the NVMe disk IO queues of 8 CPUs, we can obtain a two-dimensional array of mapping relationships as follows:
[0159] map[8][2]={{1,2},{3,4},{5,6},{7,8},{9,10},{11,12},{13,14},{15,16}}.
[0160] Therefore, after the host successfully creates the IO queue, it can request a queue mapping table (map) based on the number of CPUs and the number of queue attributes. It then binds the cpuID, queue attribute ID, and queue index one-to-one, generating a multi-dimensional array `int map[i][j]` representing the binding relationship (where i represents cpuId, j represents queue attribute ID, and the value of `map[i][j]` represents the mapped input / output command queue ID). This allows the host to directly determine the corresponding input / output command queue based on the input / output commands issued by the upper-layer application and the mapping table, thus enabling disk access.
[0161] Furthermore, after the host side binds the CPU and IO queue attributes with the IO command queue mapping relationship, it can wait for the upper-layer application on the host side to send input and output commands, generate corresponding read and write commands, and route the read and write commands to the specified IO command queue through the binding relationship. Based on the IO command queue, the read and write commands are sent to the disk array controller for disk access processing.
[0162] In some embodiments of the present invention, the method further includes:
[0163] In response to an asynchronous event occurring at the disk array controller, an asynchronous event notification is sent to the host. The host is further configured to receive the asynchronous event notification from the disk array controller and, based on the asynchronous event notification, determine whether an input / output queue attribute change event has occurred at the disk array controller. If the input / output queue attribute change event has occurred at the disk array controller, the step of sending an identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller is executed again.
[0164] In practical implementation, when the RAID controller initializes on the host side, it can automatically send an AER (Asynchronous Event Request) command to register asynchronous notifications generated by the RAID controller for a certain event. Specifically, when the IO queue attributes change internally in the RAID controller, it can first check whether there is a blocking wait for an AER command. If there is a blocking wait for an AER command, the RAID controller can assemble the response CQE (called Asynchronous Event Notice) for the AER command based on the event type and event information, and interrupt the notification to the host to retrieve the CQE from the CQ to complete the AEN processing. If there is no blocking wait for an AER command, the event generated can be directly saved in the internal event queue of the RAID controller, and AEN can be sent after the AER command is issued.
[0165] Furthermore, after receiving AEN, the host can obtain the event type and event information through Dword 0. If the event type is Notice and the event information is QueueAttribute Changed, it can send the identify command (CNS 01h) to the RAID controller again to obtain the RAID controller information. The device's ability to support multiple IO queue attributes can be obtained from the 110th byte of the RAID controller information Identify Controller data structure. That is, Bit0 is used to obtain whether the RAID controller device supports multiple IO queue attributes, and Bits 1 to 7 are used to obtain the number of IO queue attribute types of the RAID controller device.
[0166] Furthermore, the host can update the queue mapping table (map) generated in the previous stage based on the newly acquired queue attribute information and the number of CPUs. After the mapping table is updated, it can continue to wait for new IO commands to be issued, and then route them to the specified IO command queue for processing through the new binding relationship.
[0167] In some embodiments of the present invention, before receiving the read / write command and performing disk access according to the read / write command, the method further includes:
[0168] The host sends attribute information of the namespace in the disk array to the host, wherein the host is further configured to create a queue mapping table based on the processor information, the input / output queue attribute information, and the input / output command queue; receive attribute information of the namespace in the disk array sent by the disk array controller; determine the attribute information of the namespace requested by the input / output command, and determine the input / output command queue corresponding to the input / output command based on the attribute information of the namespace requested by the input / output command and the queue mapping table.
[0169] In practical applications, multiple SATA disks connected to the RAID controller can be combined into a RAID array, denoted as RAID#0. Three different namespaces can be created on RAID#0, denoted as NS#0, NS#1, and NS#2, and the IO attribute corresponding to each namespace can be marked as 1 (SATA). Multiple NVMe disks can then be combined into another RAID array, denoted as RAID#1. Three different namespaces can be created on RAID#1, denoted as NS#3, NS#4, and NS#5, and the IO attribute corresponding to each namespace can be marked as 2 (NVMe). This allows NS#0, NS#1, and NS#2 to be indirectly bound to the odd-numbered index queues on each CPU, and NS#3, NS#4, and NS#5 to the even-numbered index queues on each CPU, based on the IO attribute type. When the host performs IO read / write operations on NS#0, NS#1, and NS#2, it can directly interact with commands through the corresponding SATA attribute queues on the CPU. When the host performs IO read / write operations on NS#3, NS#4, and NS#5, it can directly interact with commands through the corresponding NVMe attribute queues on the CPU.
[0170] Based on this, after creating the queue mapping table, the disk array controller can send the attribute information (including IO queue attributes) of the marked namespaces in the disk array to the host. This allows the controller to determine the corresponding input / output command queue based on the attribute information of the namespace accessed by the input / output command request and the queue mapping table. This isolates SATA and NVMe on the host IO path, ensuring their respective performance and thus achieving overall IO acceleration for the RAID controller.
[0171] Furthermore, after receiving and saving the Namespace attribute information (including IO queue attributes) sent by the disk array controller, the host can generate read / write commands upon receiving an IO request for that Namespace (i.e., input / output commands issued by the host's upper-layer application). The host then routes the IO command to the specified IO command queue based on the Namespace's IO queue attribute information and the queue mapping table to complete an IO interaction, further optimizing disk read / write performance and reducing latency.
[0172] Specifically, the overall process of the host receiving and saving the Namespace attribute information sent by the disk array controller can include:
[0173] After the host-side initialization is complete and the command queue mapping table is generated, the RAID controller can create a RAID array, create namespaces on the array, mark the queue attributes corresponding to the namespaces, generate asynchronous notifications of namespace attribute changes, and send them to the host. Upon receiving the AEN (Authorization Entity), the host can initiate a rescan namespace request to the RAID controller (i.e., send an Identify Active Namespace ID list request). The RAID controller can assemble an identify command response, sending the list of active namespace IDs in the system to the host (the RAID controller generates a CQE for Identify Active Namespace ID list). The host can then update its locally stored namespace list and send an Identify Namespace data structure request to the RAID controller. The RAID controller can then assemble an identify command response, sending detailed namespace information (including IO queue attributes) to the host (the RAID controller generates an Identify Namespace data structure request). The host can then update the Namespace attribute information (including IO queue attributes) stored on the local end, wait for IO requests for that Namespace, and route them to the specified IO command queue according to the IO queue attributes to complete an IO interaction.
[0174] In some embodiments of the present invention, before sending the attribute information of the namespace in the disk array to the host, the method further includes:
[0175] The namespace is created in the disk array, and the attribute information corresponding to the namespace is recorded.
[0176] In practical applications, multiple SATA disks connected to the RAID controller can be combined into a RAID array, denoted as RAID#0. Three different namespaces can be created on RAID#0, denoted as NS#0, NS#1, and NS#2, and the IO attribute corresponding to each namespace can be marked as 1 (SATA). Multiple NVMe disks can then be combined into another RAID array, denoted as RAID#1. Three different namespaces can be created on RAID#1, denoted as NS#3, NS#4, and NS#5, and the IO attribute corresponding to each namespace can be marked as 2 (NVMe). This allows NS#0, NS#1, and NS#2 to be indirectly bound to the odd-numbered index queues on each CPU, and NS#3, NS#4, and NS#5 to the even-numbered index queues on each CPU, based on the IO attribute type. When the host performs IO read / write operations on NS#0, NS#1, and NS#2, it can directly interact with commands through the corresponding SATA attribute queues on the CPU. When the host performs IO read / write operations on NS#3, NS#4, and NS#5, it can directly interact with commands through the corresponding NVMe attribute queues on the CPU.
[0177] Based on this, after creating the queue mapping table, the disk array controller can send the attribute information (including IO queue attributes) of the marked namespaces in the disk array to the host. This allows the controller to determine the corresponding input / output command queue based on the attribute information of the namespace accessed by the input / output command request and the queue mapping table. This isolates SATA and NVMe on the host IO path, ensuring their respective performance and thus achieving overall IO acceleration for the RAID controller.
[0178] The following will combine Figures 6-8 The embodiments of the present invention will be further described as follows:
[0179] like Figure 6 As shown, the RAID controller queue initialization process in this embodiment of the invention can be summarized as follows:
[0180] 1. During the host initialization phase, the allocation of the RAID controller device's Bar space is completed, and the queue supported depth and the size of a single command are read from the RAID controller device's configuration space capability register;
[0181] 2. The host sends an identify command (CNS 01h) to the RAID controller to obtain controller information (IdentifyController data structure). It obtains the device's ability to support multiple IO queue attributes from the 110th byte of the Identify Controller data structure. It obtains whether the device supports multiple IO queue attributes based on Bit0, and obtains the number of multiple IO queue attributes based on Bits 1 to 7.
[0182] 3. The host obtains the number of CPUs and creates [(number of CPUs) * (number of multi-IO queue attributes)] IO command queues in memory based on the number of CPUs, the number of multi-IO queue attributes, the queue depth, and the size of a single command;
[0183] 4. After the host successfully creates the IO command queue, it writes the starting address of the queue into the control register of the RAID controller's device configuration space;
[0184] 5. The RAID controller receives the starting address of the I / O command queue written by the host via an interrupt and stores it internally within the RAID controller.
[0185] 6. The RAID controller device initializes two registers of the IO command queue to 0. The two registers represent the indices of the elements in the queue and are called head and tail, respectively. head is used to record the position index of the RAID controller device when it retrieves a command from the IO command queue, and tail is used to record the position index of the host when it puts a command into the RAID controller device's IO command queue (the host updates the tail value after adding an element to the IO command queue, and the RAID controller device updates the head value after retrieving an element from the IO command queue).
[0186] 7. After the host successfully creates the IO command queue, it requests a queue mapping table map based on the number of CPUs and the number of queue attributes, and binds the cpuID, queue attribute ID, and queue index one by one, generating a multi-dimensional array int map[i][j] with the binding relationship (i represents cpuId, j represents queue attribute ID, and map[i][j] value represents the mapped command queue ID);
[0187] 8. After the host side completes the binding of the CPU, IO queue attributes and IO command queue mapping relationship, it waits for the upper layer to send IO, generates NVMe read and write commands, and routes them to the specified IO command queue for processing through the binding relationship.
[0188] like Figure 7 As shown, the asynchronous reporting process for dynamic adjustment of RAID controller queue attributes in this embodiment of the invention can be summarized as follows:
[0189] 1. When the host is initialized, the RAID controller automatically sends an AER (Asynchronous Event Request) command to register the asynchronous notification generated by the RAID controller for a certain event;
[0190] 2. When the IO queue attributes change inside the RAID controller, it first checks whether there is a blocking wait for the AER command. If so, the RAID controller assembles the response CQE (called Asynchronous Event Notice) for the AER command according to the event type and event information, and interrupts to notify the host to retrieve the CQE from the CQ to complete the AEN processing. Otherwise, the event generated this time is stored in the event queue inside the RAID controller and AEN is sent after the AER command is issued.
[0191] 3. After receiving AEN, the host obtains the event type (Event Type) and event information (Event Information) through Dword 0. If the event type is Notice and the event information is Queue AttributeChanged, the host sends the identify command (CNS 01h) to the RAID controller again to obtain the controller information. The host obtains the RAID controller device's ability to support multiple IO queue attributes from the 110th byte of the Identify Controller data structure. Bit 0 is used to determine whether the RAID controller device supports multiple IO queue attributes, and Bits 1 to 7 are used to determine the types and number of multiple IO queue attributes of the device.
[0192] 4. Based on the newly acquired queue attribute information and the number of CPUs, the host updates the queue mapping table (map) generated during the initialization phase. After the update is complete, it continues to wait for new IO commands and routes them to the specified IO command queue for processing through the new binding relationship.
[0193] like Figure 8 As shown, the interaction process between the RAID controller namespace and the host in this embodiment of the invention can be summarized as follows:
[0194] After the host-side initialization is complete and the command queue mapping table is generated, the RAID controller can create a RAID array, create namespaces on the array, mark the queue attributes corresponding to the namespaces, generate asynchronous notifications of namespace attribute changes, and send them to the host. Upon receiving the AEN (Authorization Entity), the host can initiate a rescan namespace request to the RAID controller (i.e., send an Identify Active Namespace ID list request). The RAID controller can assemble an identify command response, sending the list of active namespace IDs in the system to the host (the RAID controller generates a CQE for Identify Active Namespace ID list). The host can then update its locally stored namespace list and send an Identify Namespace data structure request to the RAID controller. The RAID controller can then assemble an identify command response, sending detailed namespace information (including IO queue attributes) to the host (the RAID controller generates an Identify Namespace data structure request). The host can then update the Namespace attribute information (including IO queue attributes) stored on the local end, wait for IO requests for that Namespace, and route them to the specified IO command queue according to the IO queue attributes to complete an IO interaction.
[0195] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.
[0196] Reference Figure 9 The diagram illustrates a disk access device according to some embodiments of the present invention, applied to a host device, and specifically may include the following modules:
[0197] The identification command sending module 901 is used to obtain the processor information of the host and send an identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller; wherein, the disk array controller is used to respond to the identification command issued by the host and send the input / output queue attribute information supported by the disk array controller to the host.
[0198] The command queue creation module 902 is used to create an input / output command queue based on the processor information and the input / output queue attribute information.
[0199] The command queue determination module 903 is used to generate read / write commands and determine the input / output command queue corresponding to the input / output commands in response to input / output commands issued by the upper-layer application on the host side;
[0200] The read / write command sending module 904 is used to send the read / write command to the disk array controller through the input / output command queue corresponding to the input / output command; wherein, the disk array controller is also used to receive the read / write command and perform disk access according to the read / write command.
[0201] In some embodiments of the present invention, the command queue creation module 902 includes:
[0202] The quantity determination submodule is used to determine the number of processors and the type of input / output queue attributes based on the processor information and the input / output queue attribute information.
[0203] The command queue creation first submodule is used to create the input / output command queue based on the processor quantity information and the input / output queue attribute type information.
[0204] In some embodiments of the present invention, the apparatus further includes:
[0205] The queue mapping table creation module is used to create a queue mapping table based on the processor information, the input / output queue attribute information, and the input / output command queue.
[0206] In some embodiments of the present invention, the command queue determination module 903 includes:
[0207] The command queue determination submodule is used to determine the input / output command queue corresponding to the input / output command based on the input / output command and queue mapping table.
[0208] In some embodiments of the present invention, the apparatus further includes:
[0209] An asynchronous event notification receiving module is used to receive asynchronous event notifications sent by the disk array controller, and determine whether an input / output queue attribute change event has occurred on the disk array controller based on the asynchronous event notification; wherein, the disk array controller is also used to send the asynchronous event notification to the host in response to the asynchronous event occurring on the disk array controller;
[0210] The identification command retransmission module is used to re-execute the step of sending the identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller if the input / output queue attribute change event occurs at the disk array controller.
[0211] In some embodiments of the present invention, the apparatus further includes:
[0212] The namespace attribute information receiving module is used to receive the attribute information of the namespace in the disk array sent by the disk array controller; wherein, the disk array controller is also used to send the attribute information of the namespace in the disk array to the host.
[0213] In some embodiments of the present invention, the command queue determination submodule includes:
[0214] The command queue determination unit is used to determine the attribute information of the namespace requested by the input / output command, and to determine the input / output command queue corresponding to the input / output command based on the attribute information of the namespace requested by the input / output command and the queue mapping table.
[0215] In some embodiments of the present invention, the apparatus further includes:
[0216] The information acquisition module is used to acquire queue depth information and single command size information supported by the disk array controller.
[0217] In some embodiments of the present invention, the command queue creation module 902 includes:
[0218] The command queue creation second submodule is used to create the input / output command queue based on the processor information, the input / output queue attribute information, the queue depth information, and the single command size information.
[0219] In some embodiments of the present invention, the apparatus further includes:
[0220] The first address information sending module is used to send the first address information of the input / output command queue to the disk array controller; wherein, the disk array controller is also used to receive the first address information sent by the host and save the first address information into the disk array controller.
[0221] Reference Figure 10 The diagram illustrates a disk access device according to some embodiments of the present invention, applied to a disk array controller, and specifically may include the following modules:
[0222] The attribute information sending module 1001 is used to send the input / output queue attribute information supported by the disk array controller to the host in response to the identification command issued by the host; wherein, the host is used to obtain the processor information of the host and send the identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller; and create an input / output command queue according to the processor information and the input / output queue attribute information.
[0223] The read / write command receiving module 1002 is used to receive read / write commands and perform disk access according to the read / write commands; wherein, the host terminal is also used to respond to input / output commands issued by the upper-layer application of the host terminal, generate read / write commands and determine the input / output command queue corresponding to the input / output commands; and send the read / write commands to the disk array controller terminal through the input / output command queue corresponding to the input / output commands.
[0224] In some embodiments of the present invention, the apparatus further includes:
[0225] An asynchronous event notification sending module is used to send an asynchronous event notification to the host in response to an asynchronous event occurring at the disk array controller. The host is further used to receive the asynchronous event notification sent by the disk array controller and, based on the asynchronous event notification, determine whether an input / output queue attribute change event has occurred at the disk array controller. If the input / output queue attribute change event has occurred at the disk array controller, the step of sending an identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller is executed again.
[0226] In some embodiments of the present invention, the apparatus further includes:
[0227] The attribute information sending module is used to send attribute information of the namespace in the disk array to the host terminal. The host terminal is further used to create a queue mapping table based on the processor information, the input / output queue attribute information, and the input / output command queue; receive the attribute information of the namespace in the disk array sent by the disk array controller; determine the attribute information of the namespace requested by the input / output command; and determine the input / output command queue corresponding to the input / output command based on the attribute information of the namespace requested by the input / output command and the queue mapping table.
[0228] In some embodiments of the present invention, the apparatus further includes:
[0229] The attribute information recording module is used to create the namespace in the disk array and record the attribute information corresponding to the namespace.
[0230] In some embodiments of the present invention, the apparatus further includes:
[0231] The first address information receiving module is used to receive the first address information sent by the host and save the first address information to the disk array controller; wherein, the host is also used to send the first address information of the input / output command queue to the disk array controller.
[0232] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.
[0233] Some embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the transaction processing method based on a multi-core processor system as described above.
[0234] Some embodiments of the present invention also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the disk access method as described above.
[0235] Some embodiments of the present invention also provide a computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements the disk access method as described above.
[0236] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0237] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0238] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0239] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0240] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0241] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.
[0242] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the aforementioned element.
[0243] The disk access method, apparatus, electronic device, and storage medium provided above have been described in detail. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A disk access method, characterized in that, Applied to the host side, the method includes: The system acquires the processor information of the host and sends an identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller. The disk array controller, in response to the identification command issued by the host, sends the input / output queue attribute information supported by the disk array controller to the host. The input / output queue attribute information is determined by the hard disk medium connected to the disk array controller. Based on the processor information and the input / output queue attribute information, an input / output command queue is created; In response to input / output commands issued by the upper-layer application on the host side, a read / write command is generated and the input / output command queue corresponding to the input / output command is determined; The read / write command is sent to the disk array controller via the input / output command queue corresponding to the input / output command; wherein, the disk array controller is also used to receive the read / write command and perform disk access according to the read / write command; The step of creating an input / output command queue based on the processor information and the input / output queue attribute information includes: Based on the processor information and the input / output queue attribute information, determine the processor quantity information and the input / output queue attribute type information; The input / output command queue is created based on the processor quantity information and the input / output queue attribute type information; Before responding to the input / output commands issued by the upper-layer application on the host side, generating read / write commands, and determining the input / output command queue corresponding to the input / output commands, the method further includes: Create a queue mapping table based on the processor information, the input / output queue attribute information, and the input / output command queue; Determining the input / output command queue corresponding to the input / output command includes: Based on the mapping table between the input / output commands and the queues, determine the input / output command queue corresponding to the input / output command.
2. The method according to claim 1, characterized in that, The method further includes: The system receives an asynchronous event notification sent by the disk array controller and determines, based on the asynchronous event notification, whether an input / output queue attribute change event has occurred on the disk array controller. The disk array controller is also configured to send the asynchronous event notification to the host in response to an asynchronous event occurring on the disk array controller. If the input / output queue attribute change event occurs at the disk array controller, the step of sending an identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller is executed again.
3. The method according to claim 1, characterized in that, After creating the queue mapping table, the following is also included: The system receives attribute information of the namespace in the disk array sent by the disk array controller; wherein the disk array controller is also used to send the attribute information of the namespace in the disk array to the host.
4. The method according to claim 3, characterized in that, The step of determining the input / output command queue corresponding to the input / output command based on the input / output command mapping table includes: Determine the attribute information of the namespace requested by the input / output command, and determine the input / output command queue corresponding to the input / output command based on the attribute information of the namespace requested by the input / output command and the queue mapping table.
5. The method according to claim 1, characterized in that, Before obtaining the processor information of the host, the method further includes: Obtain the queue depth information and single command size information supported by the disk array controller.
6. The method according to claim 5, characterized in that, The step of creating an input / output command queue based on the processor information and the input / output queue attribute information includes: The input / output command queue is created based on the processor information, the input / output queue attribute information, the queue depth information, and the single command size information.
7. The method according to claim 1, characterized in that, After creating the input / output command queue, the following is also included: The first address information of the input / output command queue is sent to the disk array controller; wherein, the disk array controller is also used to receive the first address information sent by the host and save the first address information into the disk array controller.
8. A disk access method, characterized in that, Applied to the disk array controller, the method includes: In response to an identification command issued by the host, the system sends input / output queue attribute information supported by the disk array controller to the host. The host is configured to obtain processor information and send the identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller. The host is also configured to create an input / output command queue based on the processor information and the input / output queue attribute information. Creating the input / output command queue based on the processor information and the input / output queue attribute information includes: determining the number of processors and the type of input / output queue attribute based on the processor information and the input / output queue attribute information; and further determining the number of processors and the type of input / output queue attribute based on the input / output queue attribute information. The method involves creating an input / output command queue based on the processor quantity information and input / output queue attribute type information. The input / output queue attribute information is determined by the hard disk medium connected to the disk array controller. Before generating read / write commands and determining the corresponding input / output command queue in response to input / output commands issued by the upper-layer application on the host side, the method further includes: creating a queue mapping table based on the processor information, the input / output queue attribute information, and the input / output command queue. Determining the corresponding input / output command queue includes: determining the corresponding input / output command queue based on the input / output command and the queue mapping table. The system receives read / write commands and performs disk access based on the read / write commands; wherein, the host is further configured to respond to input / output commands issued by the upper-layer application on the host, generate read / write commands and determine the input / output command queue corresponding to the input / output commands; and send the read / write commands to the disk array controller through the input / output command queue corresponding to the input / output commands.
9. The method according to claim 8, characterized in that, The method further includes: In response to an asynchronous event occurring at the disk array controller, an asynchronous event notification is sent to the host. The host is further configured to receive the asynchronous event notification from the disk array controller and, based on the asynchronous event notification, determine whether an input / output queue attribute change event has occurred at the disk array controller. If the input / output queue attribute change event has occurred at the disk array controller, the step of sending the identification command to the disk array controller again to obtain the input / output queue attribute information supported by the disk array controller is executed again.
10. The method according to claim 8, characterized in that, Before receiving read / write commands and performing disk access based on the read / write commands, the method further includes: The host sends attribute information of the namespace in the disk array to the host, wherein the host is further configured to create a queue mapping table based on the processor information, the input / output queue attribute information, and the input / output command queue; receive attribute information of the namespace in the disk array sent by the disk array controller; determine the attribute information of the namespace requested by the input / output command, and determine the input / output command queue corresponding to the input / output command based on the attribute information of the namespace requested by the input / output command and the queue mapping table.
11. The method according to claim 10, characterized in that, Before sending the attribute information of the namespace in the disk array to the host, the method further includes: The namespace is created in the disk array, and the attribute information corresponding to the namespace is recorded.
12. The method according to claim 8, characterized in that, After sending the input / output queue attribute information supported by the disk array controller to the host, the method further includes: The host receives the starting address information sent by the host and saves the starting address information to the disk array controller; wherein, the host is also used to send the starting address information of the input / output command queue to the disk array controller.
13. A disk access device, characterized in that, Applied to the host side, the device includes: The identification command sending module is used to obtain the processor information of the host and send an identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller; wherein, the disk array controller is used to respond to the identification command issued by the host and send the input / output queue attribute information supported by the disk array controller to the host; the input / output queue attribute information is determined by the hard disk medium connected to the disk array controller; The command queue creation module is used to create an input / output command queue based on the processor information and the input / output queue attribute information. The command queue determination module is used to respond to input / output commands issued by the upper-layer application on the host side, generate read / write commands, and determine the input / output command queue corresponding to the input / output commands; The read / write command sending module is used to send the read / write command to the disk array controller through the input / output command queue corresponding to the input / output command; wherein, the disk array controller is also used to receive the read / write command and perform disk access according to the read / write command; The step of creating an input / output command queue based on the processor information and the input / output queue attribute information includes: Based on the processor information and the input / output queue attribute information, determine the processor quantity information and the input / output queue attribute type information; The input / output command queue is created based on the processor quantity information and the input / output queue attribute type information; The method further includes, before generating read / write commands and determining the input / output command queue corresponding to the input / output commands in response to input / output commands issued by the upper-layer application on the host side, the method further includes: Create a queue mapping table based on the processor information, the input / output queue attribute information, and the input / output command queue; Determining the input / output command queue corresponding to the input / output command includes: Based on the mapping table between the input / output commands and the queues, determine the input / output command queue corresponding to the input / output command.
14. A disk access device, characterized in that, The device, applied to a disk array controller, includes: An attribute information sending module is used to respond to an identification command issued by the host terminal and send input / output queue attribute information supported by the disk array controller to the host terminal. The host terminal is used to obtain processor information of the host and send the identification command to the disk array controller to obtain the input / output queue attribute information supported by the disk array controller. The host terminal is also used to create an input / output command queue based on the processor information and the input / output queue attribute information. Creating the input / output command queue based on the processor information and the input / output queue attribute information includes: determining the number of processors and the type of input / output queue attribute based on the processor information and the input / output queue attribute information. Information; Based on the processor quantity information and the input / output queue attribute type information, create the input / output command queue; the input / output queue attribute information is determined by the hard disk medium connected to the disk array controller; before generating read / write commands and determining the input / output command queue corresponding to the input / output commands in response to input / output commands issued by the upper-layer application on the host side, the method further includes: creating a queue mapping table based on the processor information, the input / output queue attribute information, and the input / output command queue; determining the input / output command queue corresponding to the input / output command includes: determining the input / output command queue corresponding to the input / output command based on the input / output command and the queue mapping table; The read / write command receiving module is used to receive read / write commands and perform disk access according to the read / write commands; wherein, the host is also used to respond to input / output commands issued by the upper-layer application of the host, generate read / write commands and determine the input / output command queue corresponding to the input / output commands; and send the read / write commands to the disk array controller through the input / output command queue corresponding to the input / output commands.
15. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the disk access method as claimed in any one of claims 1 to 7 or 8 to 12.
16. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the disk access method as claimed in any one of claims 1 to 7 or 8 to 12.
17. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the disk access method as described in any one of claims 1 to 7 or 8 to 12.