Method, device and medium for managing multi-queue tags of a redundant array of independent disks

Abstract

The application discloses a magnetic disk redundancy array multi-queue tag management method, device, equipment and medium, and relates to the technical field of storage, and comprises the following steps: distributing an input / output (I / O) request received to a hardware queue associated with a processor core sending the request; acquiring an idle tag from an independent tag resource pool corresponding to the hardware queue in a locked state; submitting an input / output (I / O) command carrying the idle tag to a submission queue of the hardware queue; acquiring a completion response carrying the idle tag from a completion queue of the hardware queue, and matching a corresponding input / output (I / O) command according to the idle tag; and recycling the idle tag to the independent tag resource pool corresponding to the hardware queue in a locked state. The method makes tag operations on different hardware queues not interfere with each other, eliminates cross-queue lock competition under multi-core concurrency, releases the multi-queue parallel processing capability of a storage device, and improves I / O throughput and reduces processing delay under high queue depth and multi-core environment.

Description

Technical Field

[0001] This invention relates to the field of storage technology, and in particular to a method, apparatus, device, and medium for managing multi-queue tags of redundant disk arrays. Background Technology

[0002] With the continuous development of data center and cloud computing technologies, storage systems are increasingly demanding high throughput and low latency, and multi-queue mechanisms have become a key technology for improving the I / O performance of storage devices.

[0003] In existing technologies, within the storage driver architecture of the Linux operating system, to fully leverage the parallel processing capabilities of multi-core processors and the performance of high-speed storage media, it is typically necessary to maintain an independent command submission and completion channel for each hardware queue or CPU core. However, resource management for these multi-queue components, such as tags, usually employs a centralized management scheme, where all hardware queues share a global tag resource pool, and a global lock protects the allocation and reclamation operations of this resource pool.

[0004] Therefore, when multiple CPU cores concurrently initiate I / O requests, concurrent access to the same tag resource pool leads to severe lock contention, causing a significant amount of CPU time to be wasted waiting for lock resources to be released, rather than effectively executing actual I / O processing tasks. Especially in scenarios with high queue depth and multi-core concurrency, this centralized management approach becomes a key bottleneck restricting the overall performance improvement of the storage system, resulting in the inability to fully utilize the hardware's multi-queue parallel processing capabilities. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and medium for managing multi-queue tags in a redundant disk array. The technical problem it aims to solve is: how to provide an efficient tag management mechanism that can effectively reduce lock contention overhead in multi-core concurrent scenarios and fully leverage the hardware's multi-queue parallel processing capabilities.

[0006] In a first aspect, embodiments of the present invention provide a method for managing multi-queue tags in a disk redundancy array, comprising: In response to a received input / output I / O request, the input / output I / O request is allocated to a hardware queue associated with the processor core that issued the input / output I / O request; Obtain an idle tag from the independent tag resource pool corresponding to the hardware queue, and lock the independent tag resource pool during the acquisition; The input / output I / O command carrying the free tag is written into the submission queue of the hardware queue and submitted to the storage device for processing; Obtain the input / output I / O completion response carrying the idle tag from the completion queue of the hardware queue, and match the corresponding input / output I / O command according to the idle tag; The idle tags are recycled to the independent tag resource pool corresponding to the hardware queue, and the independent tag resource pool is locked during recycling.

[0007] Optionally, before allocating the input / output I / O request to a hardware queue associated with the processor core that issued the input / output I / O request, the method further includes: Get the number of hardware queues supported by the storage device; An independent tag list is initialized for each hardware queue. The tag list contains a preset number of tag nodes, and each tag node carries a unique integer identifier as a tag.

[0008] Optionally, multiple independent tag resource pools are maintained, each of which corresponds uniquely to a hardware queue, and the allocation and recycling operations between independent tag resource pools corresponding to different hardware queues are independent of each other and are lock-free and parallel.

[0009] Optionally, the step of obtaining an idle tag from the independent tag resource pool corresponding to the hardware queue, and locking the independent tag resource pool during the acquisition, includes: The target tag linked list corresponding to the hardware queue is locked, and the target tag linked list stores available tag nodes; In the locked state, an available tag node is taken from the head of the target tag list to obtain the free tag; Unlock the target tag list.

[0010] Optionally, the step of reclaiming the idle tags to the independent tag resource pool corresponding to the hardware queue, and locking the independent tag resource pool during reclamation, includes: Lock the target tag linked list; In the locked state, the tag node corresponding to the free tag is returned to the end of the target tag list; Unlock the target tag list.

[0011] Optionally, the step of writing the input / output I / O command carrying the idle tag into the submission queue of the hardware queue and submitting it to the storage device for processing includes: The idle tag is embedded in a specified field of the SQE structure of the submission queue element to generate an input / output I / O command carrying the tag; Write the tagged input / output I / O command into the submission queue.

[0012] Optionally, the step of retrieving the input / output I / O completion response carrying the idle tag from the completion queue of the hardware queue, and matching the corresponding input / output I / O command according to the idle tag, includes: Obtain the CQE structure of the completion queue element from the completion queue; The free tag is parsed from the CQE structure; Based on the idle tag, the input / output I / O command corresponding to the SQE structure of the submission queue element with the same tag number is matched, and the input / output I / O request is processed.

[0013] Secondly, embodiments of the present invention also provide a disk redundancy array multi-queue tag management device, which includes a unit for performing the above-described method.

[0014] Thirdly, embodiments of the present invention also provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0015] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the above-described method.

[0016] The disk redundant array multi-queue tag management method provided in this invention, upon receiving an I / O request, allocates it to a hardware queue associated with the processor core that issued the request. This establishes a fixed mapping relationship between the I / O processing of each core and the hardware queue and independent tag resource pool bound to that core, avoiding cross-core sharing of resource pools. Furthermore, when retrieving an idle tag from the independent tag resource pool corresponding to the hardware queue, only that resource pool is locked. The scope of the lock is limited to the resource pool corresponding to the current hardware queue. Other processor cores do not need to wait for the lock to be released when accessing the independent tag resource pool corresponding to their respective hardware queues, thereby eliminating the overhead of cross-queue lock contention in multi-core concurrent scenarios. Furthermore, the tag is written to the submission queue along with the I / O command and returned by the hardware in the completion response. The driver can accurately match requests and responses through the tag without performing global search or additional mapping operations, and the matching efficiency is not affected by the number of concurrent requests. Furthermore, after processing is completed, the tag is recycled back to the original independent tag resource pool and locked during recycling, ensuring the consistency of tag resource ownership and forming a complete closed loop from allocation to recycling. The combined effect of these technologies enables multiple hardware queues of storage devices to be driven simultaneously and independently by multiple processor cores, fully leveraging the parallel processing capabilities of the hardware multi-queue system to improve I / O throughput and reduce processing latency in high queue depth and multi-core environments. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a multi-queue tag management method for a redundant disk array provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the system framework for a multi-queue tag management method for a redundant disk array provided in an embodiment of the present invention; Figure 3 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0021] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0022] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0023] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."

[0024] Please see Figure 1 This invention provides a multi-queue tag management method for redundant disk arrays. By establishing an independent tag resource pool for each hardware queue within the driver and employing fine-grained locking at the queue level during tag allocation and reclamation, tag operations between different hardware queues can be executed independently and in parallel. This fully leverages the multi-queue parallel processing capabilities of the storage device hardware, improving the overall I / O throughput and response speed of the storage system. Specifically, the method includes the following steps: S1, in response to a received input / output I / O request, allocate the input / output I / O request to a hardware queue associated with the processor core that issued the input / output I / O request.

[0025] In specific implementation, in response to a received input / output (I / O) request, the I / O request is allocated to a hardware queue associated with the processor core that issued the I / O request. This method is applied to the driver of a standalone redundant hard disk array (RAID) device with multi-queue capabilities. This driver runs in the Linux operating system kernel mode and is responsible for managing the I / O command interaction between the storage device and the control host. The I / O request refers to a read / write operation request submitted downwards from the upper block layer, which includes necessary information such as the operation type, logical block address, and data buffer.

[0026] Furthermore, the hardware queue refers to the commit queue and completion queue pair exposed to the host by the storage device through the peripheral component interconnect standard interface. In a symmetric multiprocessing architecture, each processor core is typically bound to a dedicated hardware queue to reduce cache bounce and lock contention. When a driver receives an I / O request, it obtains the identifier of the processor core currently performing the commit operation, queries the pre-defined binding relationship between the core and the hardware queue, and determines the target hardware queue to which the I / O request should be routed. Furthermore, this queue allocation based on processor core binding ensures that I / O requests initiated by the same core are always processed on the same hardware queue, which is beneficial for CPU cache locality and command processing continuity.

[0027] In some preferred embodiments, before allocating the input / output I / O request to a hardware queue associated with the processor core that issued the input / output I / O request, the method further includes: obtaining the number of hardware queues supported by the storage device; initializing an independent tag list for each hardware queue, the tag list containing a preset number of tag nodes, each tag node carrying a unique integer identifier as a tag.

[0028] In this specific implementation, the focus is on solving the problem of how the driver can pre-establish a complete correspondence between tag resources and hardware queues based on the actual hardware capabilities of the storage device during the startup phase, laying the foundation for efficient tag allocation and recycling during subsequent runtime. The specific implementation method is as follows.

[0029] First, the number of hardware queues supported by the storage device is obtained. During storage device driver initialization, the driver first interacts with the storage device controller to obtain the number of hardware queues supported by the storage device. This interaction can occur by reading specific registers in the storage device peripheral interconnect standard configuration space or by sending a management query command to the storage device controller. The number of hardware queues supported by the storage device determines the number of I / O command channels that the device can process simultaneously. Each hardware queue consists of an independent commit queue and a completion queue pair. The driver saves the obtained number of hardware queues as a global configuration parameter to guide subsequent resource initialization.

[0030] Furthermore, an independent tag linked list is initialized for each hardware queue. This tag linked list contains a preset number of tag nodes, each carrying a unique integer identifier as a tag. After obtaining the number of hardware queues, the driver creates an independent tag linked list for each of these queues. Further, the tag linked list is a specific data structure for implementing an independent tag resource pool. Each node in the linked list represents an available tag, and nodes are connected via pointers. The driver pre-allocates a preset number of tag nodes to each tag linked list. This preset number is typically determined by the submission queue depth of each hardware queue, i.e., the maximum number of pending commands that the hardware queue can simultaneously accommodate. Further, the driver assigns a unique integer identifier as a tag to each tag node. This integer is unique and non-repeating within its respective tag linked list. The initial organization order of tag nodes in the linked list can be natural number order. After initialization, each hardware queue has its own dedicated tag linked list pre-filled with available tag nodes. These tag linked lists serve as a dynamic resource pool for tags during driver operation, continuously supporting tag allocation and recycling operations. Since all tag resources have been pre-created during the initialization phase, there is no need to dynamically create or destroy tag nodes on the hot path of subsequent I / O processing. Only lightweight linked list node retrieval and replacement operations need to be performed.

[0031] This embodiment proactively acquires the number of hardware queues supported by the storage device during the driver initialization phase and pre-builds a tag linked list containing a preset number of tag nodes for each hardware queue, thus establishing the correspondence between tag resources and hardware queues in advance. Furthermore, using a linked list structure to manage tag resources ensures that tag acquisition and recycling operations only require modifying the pointers at the head or tail of the linked list, achieving constant-time efficiency and guaranteeing lightweight and low-latency operation on the I / O hot path. Moreover, the pre-creation of tag nodes concentrates memory allocation overhead into a single initialization phase, avoiding the uncertain latency and memory fragmentation issues that might be introduced by dynamically allocating and releasing tag nodes during I / O processing, thereby ensuring system performance stability and response consistency under prolonged high-load operation.

[0032] In some preferred embodiments, multiple independent tag resource pools are maintained, each independent tag resource pool uniquely corresponds to a hardware queue, and the allocation and recycling operations between independent tag resource pools corresponding to different hardware queues are independent of each other and lock-free in parallel.

[0033] In its specific implementation, this embodiment elaborates on the correspondence between the independent tag resource pool and the hardware queue, as well as the parallel characteristics of tag operations between different hardware queues, revealing the technical principle behind this solution's elimination of the global lock bottleneck from an architectural perspective. The specific implementation method is as follows.

[0034] First, multiple independent tag resource pools are maintained, each uniquely corresponding to a hardware queue. The number of independent tag resource pools maintained internally by the driver is equal to the number of hardware queues supported by the storage device, and a one-to-one binding mapping relationship is established between them. That is, I / O requests from hardware queue zero can only acquire and reclaim tags from independent tag resource pool zero, I / O requests from hardware queue one can only acquire and reclaim tags from independent tag resource pool one, and so on. Furthermore, each independent tag resource pool uses its own independent lock to protect the integrity of its internal tag data structure. This lock only has a mutual exclusion effect on accessors operating on its own tag resource pool, and does not constitute any constraint on accessors of other tag resource pools.

[0035] Furthermore, the allocation and recycling operations between independent tag resource pools corresponding to different hardware queues are independent and lock-free in parallel. When one processor core is performing a tag allocation or recycling operation on independent tag resource pool A and holds the lock on independent tag resource pool A, another processor core can simultaneously perform a tag allocation or recycling operation on independent tag resource pool B, and there is no waiting relationship between the two.

[0036] Furthermore, only when two input / output I / O requests happen to be assigned to the same hardware queue and thus need to access the same independent tag resource pool will the later-arriving request need to briefly wait for the previous request to complete its tag operation and release the lock resource. This design of isolating tag resource pools by hardware queue enables completely parallel execution of I / O processing flows on different hardware queues during the tag management phase. When the storage device supports multiple hardware queues, the parallelism of tag operations between processor cores is positively correlated with the number of hardware queues, and the overall system's I / O processing capacity can expand approximately linearly with the increase in the number of hardware queues.

[0037] This embodiment establishes a one-to-one correspondence between independent tag resource pools and hardware queues, and stipulates that the operations between independent tag resource pools corresponding to different hardware queues are independent and lock-free in parallel, completely eliminating the cross-queue lock contention bottleneck existing in traditional global tag pool schemes at the system architecture level. Furthermore, the tag operations of each hardware queue do not interfere with each other, allowing multiple cores of the processor to simultaneously perform tag allocation and recycling on their respective queues, fully releasing parallel processing capabilities. Moreover, this architecture design becomes increasingly advantageous in scenarios with a larger number of hardware queues supported by the storage device and a larger number of host processor cores, achieving near-linear scaling of I / O throughput and significantly improving the overall performance of the storage system in high-concurrency environments.

[0038] Further, see Figure 2 , Figure 2 This is a schematic diagram of the system framework for a multi-queue tag management method for a redundant disk array provided in an embodiment of the present invention. Figure 2 The diagram illustrates the hardware queue structure corresponding to each of the multiple processor cores and their associated independent tag resource pools. Taking processor cores CPU0 to CPUN as an example, each processor core is associated with a hardware queue, and each hardware queue consists of a submission queue (SQ) and a completion queue (CQ), where SQ represents the submission queue and CQ represents the completion queue. Each hardware queue has its own independent tag management linked list, i.e. Figure 2 The tag management linked list is marked in the text. This linked list is a specific implementation of an independent tag resource pool. Each node in the linked list contains a head pointer and a tag field. The nodes are linked together by pointers to form a set of available tags.

[0039] On the I / O request submission path, the driver retrieves an idle tag from the tag management list corresponding to the hardware queue associated with the current processor core, and embeds the tag into the specified field of the SQE structure of the submission queue element. Figure 2The SQE structure is illustrated by the `sge` and `tag` fields. The SQE carrying the tag is written to the commit queue (SQ), retrieved and processed by the storage device hardware. Along the completion path of the I / O request, the storage device hardware writes the processing result to the completion queue (CQ). Each CQE element in the completion queue carries the same tag as the one used during commit. Figure 2 The CQE structure is illustrated by the CQE other fields and the tag field. After parsing the tag from the CQE structure, the driver matches the corresponding original SQE according to the tag, completes the I / O request processing loop, and returns the tag to the original tag management list for subsequent requests.

[0040] S2, obtain an idle tag from the independent tag resource pool corresponding to the hardware queue, and lock the independent tag resource pool during the acquisition.

[0041] In specific implementation, an idle tag is obtained from the independent tag resource pool corresponding to the hardware queue, and the independent tag resource pool is locked during the acquisition. After determining the target hardware queue, the driver needs to allocate a tag for the input / output I / O request. The tag is a unique integer identifier used to identify the request throughout the entire I / O processing lifecycle. Furthermore, unlike traditional solutions where all hardware queues share a global tag pool protected by a global lock, in this embodiment, each hardware queue maintains its own dedicated independent tag resource pool, which only serves I / O requests allocated to that hardware queue. Further, the driver accesses the independent tag resource pool corresponding to the hardware queue, and before performing the tag acquisition operation, it first locks the independent tag resource pool. The locking operation can be implemented using a spin lock or a mutex lock, and its scope is strictly limited to the tag resource pool of the current hardware queue, without affecting the tag operations of other hardware queues. Further, in the locked state, the driver retrieves an idle tag from the tag resource pool; this idle tag is the unique identifier of the current I / O request. Once the acquisition is complete, the driver immediately unlocks the independent tag resource pool, compressing the lock holding time to only cover critical section operations that involve changes in the tag resource pool's state.

[0042] In some preferred embodiments, the step of obtaining an idle tag from the independent tag resource pool corresponding to the hardware queue, and locking the independent tag resource pool during the acquisition, includes: locking the target tag linked list corresponding to the hardware queue, the target tag linked list storing available tag nodes; in the locked state, retrieving an available tag node from the head of the target tag linked list to obtain the idle tag; and unlocking the target tag linked list.

[0043] In specific implementation, this embodiment clarifies the operational details when the independent tag resource pool is implemented using a linked list data structure, including the locking object, the operation order, and the timing of lock release. The specific implementation method is as follows.

[0044] First, the target tag linked list corresponding to the hardware queue is locked. This linked list stores available tag nodes. The independent tag resource pool is implemented internally by the driver as a tag linked list, which is the target tag linked list to be operated on. Each node in the target tag linked list corresponds to an available tag, and the head node of the list points to the first available tag that can be retrieved. Further, when the driver needs to acquire an idle tag, it first performs a locking operation on the target tag linked list corresponding to the target hardware queue. The purpose of locking is to ensure that no other concurrent operations modify the head pointer or node pointer of the linked list before the current tag acquisition operation is completed, thereby avoiding data contention issues where the same tag node is retrieved by multiple I / O requests simultaneously.

[0045] Furthermore, under the locked state, an available tag node is retrieved from the head of the target tag linked list to obtain the free tag. After successfully acquiring the lock, the driver performs a node removal operation on the head of the target tag linked list under the protection of the lock. Specifically, this involves reading the current node pointed to by the head pointer of the linked list, saving the free tag value recorded in that node, and then modifying the head pointer to point to the node following the original head node, thereby removing the retrieved node from the linked list. Furthermore, the integer identifier carried by the retrieved node is the free tag obtained in this operation. Because the entire retrieval operation is completed under lock protection, even in high-concurrency scenarios where multiple I / O requests are simultaneously allocated to the same hardware queue, it can be guaranteed that each tag node is uniquely allocated to one I / O request.

[0046] Furthermore, the target tag linked list is unlocked. After the node removal operation is completed, the driver immediately performs an unlock operation on the target tag linked list, releasing the lock resource. Furthermore, the lock holding time is strictly limited to the time frame of the critical section operation of removing the head node of the linked list, and does not extend to subsequent time-consuming operations such as I / O command construction or queue writes. This design keeps the lock granularity to a minimum, minimizing the waiting latency caused by concurrent I / O requests on the same hardware queue contending for the same lock.

[0047] This embodiment employs a tag linked list as an independent tag resource pool and explicitly stipulates that the acquisition operation follows a standardized process: first locking, then retrieving a node from the head while locked, and immediately unlocking after the operation. This ensures the atomicity of tag allocation operations and data consistency. Furthermore, the granularity of the lock is precisely controlled within the scope of a single node removal operation in a single linked list, minimizing the lock holding time. Even in high-concurrency scenarios where multiple I / O requests simultaneously wait for tags on the same hardware queue, the lock waiting time for each request is only equivalent to the time of a single linked list node operation. This fine-grained locking strategy minimizes the negative impact of lock operations on I / O processing efficiency while ensuring concurrency safety.

[0048] S3, write the input / output I / O command carrying the idle tag into the submission queue of the hardware queue and submit it to the storage device for processing.

[0049] In specific implementation, the input / output I / O command carrying the free tag is written to the submission queue of the hardware queue and submitted to the storage device for processing. After obtaining the free tag, the driver binds the tag to the current I / O request. The driver constructs an input / output I / O command, which includes fields such as opcode, logical block address, transfer length, and the free tag. The driver writes the input / output I / O command to the next available slot in the submission queue of the hardware queue. Further, the submission queue is a circular buffer shared between the driver and the storage device hardware, located in the host memory, and the storage device hardware reads commands from it via direct memory access. Further, after writing the command, the driver notifies the storage device hardware that a new command is waiting to be processed by updating the doorbell register of the submission queue. After receiving the notification, the storage device hardware retrieves the command from the submission queue and performs the actual disk read / write operation.

[0050] In some preferred embodiments, the step of writing the input / output I / O command carrying the free tag into the submission queue of the hardware queue and submitting it to the storage device for processing includes: embedding the free tag into a specified field of the submission queue element SQE structure to generate an input / output I / O command carrying the tag; and writing the input / output I / O command carrying the tag into the submission queue.

[0051] In its specific implementation, this embodiment clarifies the technical details of how tags are embedded in the submission queue element structure and how commands are written to the submission queue and the hardware is notified for processing. The specific implementation method is as follows.

[0052] The free tag is embedded in a designated field of the Submission Queue Element (SQE) structure to generate an input / output (I / O) command carrying the tag. The Submission Queue Element (SQE) is a command description structure with a pre-agreed format between the driver and the storage device hardware. This structure includes the command opcode, logical block start address, transmission length, and reserved fields defined by the driver and firmware. Further, in this embodiment, the driver and storage device firmware pre-agreed on specific reserved fields in the SQE structure to store tag information. The byte offset and bit width of these fields within the entire SQE structure are mutually agreed upon. After obtaining a free tag from the independent tag resource pool, the driver fills the integer value of the free tag into the designated field of the SQE structure according to the agreed format. Then, other parameters of the I / O request are filled into the corresponding fields of the SQE structure, completing the construction of the input / output (I / O) command carrying the tag.

[0053] Further, the tagged I / O commands are written to the submission queue. The submission queue is a contiguous physical memory region allocated by the driver for each hardware queue during the initialization phase. It is organized as a circular buffer and managed by the head and tail pointers of the hardware queue. Starting from the next available slot indicated by the tail pointer of the submission queue, the driver copies the completed SQE structure to that slot. Further, after writing, the driver updates the tail pointer of the submission queue to the new position and notifies the storage device hardware that a new command has been submitted by writing the updated tail pointer value to the doorbell register in the storage device peripheral interconnect standard address space. After obtaining the tail pointer position from the doorbell register, the storage device hardware reads the newly submitted SQE structure from the host memory via direct memory access and begins executing the I / O commands within it.

[0054] This embodiment achieves effective binding of tag information and I / O commands by embedding idle tags into designated fields of the SQE structure. This ensures that each command submitted to the storage device hardware carries its unique identifier within the driver. Furthermore, the convention for the designated fields in the SQE structure utilizes reserved bits or custom extended areas in the existing command structure, without altering the overall SQE structural layout, maintaining good compatibility with standard storage protocols. The tag is transmitted to the storage device hardware along with the command submission and returned along with completion information after command execution, providing complete data for accurate identification and matching of corresponding commands from the completion queue.

[0055] S4. Obtain the input / output I / O completion response carrying the idle tag from the completion queue of the hardware queue, and match the corresponding input / output I / O command according to the idle tag.

[0056] In specific implementation, the input / output (I / O) completion response carrying the idle tag is obtained from the completion queue of the hardware queue, and the corresponding I / O command is matched according to the idle tag. After the storage device hardware completes the execution of the I / O command, it writes the processing result into the completion queue corresponding to the submission queue. The completion queue is also a circular buffer in the host memory, where completion information is written by the storage device hardware. The storage device hardware notifies the driver of the arrival of a new completion response by sending an interrupt signal or by the driver actively polling. Further, the driver reads a completion entry from the completion queue, which carries the same tag number as when the command was submitted. After parsing the tag number, the driver uses it as a lookup key to perform a matching query in the locally maintained submitted command context table. Since the tag is unique within the same hardware queue, the driver can accurately locate the original I / O command corresponding to the completion response, thereby completing the entire processing loop of the I / O request.

[0057] In some preferred embodiments, the step of obtaining the input / output I / O completion response carrying the idle tag from the completion queue of the hardware queue, and matching the corresponding input / output I / O command according to the idle tag, includes: obtaining a completion queue element CQE structure from the completion queue; parsing the idle tag from the CQE structure; and matching the input / output I / O command corresponding to the submission queue element SQE structure carrying the same tag number according to the idle tag, thereby completing the processing of the input / output I / O request.

[0058] In its specific implementation, this embodiment details the complete process of reading queue element CQE, parsing tags, and executing reverse matching commands. The specific implementation method is as follows.

[0059] First, the Completion Queue Entry (CQE) structure is retrieved from the completion queue. After executing an I / O command, the storage device hardware writes the processing result and status information to the completion queue. The completion queue is also a circular buffer in host memory, where each slot stores a Completion Queue Entry (CQE) structure. Further, the storage device hardware writes the CQE structure to the current tail slot of the completion queue via direct memory access, and then notifies the driver by sending an interrupt signal or by polling the head and tail pointers of the completion queue. Upon notification, the driver reads the CQE structures one by one, starting from the slot indicated by the head pointer of the completion queue. Each CQE structure contains the command execution status code and a tag field that the hardware extracted from the SQE structure and returned unchanged upon submission.

[0060] Further, the free tag is parsed from the CQE structure. After reading the CQE structure, the driver extracts the tag value from the corresponding field of the CQE structure according to the pre-agreed field offset position and bit width. This tag value is completely consistent with the tag value embedded in the SQE structure during the initial submission, and is the only bridge connecting the submission end and the completion end. The driver saves the parsed tag value for subsequent matching operations.

[0061] Further, based on the idle tag, the driver matches the input / output I / O command corresponding to the SQE structure of the submission queue element carrying the same tag number, thus completing the processing of the input / output I / O request. When submitting an I / O command, the driver records the key information in the SQE structure along with the corresponding tag number in the locally maintained pending command context. This context can be organized using a data structure such as a hash table or array, with the tag number as the index key. Further, after parsing the tag value from the CQE structure, the driver uses this tag value as the index key to quickly search within the pending command context, locating the I / O command context corresponding to the original SQE structure that matches the tag value. The driver retrieves the original information of the I / O request from the context, performs subsequent processing such as data buffer synchronization, error status checking, and callbacks to the upper-level block layer, ultimately completing the entire I / O request processing flow.

[0062] This embodiment achieves a highly efficient closed-loop processing mechanism for I / O request submission and completion responses by parsing tags from the CQE structure and using these tags as indexes to quickly match the original submitted I / O commands. The tags act as a bridge connecting the submission and completion stages throughout the process, enabling the driver to accurately associate each completion response with its corresponding original request in massive concurrent I / O scenarios, avoiding the additional time overhead of traversal searches or complex matching algorithms. Furthermore, using tag values ​​directly as index keys provides constant-time lookup efficiency, ensuring that even under high-load scenarios with large queue depths, the processing latency of completion responses remains stable and controllable, providing reliable technical support for achieving high-throughput, low-latency storage I / O processing.

[0063] S5, the idle tags are recycled to the independent tag resource pool corresponding to the hardware queue, and the independent tag resource pool is locked during recycling.

[0064] In specific implementation, the idle tag is recycled to the independent tag resource pool corresponding to the hardware queue, and a locking operation is performed on the independent tag resource pool during recycling. After the I / O request is processed, the idle tag has completed its identification mission and needs to be recycled again so that subsequent requests can continue to use it. The driver recycles the idle tag to the independent tag resource pool corresponding to the hardware queue that originally acquired it. Furthermore, a locking operation is also required on the independent tag resource pool during recycling to ensure the atomicity of the recycling operation and prevent data competition with other tag allocation or recycling operations that are ongoing on the same hardware queue. After recycling is completed, the driver performs an unlocking operation on the independent tag resource pool. Thus, the tag completes the complete life cycle of being taken from the resource pool, accompanied by command submission, returned with completion response, and finally returned to the resource pool.

[0065] This embodiment constructs a management architecture that strictly binds tag resources to hardware queues by configuring an independent tag resource pool for each hardware queue within the storage device driver and stipulating that each I / O request acquires and reclaims tags from the independent tag resource pool corresponding to its assigned hardware queue. Furthermore, since each independent tag resource pool is equipped with its own independent lock, and the lock's scope is strictly limited to the tag resource pool of the current queue, there is no mutual exclusion between tag allocation and reclamation operations on different hardware queues, allowing them to be executed concurrently. Furthermore, this lock-free parallel design at the architectural level fundamentally eliminates the severe lock contention problem caused by all hardware queues sharing a single lock in traditional global tag pool schemes, enabling multiple processor cores to fully utilize the parallel processing capabilities of multiple hardware queues provided by the storage device. Furthermore, in the hot path of I / O processing, the lock holding time is compressed to only cover the lightest critical section operation of tag node access, minimizing the impact of lock overhead on I / O processing efficiency, thereby significantly improving the overall performance and response speed of the storage system in high-concurrency, high-throughput environments, while reducing the average latency of I / O requests.

[0066] In some preferred embodiments, the step of reclaiming the idle tag to the independent tag resource pool corresponding to the hardware queue, and locking the independent tag resource pool during reclamation, includes: locking the target tag list; in the locked state, placing the tag node corresponding to the idle tag back to the tail of the target tag list; and unlocking the target tag list.

[0067] In practice, the target tag linked list is locked. When I / O processing is complete and a tag needs to be returned to the resource pool, the driver determines the hardware queue to which the tag belongs and locates the target tag linked list corresponding to that hardware queue. Before returning a tag node to the linked list, the driver first performs a locking operation on the target tag linked list. The purpose of locking is also to prevent data races on the tail pointer of the linked list when the return operation is executed concurrently with other allocation or reclamation operations.

[0068] Furthermore, in the locked state, the tag node corresponding to the free tag is returned to the tail of the target tag linked list. After successfully acquiring the lock, the driver performs a node insertion operation at the tail of the target tag linked list under the protection of the lock. The driver re-attaches the tag node corresponding to the free tag to be recycled to the end of the linked list. Specifically, this involves modifying the pointer of the current tail node to point to the recycled node, and simultaneously updating the tail pointer of the linked list to point to the newly added node. The design of returning the node to the tail ensures that the recycling of tag resources follows an ordered strategy of first-in-first-out or last-in-first-out, which is beneficial for the balanced utilization of tag resources. If the linked list is arranged in natural number order during initialization, the tag retrieval and replacement operations can maintain a reasonable allocation order overall.

[0069] Furthermore, the target tag linked list is unlocked. After the node insertion operation is completed, the driver immediately performs an unlock operation on the target tag linked list, releasing the lock resources. The lock holding time for the reclamation operation is also strictly limited to the time range of the critical section operation of inserting the tail node of the linked list. Through the process of locking first, then operating, and then unlocking, which is consistent with the acquisition operation in Embodiment 4, the acquisition and reclamation of tags form a complete closed loop of concurrent security.

[0070] This embodiment establishes a standardized process for tag recycling, which involves locking the tag, placing it back at the end of the linked list while locked, and unlocking it immediately after the operation. This, together with the tag acquisition operation, forms a symmetrical concurrency control mechanism. Inserting the reclaimed tag node at the end of the linked list avoids the problem of excessive cache concentration that might occur if recently reclaimed tags are redistributed within a short period, promoting balanced rotation of tag resources among the nodes in the linked list. Furthermore, by strictly controlling the granularity of the lock to the microsecond level for a single node operation, the latency impact on concurrent I / O requests on the same hardware queue is minimized, achieving high-performance tag recycling processing while ensuring data consistency.

[0071] Corresponding to the above-described disk redundancy array (RDA) multi-queue tag management method, the present invention also provides a RDA multi-queue tag management device. This RDA multi-queue tag management device includes a unit for executing the above-described RDA multi-queue tag management method, and can be configured in a terminal or server. Specifically, the RDA multi-queue tag management device includes: An allocation unit is configured to respond to a received input / output I / O request and allocate the input / output I / O request to a hardware queue associated with the processor core that issued the input / output I / O request; The first acquisition unit is used to acquire an idle tag from the independent tag resource pool corresponding to the hardware queue, and to lock the independent tag resource pool during acquisition. The submission unit is used to write the input / output I / O command carrying the free tag into the submission queue of the hardware queue and submit it to the storage device for processing; The matching unit is used to obtain the input / output I / O completion response carrying the idle tag from the completion queue of the hardware queue, and match the corresponding input / output I / O command according to the idle tag; The recycling unit is used to recycle the idle tags to the independent tag resource pool corresponding to the hardware queue, and to lock the independent tag resource pool during recycling.

[0072] In some preferred embodiments, it further includes: The second acquisition unit is used to acquire the number of hardware queues supported by the storage device; An initialization unit is used to initialize an independent tag list for each hardware queue. The tag list contains a preset number of tag nodes, and each tag node carries a unique integer identifier as a tag.

[0073] In some preferred embodiments, multiple independent tag resource pools are maintained, each independent tag resource pool uniquely corresponds to a hardware queue, and the allocation and recycling operations between independent tag resource pools corresponding to different hardware queues are independent of each other and lock-free in parallel.

[0074] In some preferred embodiments, the step of obtaining an idle tag from the independent tag resource pool corresponding to the hardware queue, and locking the independent tag resource pool during the acquisition, includes: The target tag linked list corresponding to the hardware queue is locked, and the target tag linked list stores available tag nodes; In the locked state, an available tag node is taken from the head of the target tag list to obtain the free tag; Unlock the target tag list.

[0075] In some preferred embodiments, the step of reclaiming the idle tags to the independent tag resource pool corresponding to the hardware queue, and locking the independent tag resource pool during reclamation, includes: Lock the target tag linked list; In the locked state, the tag node corresponding to the free tag is returned to the end of the target tag list; Unlock the target tag list.

[0076] In some preferred embodiments, writing the input / output I / O command carrying the free tag into the submission queue of the hardware queue and submitting it to the storage device for processing includes: The idle tag is embedded in a specified field of the SQE structure of the submission queue element to generate an input / output I / O command carrying the tag; Write the tagged input / output I / O command into the submission queue.

[0077] In some preferred embodiments, the step of retrieving the input / output I / O completion response carrying the idle tag from the completion queue of the hardware queue, and matching the corresponding input / output I / O command according to the idle tag, includes: Obtain the CQE structure of the completion queue element from the completion queue; The free tag is parsed from the CQE structure; Based on the idle tag, the input / output I / O command corresponding to the SQE structure of the submission queue element with the same tag number is matched, and the input / output I / O request is processed.

[0078] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned disk redundant array multi-queue tag management device and each unit can be referred to the corresponding description in the foregoing method embodiments. For the sake of convenience and brevity, it will not be repeated here.

[0079] The aforementioned disk redundant array multi-queue tag management device can be implemented as a computer program, which can, for example... Figure 3 It runs on the computer device shown.

[0080] Please see Figure 3 , Figure 3 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a terminal or a server, wherein the server can be a standalone server or a server cluster composed of multiple servers.

[0081] The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0082] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, it causes the processor 502 to execute a multi-queue tag management method for a redundant array of disks.

[0083] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0084] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a disk redundant array multi-queue tag management method.

[0085] The network interface 505 is used for network communication with other devices. Those skilled in the art will understand that the above structure is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. A specific computer device 500 may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements.

[0086] The processor 502 is used to run a computer program 5032 stored in a memory to implement the steps of any of the above method embodiments.

[0087] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0088] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0089] Therefore, the present invention also provides a storage medium. This storage medium may be a computer-readable storage medium. The storage medium stores a computer program. When executed by a processor, the computer program causes the processor to perform the steps of any of the above-described method embodiments.

[0090] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), magnetic disk, or optical disk, or any other physical storage medium capable of storing program code. The computer-readable storage medium can be non-volatile or volatile.

[0091] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0092] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0093] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0094] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0095] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0096] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, this invention is also intended to include these modifications and variations as long as they fall within the scope of the claims and their equivalents.

[0097] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for managing multi-queue tags in a redundant disk array, characterized in that, include: In response to a received input / output I / O request, the input / output I / O request is allocated to a hardware queue associated with the processor core that issued the input / output I / O request; Obtain an idle tag from the independent tag resource pool corresponding to the hardware queue, and lock the independent tag resource pool during the acquisition; The input / output I / O command carrying the free tag is written into the submission queue of the hardware queue and submitted to the storage device for processing; Obtain the input / output I / O completion response carrying the idle tag from the completion queue of the hardware queue, and match the corresponding input / output I / O command according to the idle tag; The idle tags are recycled to the independent tag resource pool corresponding to the hardware queue, and the independent tag resource pool is locked during recycling.

2. The disk redundancy array multi-queue tag management method according to claim 1, characterized in that, Before allocating the input / output I / O request to a hardware queue associated with the processor core that issued the I / O request, the method further includes: Get the number of hardware queues supported by the storage device; An independent tag list is initialized for each hardware queue. The tag list contains a preset number of tag nodes, and each tag node carries a unique integer identifier as a tag.

3. The disk redundancy array multi-queue tag management method according to claim 1, characterized in that, Multiple independent tag resource pools are maintained, each of which corresponds uniquely to a hardware queue, and the allocation and recycling operations between independent tag resource pools corresponding to different hardware queues are independent and lock-free in parallel.

4. The multi-queue tag management method for redundant disk arrays according to claim 1, characterized in that, The step of obtaining an idle tag from the independent tag resource pool corresponding to the hardware queue, and locking the independent tag resource pool during the acquisition, includes: The target tag linked list corresponding to the hardware queue is locked, and the target tag linked list stores available tag nodes; In the locked state, an available tag node is taken from the head of the target tag list to obtain the free tag; Unlock the target tag list.

5. The disk redundancy array multi-queue tag management method according to claim 4, characterized in that, The step of reclaiming the idle tags into the independent tag resource pool corresponding to the hardware queue, and locking the independent tag resource pool during reclamation, includes: Lock the target tag linked list; In the locked state, the tag node corresponding to the free tag is returned to the end of the target tag list; Unlock the target tag list.

6. The disk redundancy array multi-queue tag management method according to claim 1, characterized in that, The step of writing the input / output I / O command carrying the free tag into the submission queue of the hardware queue and submitting it to the storage device for processing includes: The idle tag is embedded in a specified field of the SQE structure of the submission queue element to generate an input / output I / O command carrying the tag; Write the tagged input / output I / O command into the submission queue.

7. The method for managing multi-queue tags in a redundant disk array according to claim 6, characterized in that, The step of retrieving the input / output I / O completion response carrying the idle tag from the completion queue of the hardware queue, and matching the corresponding input / output I / O command according to the idle tag, includes: Obtain the CQE structure of the completion queue element from the completion queue; The free tag is parsed from the CQE structure; Based on the idle tag, the input / output I / O command corresponding to the SQE structure of the submission queue element with the same tag number is matched, and the input / output I / O request is processed.

8. A multi-queue tag management device for a redundant disk array, characterized in that, Includes a unit for performing the method as described in any one of claims 1-7.

9. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, can implement the method as described in any one of claims 1-7.

Images