Submission queue arbitration method of storage device and electronic device

Abstract

The invention discloses a submission queue arbitration method of a storage device and an electronic device, and relates to the technical field of computer storage, instructions sent by a host are stored in a plurality of submission queues, and the plurality of submission queues are divided into a plurality of submission queue groups in advance; a plurality of submission queue groups are divided into a plurality of queue sets in advance, and dynamic arbitration instead of a single fixed priority or a simple polling mode is adopted in hierarchical arbitration of the queue sets, the queue groups and the queues. The multi-level queue arbitration mode solves the problems that in a related arbitration mechanism, multi-tenant / multi-service service quality requirements cannot be finely distinguished, flexibility is lacked, and the overall service quality is affected, and the scheduling precision is improved by refining the queue management unit, so that the scheduling efficiency is improved in a complex multi-tenant and mixed load scene. The service quality requirements of various queues can be met more accurately, and the phenomenon of starvation of low-priority queues is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer storage, and particularly relates to a submission queue arbitration method of a storage device and an electronic device. BACKGROUND

[0002] With the wide application of NVMe (Non-volatile Memory Express, referred to as non-volatile memory) storage devices in data centers, multi-tenant, mixed load and other scenarios have higher requirements for queue arbitration mechanisms. However, related arbitration mechanisms mostly adopt a single priority arbitration mode. In the single priority arbitration mode, the priority relied on has the problem that the inaccuracy of priority setting limits the flexibility and efficiency of resource allocation, and the single priority arbitration mode cannot finely distinguish the QoS requirements of multiple tenants / multiple services, lacks flexibility, easily causes high-priority queue resource monopoly or low-priority queue starvation, and affects the overall QoS (Quality of Service, referred to as service quality). SUMMARY

[0003] The present application provides a submission queue arbitration method of a storage device and an electronic device to at least solve the problems of inaccurate priority setting, low resource allocation efficiency and insufficient fairness in the related art.

[0004] The application provides a submission queue arbitration method of a storage device, comprising the following steps: storing a plurality of instructions sent by a host into a plurality of submission queues; the plurality of submission queues are divided into a plurality of submission queue groups in advance; the plurality of submission queue groups are divided into a plurality of queue sets in advance; a queue set in the plurality of queue sets corresponds to a first priority; the queue set in the plurality of queue sets is divided according to the type of storage medium; different storage medium types correspond to different priorities; the first priority corresponding to the queue set in the plurality of queue sets is determined according to the priority corresponding to the type of storage medium of the queue set in the plurality of queue sets; a submission queue group in the plurality of submission queue groups corresponds to a second priority; a submission queue group in the plurality of submission queue groups corresponds to a plurality of namespaces; the plurality of namespaces correspond to different priorities; a namespace in the plurality of namespaces corresponds to different service types and service levels; the priority corresponding to the namespace in the plurality of namespaces is determined according to the service type and the service level; the second priority corresponding to the submission queue group in the plurality of submission queue groups is the maximum priority in the plurality of namespaces corresponding to the submission queue group in the plurality of submission queue groups; the following arbitration processes are respectively performed in a plurality of preset arbitration periods: according to the first priority corresponding to the queue set in the plurality of queue sets, a current queue set is selected from the plurality of queue sets; according to the second priority corresponding to the submission queue group in the plurality of submission queue groups, a current submission queue group is selected from the current queue set; and the instructions stored in the current submission queue in the current submission queue group are processed by a processor of the storage device.

[0005] The application further provides a submission queue arbitration device of a storage device, comprising: a queue storage module, configured to store a plurality of instructions sent by a host into a plurality of submission queues; the plurality of submission queues are divided into a plurality of submission queue groups in advance; the plurality of submission queue groups are divided into a plurality of queue sets in advance; a queue set in the plurality of queue sets corresponds to a first priority; the queue sets in the plurality of queue sets are divided according to storage medium types; different storage medium types correspond to different priorities; the first priority corresponding to a queue set in the plurality of queue sets is determined according to the priority corresponding to the storage medium type of the queue set; a submission queue group in the plurality of submission queue groups corresponds to a second priority; a submission queue group in the plurality of submission queue groups corresponds to a plurality of namespaces; the plurality of namespaces correspond to different priorities; a namespace in the plurality of namespaces corresponds to different service types and service levels; the priority corresponding to a namespace in the plurality of namespaces is determined according to the service type and the service level; the second priority corresponding to a submission queue group in the plurality of submission queue groups is the maximum priority in the plurality of namespaces corresponding to the submission queue group in the plurality of submission queue groups; and a queue arbitration module, configured to execute the following arbitration processes in a plurality of preset arbitration periods: selecting a current queue set from the plurality of queue sets according to the first priority corresponding to the queue set in the plurality of queue sets, selecting a current submission queue group from the current queue set according to the second priority corresponding to the submission queue group in the plurality of submission queue groups, and processing the instructions stored in a current submission queue in the current submission queue group by a processor of the storage device.

[0006] The application further provides an electronic device, comprising: a memory configured to store a computer program; and a processor configured to execute the computer program to implement the steps of the submission queue arbitration method of the storage device.

[0007] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the submission queue arbitration method of the storage device.

[0008] The application further provides a computer program product, comprising a computer program, and the computer program is executed by a processor to implement the steps of the submission queue arbitration method of the storage device.

[0009] By the present application, the instructions sent by the host are stored in multiple submission queues, and multiple groups of submission queues are divided into multiple submission queue groups in advance; the multiple submission queue groups are divided into multiple queue sets in advance, a three-level logical grouping structure of "queue set (Set) -> submission queue group (Group) -> submission queue (SQ)" is constructed, the priority and weight are set for different levels in combination with the storage medium type, the business type and the service level, the problem of inaccurate priority setting caused by extensive priority setting in the related art is solved, through the multi-level nested priority system, it is ensured that the resource allocation of the basic physical layer (storage medium) and the logical layer (business type) can be accurately controlled, and the strict requirements of QoS in different business and load scenarios are met; secondly, the queue set (Set) is pre-divided according to the storage medium type, the queue set of different medium types corresponds to different first priorities, it is ensured that the critical business and data access operations can first utilize the most suitable storage resources, and the problem of low resource allocation efficiency is solved; each submission queue group corresponds to multiple namespaces, each namespace corresponds to different business types and service levels, and the priority corresponding to each namespace is determined according to the business type and the service level, each submission queue group corresponds to a second priority, and the second priority corresponding to the submission queue group is determined according to the maximum priority in the multiple namespaces associated with the submission queue group, to ensure the priority processing of critical business; in the hierarchical arbitration of the queue set, the queue group and the queue, dynamic arbitration is adopted instead of a single fixed priority or a simple polling method, this multi-level queue arbitration method solves the problems of inaccurate priority setting, low resource allocation efficiency and insufficient fairness in the related arbitration mechanism, through the refined queue management unit, the accuracy of scheduling is improved, so that in the complex multi-tenant and mixed load scenario, the quality of service (QoS) requirements of various queues can be more accurately met, and the "starvation" phenomenon of low-priority queues is avoided. In summary, the multi-level arbitration mechanism can more finely meet the QoS requirements of multi-tenant / multi-business through its flexible hierarchical division and independent scheduling strategy, avoids the rigidity of resource allocation, and improves the overall service quality and user experience. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creating laborious work.

[0011] Figure 1 is a hardware structure block diagram of a submission queue arbitration method of a storage device according to an embodiment of the present application.

[0012] Figure 2is a flowchart of a submission queue arbitration method of a storage device according to an embodiment of the present application.

[0013] Figure 3 is a hierarchical structure diagram of a storage device according to an embodiment of the present application.

[0014] Figure 4 is an architecture diagram of a queue grouping architecture according to an embodiment of the present application.

[0015] Figure 5 is a structural diagram of a submission queue arbitration apparatus of a storage device according to an embodiment of the present application.

[0016] Figure 6 is a structural diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, any other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0018] It should be noted that, in the description of the present application, the terms “comprise”, “contain” or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. The terms “first”, “second” and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0019] In order for those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0020] In combination with the specific application environment architecture or specific hardware architecture on which the submission queue arbitration method of the storage device is executed, the specific application environment architecture or specific hardware architecture is described here.

[0021] The method embodiments provided in the embodiments of the present application can be executed in a server device or similar computing apparatus. Taking the case of running on a server device, Figure 1 is a hardware structure block diagram of the submission queue arbitration method of the storage device according to an embodiment of the present application. As Figure 1 shown, the server device can include one or more ( Figure 1The server device shown in FIG. 1 includes one processor 102 (the processor 102 can include, but is not limited to, a central processing unit (CPU), a microprocessor (MCU), a programmable logic device (FPGA), or a processing device of the like) and a memory 104 for storing data, wherein the server device can further include a transmission device 106 for communication function and an input and output device 108. Those skilled in the art can understand that, Figure 1 The structure shown in FIG. 1 is only schematic and does not limit the structure of the server device. For example, the server device can further include more or less components than those shown in FIG. 1, or have a different configuration from that shown in FIG. 1. Figure 1 Figure 1 The structure shown in FIG. 1 is only schematic and does not limit the structure of the server device. For example, the server device can further include more or less components than those shown in FIG. 1, or have a different configuration from that shown in FIG. 1.

[0022] The memory 104 can be used to store computer programs, such as software programs of application software and modules, for example, a computer program corresponding to the submission queue arbitration method of the storage device in the embodiments of the present application. The processor 102 can execute various functional applications and data processing by running the computer programs stored in the memory 104, that is, implement the above-mentioned method. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, a flash memory, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor 102, and the remote memory can be connected to the server device through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0023] The transmission device 106 is used to receive or send data via a network. The specific examples of the network can include a wireless network provided by a communication provider of the server device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC) which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, RF) module which is used to communicate with the Internet in a wireless manner.

[0024] The embodiments of the present application provide a submission queue arbitration method of a storage device. The method is described in detail in combination with an execution process of the submission queue arbitration method of the storage device.

[0025] The following explains the professional terms appearing in the present application:

[0026] SSD: Solid State Disk, abbreviated as solid state disk.

[0027] ​NVMe: Non-volatile Memory Express, short for Non-volatile Memory.

[0028] SLC: Single Level Cell, short for Single Layer Cell, is a type of flash memory in storage technology, each cell stores 1 bit of data, high performance, long life.

[0029] QLC: Quad Level Cell, short for Quad Layer Cell, is a type of flash memory in storage technology, each cell stores 4 bits of data, large capacity, low cost, but relatively low speed and life.

[0030] PCIe: PCI Express, short for Peripheral Component Interconnect Express.

[0031] QoS: Quality of Service, short for Quality of Service.

[0032] RR: Round Robin, short for Round Robin arbitration algorithm.

[0033] RR (Set): Round Robin (Set), short for Set Round Robin algorithm.

[0034] WRR: Weighted Round Robin Scheduling, short for Weighted Round Robin Scheduling algorithm.

[0035] WRR (SP): Weighted Round Robin (Specific Processing), short for Specific Processing Weighted Round Robin algorithm.

[0036] In this embodiment, a submission queue arbitration of a storage device is provided, Figure 2 is a flowchart of the submission queue arbitration of the storage device according to the embodiment of the application, as Figure 2 shown, the method comprises the following steps:

[0037] In step S202, the host sends a plurality of instructions to a plurality of submission queues; a plurality of submission queue groups are pre-divided into a plurality of submission queue groups; a plurality of queue sets are pre-divided into a plurality of queue sets; a queue set in the plurality of queue sets corresponds to a first priority; the queue set in the plurality of queue sets is divided according to the storage medium type; different storage medium types correspond to different priorities; the first priority corresponding to the queue set in the plurality of queue sets is determined according to the priority corresponding to the storage medium type of the queue set in the plurality of queue sets; a submission queue group in the plurality of submission queue groups corresponds to a second priority; a submission queue group in the plurality of submission queue groups corresponds to a plurality of namespaces; the plurality of namespaces correspond to different priorities; a namespace in the plurality of namespaces corresponds to different service types and service levels; the priority corresponding to the namespace in the plurality of namespaces is determined according to the service type and the service level; the second priority corresponding to the submission queue group in the plurality of submission queue groups is the maximum priority in the plurality of namespaces corresponding to the submission queue group in the plurality of submission queue groups.

[0038] In step S204, the following arbitration processes are performed in a plurality of preset arbitration periods: a current queue set is selected from the plurality of queue sets according to the first priority corresponding to the queue set in the plurality of queue sets; a current submission queue group is selected from the current queue set according to the second priority corresponding to the submission queue group in the plurality of submission queue groups; and the instructions stored in the current submission queue in the current submission queue group are processed by the processor of the storage device.

[0039] The submission queue arbitration method of the storage device of the embodiment of the present application relates to the field of computer storage technology, and is used for optimizing the queue scheduling efficiency and the quality of service (QoS) in a high-concurrency scenario.

[0040] With the wide application of NVMe storage devices in data centers, multi-tenant, mixed load and other scenarios have higher requirements for queue arbitration mechanisms. However, related arbitration mechanisms mostly use single priority or simple polling. In the single priority arbitration mode, the priority depends on the accuracy of the priority setting, which limits the flexibility and efficiency of resource allocation. In addition, the arbitration mechanism using single priority is prone to high-priority queue resource monopoly, which leads to starvation of low-priority queues, and further causes host IO timeout, application interruption and other serious problems. The simple polling arbitration mechanism is prone to failure to execute high-priority commands (such as management configuration and fault recovery commands) in priority, which reduces the system response speed and reliability. In summary, related arbitration mechanisms have problems of inaccurate priority setting, low resource allocation efficiency and insufficient fairness. To solve this problem, the embodiment of the present application proposes a multi-level arbitration mechanism, which manages the submission queue SQ of the NVMe storage device through the hierarchical structure of "queue set (Set) -> submission queue group (Group) -> submission queue (SQ)". The first priority of the queue set is determined according to the type of storage medium, to reflect the performance and persistence characteristics of different media. The second priority of the submission queue group is based on the maximum priority of the service level and the business type of the namespace managed by the submission queue group, to ensure the priority processing of critical business. In arbitration, the current queue set is selected from the Set level according to the first priority of the queue set in the multiple queue sets. Then, the current submission queue group is selected from the current queue set according to the second priority of the submission queue group in the multiple submission queue groups. Finally, the current submission queue is selected in the current submission queue group. Compared with the related arbitration mechanism of single priority or simple polling, the embodiment of the present application effectively solves the limitations of single priority or simple polling and the problem of inaccurate priority setting, realizes accurate priority setting and fine differentiation of multi-tenant and multi-service QoS, ensures fast response of high-priority tasks, and prevents low-priority queues from being "starved".

[0041] The host refers to a server or computing device that initiates I / O operations, sends read and write instructions and other management commands to the storage device. The instructions initiated by the host refer to the operation requests sent by the host to the storage device, including but not limited to read, write, management configuration and fault recovery operations, for instructing the storage device to process the corresponding operations.

[0042] A submission queue (SQ) is a pre-established queue in a storage device for receiving and storing instructions sent by a host. In the embodiments of the present application, multiple submission queues are used to receive different types of concurrent I / O instructions from multiple tenants. Different services or tenants usually implement resource isolation and independent control of service level (QoS) by creating independent submission queues (SQs). Each SQ can be configured with different priorities and weights to ensure that I / O operations between different services or tenants are scheduled according to the established service level, avoiding the impact of critical services by other low-priority services.

[0043] A submission queue group (Group) is a further division within multiple submission queues, and each submission queue group contains different submission queues to more finely control resource allocation.

[0044] Multiple queue sets (Sets) are the highest level in the arbitration mechanism of the embodiments of the present application, and each Set is composed of multiple submission queue groups (Groups). Each Set can be independently configured with priorities and weights to implement cross-group resource scheduling.

[0045] In the embodiments of the present application, the queue grouping architecture includes a three-level logical grouping of “Set→Group→Group”, i.e., the SQs are divided into a three-level logical grouping of “Set→Group→Group”. Each level is provided with configuration attribute parameters. The Set level is provided with Set set parameters required for arbitrating submission queues, for example, the Set set parameters include a configuration priority, a preset weight, and a skip flag SKIP (used to control whether the current Set participates in queue scheduling). The Group level is provided with Group group parameters required for arbitrating submission queues, for example, the Group group parameters include a configuration priority, a preset weight, a belonging Set (a Set sequence number where the current Group belongs), and a skip flag SKIP (used to control whether the current Group participates in queue scheduling). The Queue level is provided with Queue queue parameters required for arbitrating submission queues, for example, the Queue queue parameters include a configuration enabled state Enabled, a belonging Group (a Group where the current Queue belongs), and a skip flag SKIP for controlling whether the Queue participates in arbitration.

[0046] The plurality of submission queues are pre-divided into a plurality of submission queue groups, and the plurality of submission queue groups are pre-divided into a plurality of queue sets. For example, assuming that the storage device has 1025 SQ queues, the 1025 SQ queues are pre-divided into 8 Set sets, and each Set is further subdivided into 8 Group groups, forming a "Set→Group→Group" hierarchy. Specifically, Set1 to Set8 each contain 8 Groups, such as Group1_1 to Group1_8 under Set1. Each Group manages about 16 SQ queues, ensuring fine and efficient scheduling control. In the embodiments of the present application, the queue set, the submission queue group, and the submission queue constitute a hierarchy of the multi-level priority arbitration mechanism, which can effectively manage a large number of concurrent I / O instructions, avoid resource monopoly and "starvation" phenomenon, while maintaining the fairness of scheduling, and meeting the quality of service (QoS) demand in a high-concurrency scenario.

[0047] In the embodiments of the present application, each queue set includes a group of SQs sharing similar storage medium types and QoS requirements, for implementing priority scheduling based on storage medium types. Among them, the storage medium type refers to different physical media for data storage in the NVMe device, such as SLC (single layer cell), QLC (four layer cell) flash. Different medium types correspond to different priorities, reflecting their performance characteristics and cost-effectiveness. For example, Figure 3 A hierarchical structure diagram of a storage device provided by the embodiments of the present application is shown in Figure 3 As shown, the queue set Set1 is for SLC storage medium, and the queue set Set2 is for QLC storage medium, wherein Set1 and Set2 are respectively suitable for different data hot and cold situations. The queue set using SLC is suitable for storing hot data, which has higher performance and higher maximum PE number, but the capacity is limited. The queue set using QLC is suitable for storing cold data, which has lower performance and lower maximum PE number, but has larger capacity. In actual application, different Set types represent different storage medium type attributes, and different priorities can be pre-allocated according to the medium type corresponding to the queue set.

[0048] The queue set in the plurality of queue sets corresponds to a first priority; wherein the first priority corresponds to the priority of the queue set (Set), and the first priority is determined based on the storage medium type represented by the queue set. For example, the first priority corresponding to the queue set refers to the priority corresponding to the storage medium type represented by the queue set. The Set layer priority determines the processing order of the queue set in the arbitration process. The queue set can contain a plurality of queue groups, and each queue set has its own priority setting for priority comparison and scheduling between a plurality of queue sets.

[0049] The submission queue group (Group) is associated with a specific namespace set, that is, one submission queue group is associated with at least one namespace (Namespace, referred to as NS) for more detailed management of I / O requests of different service types and service levels. It can be understood that there are different Groups under different queue sets, that is, different service types and storage capacity namespaces of host applications under different queue sets. The namespace is a logical unit for isolating and managing different data sets in the NVMe device. Each namespace carries a specific service type and service level and has independent storage space and I / O path. Among them, different service types provide different QoS service level services. The service type describes the application scenario or functional category of the data submitted through the namespace, such as real-time transaction, data analysis, log recording, etc. Different service types have different requirements for I / O performance and response time. The service level reflects the quality standard of the data service provided by the namespace, and usually includes transmission rate, delay and throughput, etc. It is closely related to the QoS demand of the service type and is used to guide the setting of priority.

[0050] The plurality of namespaces correspond to different priorities, and the priority corresponding to the namespace in the plurality of namespaces is determined in advance according to the service type and the service level. For example, the weight of the service type and the weight of the service level corresponding to each namespace are set, the ratio between the weight of the service type and the weight of the service level is determined as the priority corresponding to each namespace, or the sum of the weight of the service type and the weight of the service level is determined as the priority corresponding to each namespace.

[0051] The submission queue group in the plurality of submission queue groups corresponds to a second priority. The second priority is determined according to the highest priority in the plurality of namespaces managed in the submission queue group, so as to ensure that the queue of the critical service can be processed preferentially.

[0052] The namespace set associated with each submission queue group is associated with the submission queue in the submission queue group, that is, each submission queue in each submission queue group is associated with one namespace in the namespace set associated with the submission queue group, that is, each submission queue in each submission queue group has a one-to-one or many-to-one mapping relationship with one namespace in the namespace set associated with the submission queue group. As shown in Figure 3 As shown, different Group group numbers are set in different Set types in actual application, different command space NS number attributes (NS1 / NS2…) can be set under each Group, and different QoS service level types are set by binding and mapping different SQ submission queues with NS1, NS2…, etc. during device initialization.

[0053] After the first priority determines the current queue set to be processed, the second priority is used to further refine the scheduling to determine which queue group in the set is prioritized to schedule the queues inside. The queues in the queue group can have different characteristics or service levels, and by setting the second priority, a more fine-grained resource allocation and scheduling strategy can be achieved. Through the above different Set attributes, different Group types, different QoS service attribute levels provided by the Namespace of the different Namespace, and the priority settings of different SQ submission queues, the IO performance isolation of the entire storage device and the service of different levels of QoS are completed.

[0054] Compared with the single priority or simple polling method in the related art, in the embodiment, in the arbitration process, the scheduler of the NVMe device first determines which queue set to process according to the first priority, and then selects which queue group to process in the set according to the second priority, ensuring that when multiple queue sets and multiple submission queue groups are processed, high-priority queue sets and submission queue groups can be prioritized according to the priority and QoS requirements of the business, avoiding the "starvation" phenomenon, and ensuring efficient, fair and dynamic resource scheduling in a complex and variable environment.

[0055] In the embodiment of the present application, the preset arbitration period refers to the time window set by the storage device for queue access arbitration. The queue arbitration process is performed in each period to determine which submission queue instructions are prioritized for processing. The length of the preset arbitration period is crucial to ensuring the IOPS performance target, and can be dynamically adjusted according to system requirements. In the embodiment of the present application, the length of the preset arbitration period can be dynamically determined according to the overall performance target of the storage system and the hardware characteristics. For example, if the target of the system is to achieve 1M IOPS (input / output operations per second), and the minimum arbitration period allowed by the device hardware is 100ns, then the preset arbitration period can be set to 100ns. The length of the preset arbitration period can also be predictively configured in combination with the QoS requirements of the business and historical load data. For example, by analyzing the load of the storage device in the past week, it is found that from 9am to 11am on weekdays is a high-load period, at this time the arbitration period can be set to be shorter (such as 100ns) to quickly respond to I / O requests and ensure performance and response speed during peak business hours. During the night or low-load period, the arbitration period can be appropriately lengthened (such as 200ns) to reduce unnecessary hardware wake-up and energy consumption, while maintaining the stability and efficiency of the system during low load.

[0056] The arbitration process is performed in each preset arbitration period, and the arbitration process specifically includes: at the beginning of the arbitration period, determining a Set as the current queue set from the plurality of preset queue sets through a preset screening mechanism; in the current queue set, further screening and determining a Group as the current submission queue group through the preset screening mechanism; in the current submission queue group, selecting a specific submission queue as the current submission queue through the preset screening mechanism. The preset screening mechanism can be a priority and weight screening mechanism, i.e., screening according to the priority and weight of each to-be-scheduled object, can be a load-based dynamic scheduling mechanism, i.e., dynamically selecting a to-be-scheduled object with relatively low load according to the load of each to-be-scheduled object, or can be a time slice-based fair scheduling mechanism, i.e., allocating a fixed time slice to each to-be-scheduled object to ensure that each to-be-scheduled object can obtain a processing opportunity within a certain time. It can also be other screening mechanisms, which will not be repeated here.

[0057] In each preset arbitration period, a single arbitration process can be performed, or multiple arbitration processes can be performed, depending on the performance requirements and the length of the preset arbitration period. The design of performing a single arbitration process in a preset arbitration period is suitable for ensuring that all queues have the opportunity to be serviced within a fixed time interval, which helps to ensure fairness and avoid the "starvation" phenomenon. The design of performing multiple arbitration processes in a preset arbitration period is suitable for scenarios that require high IOPS (input / output operations per second). By increasing the screening frequency, the system's throughput and response speed can be improved. For example, if the system needs to support extremely high IOPS, the preset arbitration period may need to be shortened and the screening rounds (i.e., arbitration processes) may need to be increased to shorten the time each queue waits to be processed.

[0058] The current queue set refers to the set of queues selected to participate in arbitration decision-making within a certain period. The current submission queue group refers to the submission queue group selected from the current queue set. The current submission queue refers to the specific queue selected in the current submission queue group, which stores host instructions waiting to be processed by a designated core. The designated core refers to a processor core that is pre-selected or configured to specifically process instructions from the current submission queue selected in the current queue set and the current submission queue group in the multi-processor architecture of the storage device.

[0059] In some embodiments, in the case where the designated submission queue group in the current queue set does not contain a submission queue storing instructions, the designated submission queue group is set not to participate in the arbitration process until a submission queue in the designated submission queue group stores instructions, and the designated submission queue group is controlled to participate in the arbitration process.

[0060] The specified submission queue group refers to a submission queue group in the current queue set that does not store instructions. In the case where the specified submission queue group exists in the current queue set, the specified submission queue group is marked as not participating in the queue scheduling of the current arbitration period until at least one SQ starts storing instructions.

[0061] In a certain queue set (Set), if there is a submission queue group (Group) in which all submission queues (SQs) currently do not store any instructions (i.e., the queue is empty), then the queue group will be set to not participate in the current arbitration process. This is to avoid meaningless comparison and screening of empty queues in the arbitration process, thereby saving computing resources and reducing unnecessary arbitration delay. Once any submission queue (SQ) in the empty queue group starts storing instructions (i.e., the queue state changes from empty to non-empty), the participation state of the queue group (Group) will be restored and rejoin the arbitration process. This means that the participation state of the queue group is dynamically changing, which can quickly respond to changes in the queue state and ensure that queues with instructions to be processed can be serviced in a timely manner.

[0062] Optionally, the host sends I / O instructions to the NVMe device through the PCIe bus. The storage device controller of the NVMe device stores these instructions into the submission queue (SQ) pre-allocated to the namespace according to the namespace identifier of its source. The firmware of the NVMe device divides the SQs into different submission queue groups (Group) according to the service type and service level of the namespace, and further divides the queue groups (Group) into queue sets (Set) according to the storage medium type (e.g., SSD). The firmware assigns a first priority to each queue set according to the priority rules preset for different storage medium types. The firmware calculates a second priority of each submission queue group (Group), which is equal to the highest priority of the namespace in its associated namespace set, i.e., the firmware traverses all namespaces in each Group and selects the maximum priority as the second priority of the Group. At the beginning of the preset arbitration period, the arbitration logic module first selects the queue set to be processed in the current period according to the first priority of the queue set (Set). Then, from the selected queue set, the submission queue group (Group) to be processed in the current period is selected according to the second priority of the submission queue group (Group). Finally, the arbitration module selects the specific submission queue (SQ) to be processed in the current period in the selected queue group according to the priority of the queue itself and the scheduling strategy (such as WRR, RR, etc.). The instructions in the selected submission queue are passed to the processor of the NVMe device for data read / write, state update, etc. The processor feeds back the processing result to the host and prepares for the next arbitration period.

[0063] By the embodiment of the present application, the instructions sent by the host are stored in multiple submission queues, and multiple groups of submission queues are divided into multiple submission queue groups in advance; the multiple submission queue groups are divided into multiple queue sets in advance, a three-level logical grouping structure of "queue set (Set) → submission queue group (Group) → submission queue (SQ)" is constructed, and priorities and weights are set for different levels in combination with storage medium types, business types and service levels, so as to solve the problem of inaccurate priority setting caused by extensive priority setting in the related art, and ensure that the resource allocation of the basic physical layer (storage medium) and the logical layer (business type) can be accurately controlled, so as to meet the strict requirements of QoS in different business and load scenarios. Secondly, the queue set (Set) is pre-divided according to the storage medium type, the queue set of different medium types corresponds to different first priorities, so as to ensure that the critical business and data access operations can first utilize the most suitable storage resources, and solve the problem of low resource allocation efficiency. Each submission queue group corresponds to multiple namespaces, each namespace corresponds to different business types and service levels, and the priority of each namespace is determined according to the business type and the service level. Each submission queue group corresponds to a second priority, and the second priority of the submission queue group is determined according to the maximum priority in the multiple namespaces associated with the submission queue group, so as to ensure the priority processing of critical business. In the hierarchical arbitration of the queue set, the queue group and the queue, dynamic arbitration is adopted instead of a single fixed priority or a simple polling method. This multi-level queue arbitration method solves the problems of inaccurate priority setting, low resource allocation efficiency and insufficient fairness in the related arbitration mechanism. By refining the queue management unit, the accuracy of scheduling is improved, so that in the complex multi-tenant and mixed load scenarios, the quality of service (QoS) requirements of various queues can be more accurately met, and the "starvation" phenomenon of low-priority queues is avoided. In summary, the multi-level arbitration mechanism can more finely meet the QoS requirements of multi-tenant / multi-business through its flexible hierarchical division and independent scheduling strategy, avoids the rigidity of resource allocation, and improves the overall service quality and user experience.

[0064] In one example embodiment, before storing the multiple instructions sent by the host into the multiple submission queues, the above method further comprises:

[0065] According to the preset input / output operation rate, the length of the preset arbitration period is determined; the preset input / output operation rate is negatively related to the length of the preset arbitration period.

[0066] The preset input / output operation rate refers to a target input / output operation per second (IOPS) set by the NVMe storage device in design. In this embodiment, an adaptive method of preset arbitration period is proposed, which is used to dynamically adjust the length of the preset arbitration period according to the total number of submission queues, and adjust the number of queues for equal arbitration according to the number of CPUs. The adaptive method is based on grouping and parallel processing, compatible with different number of NVMe queue / grouping parameters, and realizes fast arbitration. The adaptive method adopts a distributed parallel processing strategy to realize flexible arbitration configuration of firmware, that is, the length of the preset arbitration period is adjusted according to the IOPS target preset by the system. The preset input / output operation rate is negatively related to the length of the preset arbitration period, which means that the higher the IOPS required by the system, the shorter the length of the arbitration period must be set in order to meet the higher input / output operation rate. This is because a shorter arbitration period can schedule queues more frequently, so that the device can respond and process I / O requests faster, and ultimately achieve higher IOPS performance. For example, if the system needs to support 4M IOPS, the arbitration period needs to be ≤ 1 / 4M second (that is, 250 nanoseconds) to ensure that 4 million I / O operations per second can be processed. Conversely, if the IOPS required by the system is lower, the length of the arbitration period can be appropriately extended, because a lower IOPS target allows more time for arbitration decisions, without worrying about affecting the overall response speed or processing capacity.

[0067] In some embodiments, the embodiment also provides a strategy of dynamically adjusting the arbitration period based on the frequency of the processor (such as CPU) of the storage device, that is, at the firmware implementation level, the length of the preset arbitration period can be finely adjusted according to the frequency of the processor (such as CPU) of the storage device actually running in the storage device. This means that the firmware will calculate the number of arbitrations that can be performed within each preset arbitration period according to the running frequency of the CPU, to ensure that the CPU completes at least one queue arbitration process within each arbitration period, both meeting the IOPS target and adapting to the specific performance of the CPU.

[0068] Through this embodiment, the length of the preset arbitration period is dynamically adjusted according to the preset input / output operation rate, and the length of the preset arbitration period is negatively related to the preset input / output operation rate. When the system needs to achieve a higher IOPS target, the arbitration period will be set shorter to ensure that I / O requests can be responded and processed quickly, improve scheduling efficiency, and solve the problem of insufficient high concurrency efficiency in related technologies.

[0069] In one exemplary embodiment, a submission queue in the plurality of submission queues corresponds to a configuration enabled state and a skip flag; the configuration enabled state represents the working state of the corresponding submission queue; and the skip flag is used to mark whether the corresponding submission queue participates in arbitration.

[0070] wherein the enabled state is a binary flag indicating whether a submission queue (SQ) in the NVMe storage device has been activated and is ready to receive and process I / O instructions sent from the host. If the enabled state of a queue is true (i.e., the queue is enabled), the queue will be considered as a candidate queue in the arbitration process and can participate in the comparison of priority and weight, and thus has a chance to be scheduled for execution. In other words, a queue configured with an enabled state indicates that the queue is created and normally activated by the kernel device driver. If the enabled state of a queue is false (i.e., the queue is disabled), the queue will not be considered as a candidate queue in the arbitration process and will be directly excluded from the scheduling of the current arbitration cycle.

[0071] The skip flag (SKIP) is another key identifier indicating whether a submission queue in the NVMe storage device should be excluded from the current arbitration cycle, i.e., whether the queue participates in the arbitration decision this time. If the skip flag of a queue is set to true (i.e., skip state), the queue will not be included in the candidate queue set of the current arbitration cycle regardless of its priority or weight settings, thus ensuring the accuracy and efficiency of the arbitration process.

[0072] In some embodiments, before selecting the current queue set from the plurality of queue sets, the above method further comprises:

[0073] Pruning the instruction submission queue from the plurality of submission queues that satisfies at least one of the following conditions: the corresponding configuration enabled state is a disabled state, the corresponding skip flag indicates not participating in arbitration, and there is no available handle in the corresponding output queue; wherein the output queue is used to store instructions in the instruction submission queue corresponding to the output queue.

[0074] wherein the configuration enabled state of the submission queue being a disabled state indicates that the enabled state of the queue is false (i.e., the queue is disabled), and the queue will not be considered as a candidate queue in the arbitration process and will be directly excluded from the scheduling of the current arbitration cycle. The skip flag of the submission queue indicating not participating in arbitration indicates that the skip flag of the queue is set to true (i.e., skip state), and the queue will not be included in the candidate queue set of the current arbitration cycle. In other words, the remaining queue should be a non-SKIP queue.

[0075] The output queue (OBQ) is a component in the NVMe storage device used to cache and transmit I / O instructions selected from the winning instruction submission queue (SQ) in the arbitration process for processing by the corresponding CPU core.

[0076] A handle is an identifier used to reference or access an object (such as a file, window, memory block, queue, etc.) within an operating system. Handles are typically assigned by the operating system and are opaque pointers or integers that applications use to manipulate or access the object without needing to concern themselves with the object's specific implementation or storage location. HandleCnt refers to the available contexts for storing commands obtained from the host. HandleCnt refers to the handles in the output queue (OBQ) of the NVMe storage device that are currently available for caching and processing new I / O instruction data. When the OBQ has available handles (i.e., HandleCnt > 0, indicating that there is at least one free context slot in the OBQ), it means that it has sufficient resources to receive instructions from the selected queue in the arbitration process, and thus can process these instructions in a timely manner, ensuring the efficiency of queue scheduling and the response speed of the system.

[0077] Through the embodiment, before selecting the current queue set from the plurality of queue sets, eliminating the queues in the corresponding OBQ that do not have available handles can avoid resource contention, ensure that the queue selected in the arbitration can immediately perform I / O operations, avoid invalid scheduling and resource waiting, and improve resource utilization; eliminating the queues in the disabled state can avoid scheduling the queues in failure or maintenance, which helps to prevent the spread of system failure and maintain stable operation of the system; eliminating the queues represented by the skip flag and not participating in arbitration can significantly reduce the size of the queue set participating in arbitration, thereby speeding up the arbitration process, reducing arbitration delay, and improving the overall response speed of the storage system.

[0078] In an example embodiment, the current queue set is selected from the plurality of queue sets according to the first priority corresponding to the queue set of the plurality of queue sets, and the current submission queue group is selected from the current queue set according to the second priority corresponding to the submission queue group of the plurality of submission queue groups, including:

[0079] The queue set with the highest first priority in the plurality of queue sets is determined as the current queue set, and the submission queue group with the highest second priority in the current queue set is determined as the current submission queue group.

[0080] In some embodiments, each submission queue can also correspond to a third priority, and in the process of screening the current submission queue from the current submission queue group, the submission queue with the highest third priority in the current submission queue group can be determined as the current submission queue.

[0081] Optionally, the queues are compared in the order of Set->Group->Queue level, and the queue with the highest priority is selected. Specifically, in the Set level, the first priority of the Set is compared (e.g., the smaller the value, the higher the priority), and the queue set with the highest first priority is selected as the current queue set; in the Group level, the second priority of the Group under the current queue set is compared (e.g., the smaller the value, the higher the priority), and the submission queue group with the highest second priority is selected as the current submission queue group; in the Queue level, the third priority of the submission queue under the current submission queue group is compared, and the submission queue with the highest third priority is selected as the current submission queue.

[0082] Through the embodiment, the queue set and the queue group with the highest priority are always selected, ensuring that the I / O request of a critical operation (such as system configuration update, fault recovery, etc.) can be responded and processed at the fastest speed, thereby significantly improving the reaction speed of the storage device to emergency events.

[0083] In an example embodiment, the above method further includes:

[0084] In the case where the plurality of queue sets include a plurality of queue sets with the highest first priority, the current queue set is selected from the plurality of queue sets with the highest first priority by fair polling; in the case where the current queue set includes a plurality of submission queue groups with the highest second priority, the current submission queue group is selected from the plurality of submission queue groups with the highest second priority by fair polling.

[0085] In the process of comparing in the order of Set->Group->Queue level, the scenario of the same priority may occur. In the scenario of the same priority, the queue, group or set (Set) with the same priority needs to be cyclically scheduled in a predetermined order. In this case, the fair polling can be used to select the final winning object.

[0086] Fair polling is a scheduling algorithm designed to ensure that resources (such as processor time, access opportunities of storage devices, etc.) are fairly distributed among multiple requesters (such as queue sets, queue groups). In the case where the plurality of queue sets or submission queue groups have the same highest priority, the fair polling selects the next scheduling object by sequential circulation (i.e., in a fixed order), and achieves fair scheduling by dynamic polling order, avoiding the problem of resource monopoly caused by fixed priority, and avoiding the problem of some queues being ignored or “starved” for a long time due to the fixed priority distribution mechanism.

[0087] The core of the fair round-robin is to maintain a round-robin pointer, which indicates the last scheduled queue set or group, and each time the pointer moves forward to point to the next queue set or group, ensuring that all sets or groups of the same priority have a chance to be scheduled in each round-robin cycle. For example, assume that there are the following queue configurations in an NVMe storage device: queue sets (Set): there are three Sets (S1, S2, S3) which have the same first priority (e.g. PRIORITY = 1). In the current queue set S1, there are four Groups (G1, G2, G3, G4) which have the same second priority (e.g. PRIORITY = 2). The round-robin pointer initially points to the queue set S1. In the arbitration cycle, when multiple queue sets have the highest first priority, the fair round-robin selects the next scheduled queue set in the order of S1→S2→S3. If S1 is selected to be scheduled in the current round-robin cycle, the pointer moves to S2, and in the next arbitration cycle, even if S1 has the highest priority again, S2 will be preferentially scheduled, and the round-robin pointer will not return to S1 until all Sets have been scheduled, and a new round-robin cycle begins.

[0088] In some embodiments, in addition to the above embodiments that select the queue set with the highest priority and submit the queue group, in the process of selecting the current queue set and the current submission queue group, the fair round-robin can also be used to select, that is, to select the current queue set from multiple queue sets and to select the current submission queue group from the current queue set, including: selecting the current queue set from multiple queue sets by fair round-robin; selecting the current submission queue group from the current queue set by fair round-robin.

[0089] The fair round-robin has been explained in the above embodiments and will not be repeated here. Using the fair round-robin to select the current queue set and the current submission queue group can ensure that each queue set and each queue group has a chance to be scheduled, avoiding any group being ignored for a long time, thereby achieving uniform allocation of resources and improving overall system performance.

[0090] Through this embodiment, fair round-robin is implemented between queue sets or queue groups of the same priority, which can ensure that resources are not monopolized by a particular set or group for a long time, but can be fairly shared among units of the same priority, avoiding the "starvation" phenomenon caused by uneven resource allocation, and simplifying the selection decision among multiple highest priority queue sets or groups, without the need for additional complex algorithms or calculations to determine which set or group is more priority, thereby reducing the complexity and delay of scheduling decisions, speeding up the scheduling process, and improving the overall response speed of the system.

[0091] In an example embodiment, when the number of submission queues exceeds the level of thousands (such as 1025 SQs), the related serial arbitration method significantly increases the delay, which is difficult to meet the arbitration cycle requirement above the preset processing rate (such as 4M / s). Therefore, to solve this problem, in the embodiment, all submission queues are evenly distributed according to the number of cores of the processor in the storage device to form multiple groups (i.e., multiple groups of submission queues), and the number of submission queues in each group is the same to realize parallel arbitration and processing. Assuming that the storage device has 1025 SQ queues and the processor of the storage device has 4 cores, each core corresponds to 256 SQ queues.

[0092] The multiple groups of submission queues correspond to the multiple cores of the processor in the storage device, that is, each group (i.e., each group of submission queues) is assigned to a designated core of the processor for arbitration and processing to improve the concurrent processing capability. It can be understood that in multiple preset arbitration cycles, the multiple cores in the storage device independently arbitrate their corresponding queue sets, queue groups, and queues to realize parallel arbitration processing, significantly shorten the arbitration cycle, improve the system response speed and concurrent processing capability, and solve the efficiency bottleneck faced by the serial arbitration method in the related art in a high-concurrency scenario.

[0093] The processor in the storage device usually includes multiple cores. The core of the processor is a part of the internal processing unit of the storage device, and each core can independently execute instructions. In the embodiment, the multiple cores process the queues they are responsible for in parallel to enhance the overall performance of the device. Each core can process instructions from multiple corresponding submission queues (SQs) in parallel, which helps to improve the concurrent processing capability and overall performance of the storage device. By assigning different SQs to different cores or having each core manage a group of SQs, the waiting time for instruction processing can be effectively reduced, resource utilization can be optimized, and the response speed of I / O operations can be accelerated, especially in a data center environment that processes a large number of concurrent requests.

[0094] In some embodiments, the instructions stored in the current submission queue in the current submission queue group are processed by the processor of the storage device, including:

[0095] The current submission queue is sequentially selected from the current submission queue group by fair polling, and the instructions stored in the selected current submission queue are stored in the output queue group. The instructions stored in the output queue group are processed in parallel by the designated core corresponding to the current submission queue. The output queue group corresponds to the designated core. The output queue group includes multiple output queues. The multiple output queues are used to store the instructions stored in the selected current submission queue.

[0096] The current submission queue can be selected by a priority selection manner or a fair round robin manner. In this embodiment, the current submission queue is selected by the fair round robin manner. As known from the above embodiment, the current queue set and the current submission queue group can be selected by the priority selection manner, and the current submission queue can be selected by the fair round robin manner.

[0097] The output queue group refers to a queue set inside the NVMe storage device for further processing and sending the I / O instruction to the physical storage medium. The output queue group includes a plurality of output queues, and each output queue is designed to store the instruction dispatched from the current submission queue. In this embodiment, the output queue group is designed to correspond to a specified core, so that each core independently processes the instruction in its output queue, improving the parallel processing efficiency. It can be understood that, in order to realize high concurrency processing, each output queue group is allocated to an independent core, avoiding the competition of resources among cores and accelerating the instruction processing speed.

[0098] Through this embodiment, by allocating the instruction to a plurality of output queues and processing them in parallel by the specified CPU core, the parallel processing capability of the system is greatly enhanced, the waiting time for processing the instruction is reduced, and the overall I / O performance and response speed are improved. The correspondence between the output queue group and the specified core ensures that each core is responsible for processing a group of output queues, which not only reduces the resource competition among cores, but also improves the efficiency of resource allocation, so that each core can efficiently utilize its processing capability.

[0099] In an example embodiment, the fair round robin manner includes the following steps:

[0100] I. In a non-first dispatching process, determining a first offset of an object identifier of a current to-be-compared object corresponding to a current dispatching process relative to an object identifier of a to-be-compared object pointed to by a polling pointer, and determining a second offset of an object identifier of a current temporary winning object corresponding to the current dispatching process relative to the object identifier of the to-be-compared object pointed to by the polling pointer; the current to-be-compared object is one of: a queue set, a submission queue group, and a submission queue; the polling pointer is used to point to a to-be-compared object polled in a last dispatching process of the current dispatching process; the current temporary winning object refers to a target object selected in the last dispatching process;

[0101] II. In a case where the first offset is less than the second offset, determining that a polling order of the current to-be-compared object is prior to that of the current temporary winning object, setting the current temporary winning object as the current to-be-compared object, setting the polling pointer to point to the current to-be-compared object, and determining the current to-be-compared object as a target object selected in the current dispatching process; the target object includes one of: the current queue set, the current submission queue group, and the selected current submission queue.

[0102] The scheduling process refers to a process of selecting a target object for instruction processing from a queue set, a commit queue group, or a commit queue. The scheduling process of each cycle is independent, but continuity and fairness are maintained through a polling pointer and offset mechanism. The target object is an object selected through a fair polling process and can be a current queue set, a current commit queue group, or a selected current commit queue. It is the direct output of the scheduling process.

[0103] The current to-be-compared object refers to an object whose scheduling priority is being evaluated by the system in the scheduling cycle. It can be a queue set, a commit queue group, or a commit queue, and varies according to the level of the scheduling process. The object identifier of the to-be-compared object is an identifier for uniquely identifying each to-be-compared object, such as the ID of a queue set, a commit queue group, or a commit queue.

[0104] The polling pointer is used to record the winning object in the last scheduling process during the fair polling process, serving as the starting point of the current scheduling cycle to achieve cyclic scheduling and fairness guarantee.

[0105] The current temporary winning object refers to the target object selected in the last scheduling process of the current scheduling process in a non-first scheduling process. It is used for offset comparison with the current to-be-compared object to determine the scheduling order.

[0106] The first offset refers to the offset of the object identifier of the current to-be-compared object relative to the object identifier pointed to by the polling pointer in the last scheduling process in a non-first scheduling process. It is used to determine the scheduling order. Specifically, the first offset is determined by calculating the numerical difference between the object identifier of the current to-be-compared object and the object identifier pointed to by the polling pointer, and then taking the modulus of the total number of objects. The first offset is used to evaluate the order of the current to-be-compared object in the polling sequence. When the first offset is smaller, it means that the current to-be-compared object is closer to the object pointed to by the polling pointer after the last scheduling, and thus has a higher scheduling priority in the current scheduling.

[0107] Similarly, the second offset refers to the offset of the object identifier of the current temporary winning object relative to the object identifier pointed to by the polling pointer in the last scheduling process in a non-first scheduling process. It is used for comparison of the scheduling order. Like the first offset, the second offset is also determined by the numerical difference between the object identifiers and the modulus operation. It reflects the position information of the temporary winning object in the polling sequence. By comparing the first offset and the second offset, the system can determine whether the current to-be-compared object should be selected as the target object of the current scheduling process before the temporary winning object.

[0108] When the first offset is smaller than the second offset, it indicates that the current to-be-compared object is located more backward in the polling sequence relative to the polling pointer, i.e., it has waited for a longer time after the last scheduling. Based on the principle of fairness, the system will preferentially schedule the object that has waited for a longer time, so as to set the current temporary winning object as the current to-be-compared object, update the polling pointer to point to the current to-be-compared object, and determine the current to-be-compared object as the target object of the current scheduling process, so as to ensure that all queues or queue groups are fairly treated in scheduling, and to avoid the problems of long-term imbalance of resource allocation and "starvation" of a queue.

[0109] In an example embodiment, the above method further comprises:

[0110] In the first scheduling process, the polling pointer is set to be invalid, the object with a smaller object identifier is set as the current temporary winning object from among the current temporary winning object and the current to-be-compared object, and the polling pointer is pointed to the current to-be-compared object.

[0111] In this embodiment, the storage device initializes the scheduling process by simple numerical value comparison when starting to schedule a queue, selects the object with the smallest identifier as the winner of the first scheduling, and updates the polling pointer to this object, thereby providing a clear starting point for the subsequent scheduling period and ensuring the continuity and fairness of the scheduling process. This mechanism avoids the uncertainty that may occur in the first scheduling and lays a foundation for the subsequent fair polling strategy based on the offset.

[0112] In some embodiments, the fair polling method can adopt the RR_Select() polling scheduling mechanism, wherein the input parameters of the RR_Select() polling scheduling mechanism include the following contents:

[0113] total: total number of to-be-compared objects (such as total number of queues, total number of Groups, total number of Sets);

[0114] prev_id: ID of the to-be-compared object that was polled in the last scheduling process of the current scheduling process (polling pointer, records the historical scheduling position);

[0115] curr_id: ID of the current to-be-compared object;

[0116] winner_id: ID of the current temporary winning object.

[0117] The output parameters of the RR_Select() polling scheduling mechanism include the following contents:

[0118] TRUE: curr_id should be given priority over winner_id in the polling order;

[0119] FALSE: curr_id should not be prioritized over winner_id.

[0120] The pseudo-code implementation logic of the RR_Select() round-robin scheduling mechanism is as follows:

[0121] def RR_Select(total, prev_id, curr_id, winner_id):

[0122] # If prev_id is invalid (e.g., first scheduling), directly compare the numerical values

[0123] if prev_id >= total:

[0124] return curr_id < winner_id

[0125] # Calculate the order of curr_id and winner_id relative to prev_id

[0126] curr_offset = (curr_id - prev_id) % total

[0127] winner_offset = (winner_id - prev_id) % total

[0128] # If the offset of curr_id is smaller, it means it is earlier in the round-robin order

[0129] return curr_offset < winner_offset

[0130] wherein curr_offset represents the first offset; winner_offset represents the second offset.

[0131] For example, assume that the objects to be compared are Queues, and the NVMe storage device includes three Queues, Queue1, Queue2 and Queue3. In the first dispatch process, the polling pointer is initialized to be invalid, i.e., the polling pointer prev_id is set to an invalid value, such as -1. Assume that the system starts the first dispatch, and the objects of Queue1 and Queue2 are compared, and the object identifiers of Queue1 and Queue2 are 1 and 2 respectively, the winner_id (the current temporary winning object) will be set to the object with the smaller ID, thus winner_id = 1, and Queue2 becomes the current object to be compared, thus curr_id (the ID of the current object to be compared) is set to 2, i.e., curr_id = 2. Since it is the first dispatch, the polling pointer prev_id is updated to the value of curr_id, i.e., prev_id = 2. The RR_Select() algorithm outputs FALSE. In the next dispatch process, Queue3 becomes the current object to be compared, i.e., curr_id = 3. According to the position of the current object to be compared Queue3 (ID is 3) relative to the polling pointer prev_id (current value is 2), (3-2)%N = 1 is calculated (here N is the total number of objects, and is assumed to be 3, because only three queues are considered), wherein (3-2)%3 = 1 indicates that Queue3 is directly after Queue2. Therefore, the first offset curr_offset is 1. According to the position of the current temporary winning object Queue1 (ID is 1) relative to the polling pointer prev_id, (1-2)%N = 2 is calculated, wherein the result of (1-2) is -1, and -1%3 needs to be calculated by converting understanding, -1 can be understood as the offset from Queue2 to Queue1, and (1-2)%3 = 2 indicates that Queue1 has an offset of 2 positions relative to Queue2 in the circular dispatch, i.e., Queue3 is passed back to Queue1. Therefore, winner_offset is 2. Since curr_offset is less than winner_offset, the output of the RR_Select() algorithm is TRUE. At this time, Queue3 (curr_id is 3) becomes the new winning object, and winner_id is set to 3, and prev_id is updated to the ID of Queue3, i.e., 3. Based on the judgment result of RR_Select(), Queue3 becomes the target object of the current dispatch process, which means that the I / O instructions in it will be preferentially transmitted to the output queue group associated therewith, and then processed by the corresponding core.

[0132] By the embodiment, the polling pointer is initialized as invalid at the first time of scheduling, and the initial queue scheduling order is determined based on the object identification size, the first current temporary winning object is determined through simple object identification comparison, the initial start efficiency of the arbitration mechanism is improved, and the initialization configuration is simplified; in the non-first time of scheduling, the concepts of the first offset and the second offset are introduced to determine the scheduling priority, the fairness of the queue in scheduling is ensured, and even in a high concurrency scenario, the situation that a low priority queue is ignored for a long time or starved can be effectively avoided.

[0133] In an example embodiment, the current to-be-compared object corresponds to a preset weight. The preset weight refers to an initial weight value allocated to the current to-be-compared object, which is used for queue scheduling decision in the multi-level priority arbitration mechanism. The higher the preset weight of the to-be-compared object, the higher the probability of being selected in scheduling.

[0134] In some embodiments, after determining the current to-be-compared object as the target object selected in the current scheduling process, the above method further includes:

[0135] According to a fixed step, the latest weight of the current to-be-compared object is reduced to obtain an updated latest weight. The latest weight refers to the weight value of the current to-be-compared object updated according to a fixed step after being determined as the target object in the historical scheduling process. In the case that the updated latest weight is 0, the latest weight of the current to-be-compared object is reset to the preset weight, and the polling pointer is updated to point to the current to-be-compared object.

[0136] It can be understood that the latest weight refers to the updated value of the weight value of the current to-be-compared object after winning in the last arbitration after winning. The latest weight reflects the dynamic priority of the queue and the resource occupation in the last scheduling period. Each time the current to-be-compared object is selected as the winning object, the preset weight of the current to-be-compared object is reduced according to a preset fixed step to obtain an updated latest weight, thereby realizing dynamic weight adjustment and maintaining the fairness of scheduling.

[0137] In the process of updating the weight of the current to-be-compared object, the weight counter (WCNT) is used for counting and updating. Specifically, the weight counter of the current to-be-compared object is initialized, that is, WCNT=WEIGHT (i.e., the preset weight), and after the current to-be-compared object is selected each time, the counting value of the weight counter (i.e., the latest weight of the current to-be-compared object) is reduced by a fixed step to obtain the updated latest weight. When the updated latest weight of the current to-be-compared object decreases to 0, that is, WCNT=0, it indicates that the resource allocation of the current to-be-compared object has reached its minimum limit. At this time, by resetting the latest weight of the current to-be-compared object to the preset weight, it is ensured that the current to-be-compared object can continue to participate in scheduling, restores its scheduling opportunity, and avoids permanent resource starvation. When the updated latest weight is 0, the polling pointer is updated to point to the current to-be-compared object, ensuring that the next scheduling starts from it, and realizing fair round-robin scheduling.

[0138] As can be seen from the above embodiment, the current to-be-compared object can be any of the queue set (Set), the submission queue group (Group), and the submission queue (Queue). Therefore, in this embodiment, the polling pointer can be the polling pointer of the queue set, that is, s_winner_SPREV, or the polling pointer of the submission queue group, that is, g_winner_GPREV, or the polling pointer of the submission queue, that is, q_winner_QPREV, and the type of the polling pointer depends on the type of the current to-be-compared object. For example, the global variables corresponding to the queue set (Set), the submission queue group (Group), and the submission queue (Queue) are initialized respectively; wherein the global variables corresponding to the queue set (Set) include s_winner (the current polled Set value) and s_winner_SPREV; the global variables corresponding to the submission queue group (Group) include g_winner (the current polled Group) and g_winner_GPREV; and the global variables corresponding to the submission queue (Queue) include q_winner (the current polled queue value) and q_winner_QPREV. For the submission queue group (Group), the polling pointer thereof is g_winner_GPREV; if the weight counter (WCNT) of the Group decreases to 0, it is reset to the preset weight WEIGHT of the Group, and the polling pointer g_winner_GPREV of the Group is updated to point to the current to-be-compared object. Similarly, when the weight counter (WCNT) of the Set decreases to 0, the polling pointer s_winner_SPREV of the Set is reset and updated.

[0139] For example, there are three submission queues on the NVMe storage device, Queue1, Queue2, Queue3, whose IDs are 1, 2, 3 respectively. Assuming that in a series of dispatches, the preset order of the winning object is as follows:

[0140] The first dispatch: Queue1 becomes the winning object, and the latest weights of Queue1, Queue2 and Queue3 are all preset weights WEIGHT.

[0141] The second dispatch: Queue2 becomes the winning object, the latest weight of Queue1 is reduced by a fixed step STEP, and the updated latest weight is WEIGHT-STEP, and the latest weights of Queue2 and Queue3 are both preset weights WEIGHT.

[0142] The third dispatch: Queue3 becomes the winning object, the latest weight of Queue2 is reduced, and the updated latest weight is WEIGHT-STEP, and the latest weight of Queue3 is WEIGHT because it wins for the first time.

[0143] The fourth dispatch: Queue1 becomes the winning object, the latest weight of Queue3 is reduced, and the updated latest weight is WEIGHT-STEP, and the latest weight of Queue1 is reduced again, and the updated latest weight is WEIGHT-2STEP.

[0144] Assuming that in the process of N dispatches, Queue1 is the winning object, and the latest weight of Queue1 has been reduced to 0 due to multiple dispatches. At this time, the latest weight of Queue1 is reset to the preset weight WEIGHT. In subsequent dispatches, Queue1 restores the same competitive position as Queue2 and Queue3.

[0145] Figure 4 is an architecture diagram of a queue grouping architecture according to an embodiment of the present application, as Figure 4As shown, the multiple submission queues (submission queue 1-N) are divided into a three-layer structure of Set→Group→Queue, wherein the current queue set is selected using the WRR (SP) algorithm in the Set layer, wherein the WRR (SP) algorithm refers to a specific processing weighted round robin algorithm (or single point weighted round robin algorithm), which is an algorithm for weighted scheduling of multiple queue sets based on the weighted round robin algorithm, which prioritizes certain specific queue sets according to pre-set weights to ensure that high-weight queues or tasks can obtain more processing opportunities. In the case of the same priority, the RR (Set) algorithm is used to poll and schedule the multiple queue sets to ensure that the queues in each set have fair processing opportunities. The WRR (SP) algorithm is used to process the multiple submission queue groups in the current queue set to obtain the current submission queue group. In the case of the same priority, the RR (Set) algorithm is used to poll and schedule the multiple submission queue groups in the current queue set to obtain the current submission queue group. The RR (Set) algorithm is used to poll and process the submission queues in the current submission queue group to obtain the current submission queue, and the instructions in the current submission queue are stored in the output queue group for processing by the processor.

[0146] Optionally, the embodiment performs arbitration processing of multiple NVMe submission queues by setting an initialization module, a candidate screening module, a multi-level arbitration module, and a winning processing state updating module, implements multi-level priority nested scheduling (Set→Group→Queue), meets multi-tenant QoS, ensures the fairness of queue processing under different weights and the same priority, and uses sequential circular scheduling queue polling to avoid the problem of host IO timeout caused by starvation of a certain queue.

[0147] Through the embodiment, the preset weight of the current comparison object is dynamically adjusted and combined with polling, the higher the preset weight, the more arbitration cycles are obtained, while ensuring fairness under the same priority, and sequential circular scheduling queue polling is used to avoid starvation of a certain queue to ensure scheduling fairness.

[0148] Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be realized by means of software and a necessary general hardware platform, of course, it can also be realized by hardware, but in many cases the former is a better embodiment.

[0149] The embodiment of the present application also provides a submission queue arbitration device of a storage device, Figure 5 is a structural block diagram of submission queue arbitration of a storage device according to an embodiment of the present application, as Figure 5 shown, the device comprises:

[0150] The queue storage module 502 is configured to store a plurality of instructions sent by a host into a plurality of submission queues; the plurality of submission queues are divided into a plurality of submission queue groups in advance; the plurality of submission queue groups are divided into a plurality of queue sets in advance; a queue set in the plurality of queue sets corresponds to a first priority; the queue sets in the plurality of queue sets are divided according to storage medium types; different storage medium types correspond to different priorities; the first priority corresponding to a queue set in the plurality of queue sets is determined according to the priority corresponding to the storage medium type of the queue set; a submission queue group in the plurality of submission queue groups corresponds to a second priority; a submission queue group in the plurality of submission queue groups corresponds to a plurality of namespaces; the plurality of namespaces correspond to different priorities; a namespace in the plurality of namespaces corresponds to different service types and service levels; the priority corresponding to a namespace in the plurality of namespaces is determined according to the service type and the service level; the second priority corresponding to a submission queue group in the plurality of submission queue groups is the maximum priority in the plurality of namespaces corresponding to the submission queue group in the plurality of submission queue groups.

[0151] The queue arbitration module 504 is configured to perform the following arbitration processes in a plurality of preset arbitration periods: selecting a current queue set from the plurality of queue sets according to the first priority corresponding to the queue set; selecting a current submission queue group from the current queue set according to the second priority corresponding to the submission queue group; and processing the instructions stored in the current submission queue in the current submission queue group by a processor of a storage device.

[0152] Through the above device, the instructions sent by the host are stored in the plurality of submission queues, and the plurality of submission queue groups are divided into a plurality of submission queue groups in advance; the plurality of submission queue groups are divided into a plurality of queue sets in advance; in the hierarchical arbitration of the queue sets, the queue groups, and the queues, a dynamic arbitration is adopted instead of a single fixed priority or a simple polling mode; the multi-level queue arbitration mode solves the problems of inaccurate priority setting, low resource allocation efficiency, and insufficient fairness in the related arbitration mechanism; through the refined queue management unit, the accuracy of scheduling is improved, so that in a complex multi-tenant and mixed load scenario, the quality of service (QoS) requirements of various queues can be more accurately met, and the "starvation" phenomenon of low-priority queues is avoided. In summary, the multi-level arbitration mechanism can more finely meet the QoS requirements of multi-tenants and multi-services through its flexible hierarchical division and independent scheduling strategy, avoids the rigidity of resource allocation, and improves the overall service quality and user experience.

[0153] Optionally, the queue arbitration module 504 is further configured to determine the length of the preset arbitration period according to a preset input / output operation rate; the preset input / output operation rate is negatively related to the length of the preset arbitration period.

[0154] Optionally, the submission queue in the plurality of submission queues corresponds to a configuration enable state and a skip flag; the configuration enable state represents a working state of the corresponding submission queue; the skip flag is used to mark whether the corresponding submission queue participates in arbitration; before the current queue set is selected from the plurality of queue sets, the queue arbitration module 504 is further configured to eliminate an instruction submission queue in the plurality of submission queues that meets at least one of the following conditions: the corresponding configuration enable state is a disabled state, the corresponding skip flag represents not participating in arbitration, and there is no available handle in the corresponding output queue; wherein the output queue is used to store instructions in the instruction submission queue corresponding to the output queue.

[0155] Optionally, the queue arbitration module 504 is further configured to determine, as the current queue set, a queue set with the highest first priority in the plurality of queue sets; and determine, as the current submission queue group, a submission queue group with the highest second priority in the current queue set.

[0156] Optionally, the queue arbitration module 504 is further configured to, in a case where the plurality of queue sets include a plurality of queue sets with the highest first priority, select the current queue set from the plurality of queue sets with the highest first priority in a fair polling manner; and in a case where the current queue set includes a plurality of submission queue groups with the highest second priority, select the current submission queue group from the plurality of submission queue groups with the highest second priority in a fair polling manner.

[0157] Optionally, the processor includes a plurality of cores; the plurality of submission queues are pre-divided into a plurality of groups of submission queues according to the number of cores of the processor, and the plurality of groups of submission queues correspond one-to-one to the plurality of cores of the processor; the queue arbitration module 504 is further configured to select the current submission queue from the current submission queue group in a fair polling manner, store the instructions stored in the selected current submission queue into an output queue group, and process the instructions stored in the output queue group in parallel through a designated core corresponding to the current submission queue; the output queue group corresponds to the designated core; the output queue group includes a plurality of output queues; and the plurality of output queues are used to store the instructions stored in the selected current submission queue.

[0158] Optionally, the queue arbitration module 504 is further configured to, in the non-first scheduling process, determine a first offset of an object identifier of a current to-be-compared object corresponding to a current scheduling process relative to an object identifier of a to-be-compared object pointed to by a polling pointer, and determine a second offset of an object identifier of a current temporary winning object corresponding to the current scheduling process relative to the object identifier of the to-be-compared object pointed to by the polling pointer; the current to-be-compared object is one of the following: a queue set, a submission queue group, and a submission queue; the polling pointer is used to point to a to-be-compared object polled in a last scheduling process of the current scheduling process; the current temporary winning object refers to a target object selected in the last scheduling process; in a case where the first offset is less than the second offset, it is determined that a polling order of the current to-be-compared object is prior to that of the current temporary winning object, the current temporary winning object is set as the current to-be-compared object, the polling pointer is set to point to the current to-be-compared object, and the current to-be-compared object is determined as the target object selected in the current scheduling process; the target object includes one of the following: a current queue set, a current submission queue group, and a selected current submission queue.

[0159] Optionally, the queue arbitration module 504 is further configured to, in the first scheduling process, set the polling pointer to be invalid, set, as the current temporary winning object, an object with a smaller object identifier from among the current temporary winning object and the current to-be-compared object, and set the polling pointer to point to the current to-be-compared object.

[0160] Optionally, the current to-be-compared object corresponds to a preset weight; after the current to-be-compared object is determined as the target object selected in the current scheduling process, the queue arbitration module 504 is further configured to reduce a latest weight of the current to-be-compared object by a fixed step to obtain an updated latest weight; the latest weight refers to a weight value of the current to-be-compared object updated by the fixed step after the current to-be-compared object is determined as the target object in a historical scheduling process; in a case where the updated latest weight is 0, the latest weight of the current to-be-compared object is reset to the preset weight, and the polling pointer is updated to point to the current to-be-compared object.

[0161] The description of the features in the embodiment of the submission queue arbitration device of the storage device can be referred to the related description of the embodiment of the submission queue arbitration method of the storage device, which will not be repeated here.

[0162] The embodiment of the present application also provides an electronic device, including a memory and a processor, the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned embodiments of the submission queue arbitration method of the storage device.

[0163] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, wherein the computer program is configured to execute the steps in any of the above-mentioned embodiments of the submission queue arbitration method of the storage device when running.

[0164] In an example embodiment, the computer readable storage medium described above can include, but is not limited to, a U disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0165] Embodiments of the present application also provide a computer program product including a computer program, which, when executed by a processor, implements the steps in any of the above-mentioned storage device submission queue arbitration method embodiments.

[0166] Embodiments of the present application also provide another computer program product including a non-volatile computer readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the steps in any of the above-mentioned storage device submission queue arbitration method embodiments.

[0167] According to yet another aspect of the embodiments of the present application, a computer program product is provided, which includes computer programs / instructions containing program codes for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication part 609, and / or installed from the detachable medium 611. When the computer program is executed by the central processing unit 601, various functions provided by the embodiments of the present application are performed. The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0168] Figure 6 The computer system structure block diagram of the electronic device for implementing the embodiments of the present application is schematically shown. As shown in the figure, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to the programs stored in the ROM 602 or the programs loaded from the storage part 608 to the RAM 603. In the random access memory 603, various programs and data required for system operation are also stored. The central processing unit 601, the read-only memory 602, and the random access memory 603 are connected to each other through the bus 604. The input / output (I / O) interface 605 is also connected to the bus 604. Figure 6

[0169] ​The following components are connected to the I / O interface 605: an input part 606 including a keyboard, a mouse, etc.; an output part 607 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc.; a storage part 608 including a hard disk, etc.; and a communication part 609 including a network interface card such as a LAN card, a modem, etc. The communication part 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output interface 605 as necessary. A removable media 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 610 as necessary, so that a computer program read out therefrom is installed in the storage part 608 as necessary.

[0170] In particular, according to embodiments of the present application, the processes described in the various method flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program code for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication part 609, and / or installed from the removable media 611. When the computer program is executed by the central processing unit 601, various functions defined in the system of the present application are performed.

[0171] It should be noted that, Figure 6 The computer system 600 of the electronic device shown is merely an example and should not impose any limitation on the functions and usage range of embodiments of the present application.

[0172] Those skilled in the art will further appreciate that the individual steps of the examples described in connection with the embodiments disclosed herein can be realized by electronic hardware, computer software, or a combination of both. The various illustrative components, blocks, steps and the like described herein can be implemented or carried out by a combination of hardware and software, either on a general purpose computer system or on a specialized computer system. The particular methods of implementation are not limiting to the scope of the present application. The illustrative components, blocks, steps and the like are not limited to any particular order or sequence. Unless otherwise specified, the use of the ordinal adjectives (e.g., first, second, etc.) are to

[0173] The above describes in detail the method for submitting queue arbitration of the storage device and the electronic device provided by the present application. The principles and implementation manners of the present application are described by using specific examples, and the above description of the embodiments is only applicable to help understand the method of the present application and the core idea thereof. It should be pointed out that, for those skilled in the art, some improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A method for arbitrating a submission queue of a storage device, characterized in that, include: Store multiple instructions sent by the host into multiple submission queues; The multiple submission queues are pre-divided into multiple submission queue groups; The multiple submission queue groups are pre-divided into multiple queue sets; each queue set corresponds to a first priority; the queue sets are divided according to storage medium type; different storage medium types correspond to different priorities; the first priority of each queue set is determined based on the priority corresponding to the storage medium type of the queue set; each submission queue group corresponds to a second priority; each submission queue group corresponds to multiple namespaces; each namespace corresponds to a different priority; each namespace corresponds to a different business type and service level; the priority of each namespace is determined based on the business type and service level; the second priority of each submission queue group is the highest priority among the multiple namespaces corresponding to the submission queue group. The following arbitration process is executed in multiple preset arbitration cycles: the current queue set is selected from the multiple queue sets according to the first priority corresponding to the queue set in the multiple queue sets; the current submission queue group is selected from the current queue set according to the second priority corresponding to the submission queue group in the multiple submission queue groups; and the instructions stored in the current submission queue in the current submission queue group are processed by the processor of the storage device.

2. The method according to claim 1, characterized in that, Before storing the multiple instructions sent by the host into multiple submission queues, the method further includes: The length of the preset arbitration period is determined based on the preset input / output operation rate; the preset input / output operation rate is negatively correlated with the length of the preset arbitration period.

3. The method according to claim 1, characterized in that, The multiple submission queues have corresponding configuration enabled status and skip flags; the configuration enabled status represents the working status of the corresponding submission queue. The skip flag is used to indicate whether the corresponding submission queue participates in arbitration; Before selecting the current queue set from the plurality of queue sets, the method further includes: Remove instruction submission queues from the plurality of submission queues that meet at least one of the following conditions: the corresponding configuration enable state is disabled, the corresponding skip flag indicates that it does not participate in arbitration, and the corresponding output queue does not have a usable handle; wherein, the output queue is used to store instructions in the instruction submission queue corresponding to the output queue.

4. The method according to claim 1, characterized in that, The step of selecting the current queue set from the plurality of queue sets according to the first priority corresponding to the queue set, and selecting the current submission queue group from the current queue set according to the second priority corresponding to the submission queue group from the plurality of submission queue groups, includes: The set of queues with the highest first priority among the multiple queue sets is determined as the current queue set; the submission queue group with the highest second priority among the current queue set is determined as the current submission queue group.

5. The method according to claim 4, characterized in that, The method further includes: When the multiple queue sets include multiple queue sets with the highest first priority, the current queue set is selected from the multiple queue sets with the highest first priority through a fair round-robin method. If the current queue set includes multiple submission queue groups with the highest second priority, the current submission queue group is selected from these multiple submission queue groups with the highest second priority through fair round-robin.

6. The method according to claim 1, characterized in that, The processor includes multiple cores; the multiple commit queues are pre-divided into multiple groups according to the number of cores of the processor, and the multiple groups of commit queues correspond one-to-one with the multiple cores of the processor; the processing of instructions stored in the current commit queue in the current commit queue group by the processor of the storage device includes: The current commit queue is selected sequentially from the current commit queue group using a fair round-robin method. The instructions stored in the selected current commit queue are stored in the output queue group. The instructions stored in the output queue group are processed in parallel by a designated core corresponding to the current commit queue. The output queue group corresponds to the designated core. The output queue group includes multiple output queues. The multiple output queues are used to store the instructions stored in the selected current commit queue.

7. The method according to claim 5 or 6, characterized in that, The fair polling method includes the following steps: During non-initial scheduling processes, a first offset of the object identifier of the current object to be compared relative to the object identifier of the object to be compared pointed to by the polling pointer is determined, and a second offset of the object identifier of the current temporarily stored winning object relative to the object identifier of the object to be compared pointed to by the polling pointer is determined; the current object to be compared is one of the following: a queue set, a submission queue group, or a submission queue; the polling pointer is used to point to the object to be compared that was polled in the previous scheduling process of the current scheduling process; the current temporarily stored winning object refers to the target object selected in the previous scheduling process; If the first offset is less than the second offset, the polling order of the current object to be compared is determined to be higher than that of the current temporary winning object. The current temporary winning object is set as the current object to be compared, the polling pointer is set to point to the current object to be compared, and the current object to be compared is determined as the target object selected by the current scheduling process. The target object includes one of the following: the current queue set, the current submission queue group, and the selected current submission queue.

8. The method according to claim 7, characterized in that, The method further includes: During the initial scheduling process, the polling pointer is set to invalid, and the object with the smaller object identifier between the currently temporary winning object and the currently comparable object is set as the currently temporary winning object. The polling pointer is then pointed to the currently comparable object.

9. The method according to claim 7, characterized in that, The current object to be compared corresponds to a preset weight; After determining the current object to be compared as the target object selected by the current scheduling process, the method further includes: The latest weight of the current object to be compared is reduced by a fixed step size to obtain the updated latest weight; the latest weight refers to the weight value of the current object to be compared after it was determined to be the target object in the historical scheduling process and updated by the fixed step size. If the latest weight after the update is 0, reset the latest weight of the current object to be compared to the preset weight, and update the polling pointer to point to the current object to be compared.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the method as claimed in any one of claims 1 to 9.

Images