Implementation Method, Device, Medium and Product of a Programmable I / O Scheduler

Abstract

The present invention discloses an implementation method, device, medium and product of a programmable I / O scheduler. The method of the present invention includes creating a placeholder I / O scheduler to serve as a shadow implementation of the programmable I / O scheduler; defining and implementing the struct bpf_struct_ops operation object of the placeholder I / O scheduler and registering it into the eBPF subsystem of the kernel to expose the hook points of the placeholder I / O scheduler to users outside the kernel, so that the I / O scheduler written by users outside the kernel using eBPF programs can be dynamically loaded into the kernel to replace the placeholder I / O scheduler. The present invention aims to implement a programmable I / O scheduler to meet the I / O capacity requirements of specific user scenarios, customize an I / O scheduling strategy that conforms to the characteristics of its underlying hardware and upper-layer services online, enable users to customize and expand the I / O scheduler online, flexibly and conveniently, and greatly save the required development costs and cycles.

Description

Technical Field

[0001] The present invention relates to the field of operating systems, and particularly to a method, apparatus, medium and product for implementing a programmable I / O scheduler. Background Art

[0002] The I / O scheduler is crucial for the Linux operating system and storage performance. Especially in a multi-tasking and complex storage environment, an efficient I / O scheduler can greatly improve the efficiency of the storage system and optimize the service level of system services by adjusting the scheduling policy and optimizing disk utilization. On the one hand, with the rapid development of wafer technology and integrated circuit technology, the underlying storage components are being rapidly iterated. On the other hand, with the rise of new technologies such as cloud computing and AI, the complexity and variability of upper-layer business loads are becoming increasingly evident. The combination of these two aspects poses new requirements for the storage efficiency of the existing Linux system, especially for the I / O scheduler. For example, the current I / O scheduler is difficult to maximize the capabilities of emerging diverse storage devices, such as solid-state drives (SSDs) and non-volatile memory (NVMe) devices; another example is that the current I / O scheduler design mostly faces general scenarios and lacks consideration for specific scenarios such as hybrid deployment scenarios and low-latency scenarios. If there is a method or system that can allow users to conveniently, quickly and securely customize an I / O scheduling policy that conforms to the characteristics of their underlying hardware and upper-layer business online, it will be a great improvement for both the storage efficiency of the Linux system and the user's business experience.

[0003] Current I / O schedulers in the Linux general block layer, such as Deadline, BFQ (Budget Fair Queuing), etc., are all designed for general scenarios and can meet the I / O requirements of most scenarios. However, if you want to maximize I / O efficiency or perform storage QOS (Quality of Service) for specific scenarios, you need to customize the I / O scheduler separately; the process generally includes the following steps: 1) Obtain the Linux kernel source code and build a kernel module development environment; 2) Develop a specific I / O scheduler; 3) Compile & debug the newly developed I / O scheduler; 4) Stop the current business system and replace it with the new I / O scheduler. From the above process, it can be seen that whether developing or testing a new I / O scheduler to meet business requirements requires downtime and the process is very cumbersome and challenging. Moreover, the above process may not succeed the first time, and often requires repeated downtime, development and debugging to meet the requirements. Summary of the Invention

[0004] Technical problems to be solved by the present invention: In view of the above problems of the prior art, the present invention provides a method, device, medium and product for implementing a programmable I / O scheduler. The present invention aims to implement a programmable I / O scheduler to meet the I / O capacity requirements of user-specific scenarios, online customize I / O scheduling policies that conform to the characteristics of the underlying hardware and upper-layer services, enabling users to customize and expand the I / O scheduler online, flexibly and conveniently, and greatly saving the required development costs and cycles.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0006] A method for implementing a programmable I / O scheduler, comprising the following steps:

[0007] S1, creating a placeholder I / O scheduler in the Linux general block layer as a shadow implementation of the programmable I / O scheduler;

[0008] S2, defining a struct bpf_struct_ops operation object for the placeholder I / O scheduler, where the struct bpf_struct_ops operation object is a structure in the Linux kernel for supporting eBPF programs, and is used to provide an operation interface for the data structure of the placeholder I / O scheduler in the kernel;

[0009] S3, implementing the struct bpf_struct_ops operation object of the placeholder I / O scheduler;

[0010] S4, registering the struct bpf_struct_ops of the placeholder I / O scheduler into the eBPF subsystem of the kernel to expose the hook points of the placeholder I / O scheduler to users outside the kernel, so that the I / O scheduler written by users outside the kernel using eBPF programs can be dynamically loaded into the kernel to replace the placeholder I / O scheduler.

[0011] Optionally, when creating the placeholder I / O scheduler in the Linux general block layer in step S1, it includes creating the placeholder I / O scheduler using the I / O scheduler definition data structure struct elevator_type, and generating a set of operation interface functions, operation attributes and names required for the I / O scheduler definition data structure struct elevator_type, where the operation attributes only include read / write attributes.

[0012] Optionally, defining the struct bpf_struct_ops operation object of the placeholder I / O scheduler in step S2 includes:

[0013] S2.1, Declare the placeholder I / O scheduler to be implemented in the kernel / bpf / bpf_struct_ops_types.h header file of the Linux system kernel;

[0014] S2.2, Create a block / bpf_dummy.c source file in the system. The block / bpf_dummy.c source file is used for the placeholder function of the BPF program loader. Define the placeholder I / O scheduler to be implemented using the struct bpf_struct_ops in the block / bpf_dummy.c source file, and define in the struct bpf_struct_ops: a validator for the eBPF program's legality, an initialization interface function, registration and deregistration interface functions, the core operation interface of the I / O scheduler, and a name.

[0015] Optionally, the struct bpf_struct_ops operation object for implementing the placeholder I / O scheduler in step S3 includes:

[0016] S3.1, Implement a validator operation table, which is used to define a validator for the eBPF program's legality. The validator is used to verify whether the eBPF program has unsafe memory access or an infinite loop, and determines that the eBPF program fails the verification and rejects the loading of the I / O scheduler written by the user using the eBPF program when there is unsafe memory access or an infinite loop;

[0017] S3.2, Implement an initialization interface function, which is used to initialize the metadata BTF of the eBPF program, so that the out-of-kernel programmable I / O scheduling instance can quickly access the kernel's data structures and symbol information based on the information in the metadata BTF;

[0018] S3.3, Implement registration and deregistration interface functions. The registration interface function is executed when the eBPF program is loaded and is used to register the placeholder I / O scheduler to the I / O scheduler linked list of the general block layer; the deregistration interface function is executed when the eBPF program is deregistered and is used to remove the placeholder I / O scheduler from the I / O scheduler linked list of the general block layer;

[0019] S3.4, Implement the core operation interface of the I / O scheduler to execute the scheduling operation of the placeholder I / O scheduler and export it.

[0020] Optionally, the scheduling operations of the placeholder I / O scheduler include:

[0021] a: Implement a scheduling queue initialization operation, including initializing the placeholder I / O scheduler and configuring the length and initial state of the scheduling request queue of the placeholder I / O scheduler;

[0022] b: Implement the enqueue operation of the scheduling queue, including monitoring the received I / O requests, inserting the received I / O requests into the scheduling request queue of the placeholder I / O scheduler, and the scheduling request queue adopts a first-in-first-out queue;

[0023] c: Implement the merge operation of the scheduling queue, including for the I / O requests in the scheduling request queue, determining whether there are multiple I / O requests with consecutive logical block addresses (LBAs) in each I / O request. If there are multiple I / O requests with consecutive logical block addresses (LBAs), then merge the multiple I / O requests with consecutive logical block addresses (LBAs) into one I / O request to improve the I / O processing efficiency;

[0024] d: Implement the dequeue operation of the scheduling queue, including using the first-in-first-out algorithm to take out the I / O requests in the scheduling request queue and dispatch them to the underlying storage medium;

[0025] e: Implement the completion operation of the scheduling queue, including notifying the upper layer of the I / O request after the storage medium processes the I / O request, and responding to the processing of the completion of the I / O request, including updating the I / O completion status of the placeholder I / O scheduler and updating the internal status of the placeholder I / O scheduler.

[0026] Optionally, registering the struct bpf_struct_ops of the placeholder I / O scheduler to the eBPF subsystem of the kernel in step S4 includes: registering all the metadata BTF of the placeholder I / O scheduler to the BPF subsystem of the kernel using the BPF program registration function interface; registering the structure of the placeholder I / O scheduler to the program list of the BPF subsystem of the kernel using the specified registration function interface.

[0027] Optionally, registering all the metadata BTF of the placeholder I / O scheduler to the BPF subsystem of the kernel using the BPF program registration function interface means registering all the metadata BTF of the placeholder I / O scheduler to the BPF subsystem of the kernel using the register_btf_kfunc_id_set() function interface of the BPF program; registering the structure of the placeholder I / O scheduler to the program list of the BPF subsystem of the kernel using the specified registration function interface means registering the structure of the placeholder I / O scheduler to the program list of the BPF subsystem of the kernel using the register_bpf_struct_ops() function interface.

[0028] In addition, the present invention also provides an implementation device of a programmable I / O scheduler, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the implementation method of the programmable I / O scheduler.

[0029] In addition, the present invention also provides a computer-readable storage medium storing a computer program or instruction, which is programmed or configured to execute the implementation method of the programmable I / O scheduler through a processor.

[0030] In addition, the present invention also provides a computer program product including a computer program or instruction, which is programmed or configured to execute the implementation method of the programmable I / O scheduler through a processor.

[0031] Compared with the prior art, the present invention can mainly achieve the following beneficial effects: (1) Strong practicability. The programmable I / O scheduler implemented by the present invention enables users to conveniently, quickly, and securely customize online the I / O scheduling strategy that conforms to the characteristics of their underlying hardware and upper-layer services. (2) Originality of the implementation method. The implementation method based on eBPF has low coupling and strong customizability. Most importantly, it has very low invasiveness to the Linux kernel, and an I / O scheduler instance can be implemented online without modifying the kernel source code. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a basic flowchart of the method of the embodiment of the present invention.

[0033] Figure 2 is a flowchart of the operation object of struct bpf_struct_op implemented in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] The present invention provides an implementation method and device for a programmable I / O scheduler. An I / O scheduler framework is implemented through eBPF technology, enabling users to customize and expand the I / O scheduler online, flexibly, and conveniently, and greatly saving the required development cost and cycle. Among them, eBPF (Extended Berkeley Packet Filter) is a technology for executing user-defined code in the Linux kernel, which enables users to dynamically load and execute programs in the kernel space without modifying the kernel source code. The emergence of eBPF provides feasibility for this requirement. To enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0035] As Figure 1 shown, the present embodiment provides an implementation method for a programmable I / O scheduler, including the following steps:

[0036] S1, create a placeholder I / O scheduler (dummy I / O scheduler) in the Linux general block layer as a shadow implementation of the programmable I / O scheduler;

[0037] S2. Define the struct bpf_struct_ops operation object of the placeholder I / O scheduler. The struct bpf_struct_ops operation object is a structure in the Linux kernel used to support eBPF programs, and is used to provide an operation interface for the data structure of the placeholder I / O scheduler in the kernel.

[0038] S3. Implement the struct bpf_struct_ops operation object of the placeholder I / O scheduler.

[0039] S4. Register the struct bpf_struct_ops of the placeholder I / O scheduler into the eBPF subsystem of the kernel to expose the hook point of the placeholder I / O scheduler to users outside the kernel, so that the I / O scheduler written by users outside the kernel using eBPF programs can be dynamically loaded into the kernel to replace the placeholder I / O scheduler. The HOOK point refers to a specific location where code can be dynamically inserted, allowing developers or system modules to insert custom code before or after a specific operation. It is usually used in subsystems such as networking, I / O, and scheduling to extend kernel functionality or implement specific operations. The implementation method of the programmable I / O scheduler in this embodiment combines the placeholder I / O scheduler and the I / O scheduler written using eBPF programs to implement a programmable I / O scheduler, integrating the security, flexibility, and extensibility of eBPF, capable of meeting the I / O capacity requirements of specific user scenarios, online customizing I / O scheduling policies that conform to the characteristics of its underlying hardware and upper-layer services, enabling users to customize and extend the I / O scheduler online, flexibly, and conveniently, and significantly saving the required development costs and cycles.

[0040] In step S1 of this embodiment, a placeholder I / O scheduler (dummy I / O scheduler) is created in the Linux general block layer as a shadow implementation of the programmable I / O scheduler; the I / O scheduler of the Linux system is the core component of the block general block layer, responsible for operations such as enqueuing, arranging, and dispatching all I / O requests flowing through the general block layer. Therefore, it is necessary to first implement a simple dummy scheduler in the general block layer. The so-called dummy scheduler refers to a scheduler that has basic operation interfaces and attributes, but the implementation can be the most basic, mainly playing a placeholder role. The I / O scheduling instance implemented based on the eBPF program outside the kernel can dynamically replace it. When creating a placeholder I / O scheduler (dummy I / O scheduler) in the Linux general block layer in step S1 of this embodiment, it includes using the I / O scheduler definition data structure struct elevator_type to create the placeholder I / O scheduler, and generating the set of operation interface functions, operation attributes, and name required for the I / O scheduler definition data structure struct elevator_type, where the operation attributes only include read / write attributes.

[0041] The core member definitions of the I / O scheduler definition data structure struct elevator_type are as follows:

[0042] struct elevator_type {

[0043] struct elevator_mq_ops ops; / / The set of operation interface functions of the I / O scheduler

[0044] struct elv_fs_entry elevator_attrs; / / Attributes used to interact with the user space

[0045] const char elevator_name; / / The name of the scheduler

[0046] struct module elevator_owner; / / Pointer to the kernel module where the I / O scheduler is located

[0047] struct list_head list; / / Linked to the global scheduler type list

[0048] };

[0049] To implement a dummy I / O scheduler, it is actually to implement the interfaces and attributes required by the structure. In this embodiment, the interfaces and attributes required by the structure include: implementing the operation interface function set of the I / O scheduler, which is described by the function struct elevator_mq_ops ops and includes the initialization operation of the scheduler, allocating necessary data structures, scheduling of I / O requests, etc. Implement the operation attributes of the dummy scheduler, which are described by the struct elv_fs_entry structure and are used for interaction with the user space; the dummy scheduler is only implemented as a shadow of a programmable I / O scheduler and only needs to implement simple read and write attributes. Define the name of the dummy scheduler. The name of the I / O scheduler is used to uniquely identify a scheduler. As an alternative implementation, in order to improve recognition, it is named "bpf_dummy_iosched" in this embodiment.

[0050] In step S2, define the struct bpf_struct_ops operation object of the dummy I / O scheduler. The struct bpf_struct_ops operation object is a type of eBPF program that can be used to provide an operation interface for specific data structures in the kernel. This type of eBPF program is used in conjunction with the structured operation object (struct bpf_struct_ops) in the kernel to clarify how to perform operations on specific data structures in the kernel; by defining the struct bpf_struct_ops operation object of the I / O scheduler, it indicates to the eBPF subsystem that a HOOK point for the I / O scheduler will be implemented, as well as the specific operations and interfaces to be implemented at this HOOK point.

[0051] In step S2 of this embodiment, the struct bpf_struct_ops operation object of the dummy I / O scheduler includes:

[0052] S2.1, Declare the dummy I / O scheduler to be implemented (i.e., "bpf_dummy_iosched") in the kernel / bpf / bpf_struct_ops_types.h header file of the Linux system kernel. The statement is as follows:

[0053] BPF_STRUCT_OPS_TYPE(bpf_dummy_iosched);

[0054] S2.2. Create a source code file block / bpf_dummy.c in the system. The source code file block / bpf_dummy.c is used for the placeholder function of the BPF program loader. In the source code file block / bpf_dummy.c, use the structure struct bpf_struct_ops to define the placeholder I / O scheduler to be implemented, and in the structure struct bpf_struct_ops, the following are defined: a validator for the eBPF program's legality, an initialization interface function, registration and deregistration interface functions, a core operation interface for the I / O scheduler, and a name. For example, in this embodiment, the definition in the structure struct bpf_struct_ops is as follows:

[0055] struct bpf_struct_ops bpf_bpf_dummy_iosched = {

[0056] .verifier_ops =&bpf_iosched_verifier_ops, / / Define the validator for the eBPF program's legality

[0057] .init = bpf_iosched_init, / / Define the initialization interface function

[0058] .reg = bpf_iosched_reg, / / Define the registration interface function

[0059] .unreg = bpf_iosched_unreg, / / Define the deregistration interface function

[0060] .cfi_stubs =&__bpf_ops_iosched_ops, / / Define the core operation interface for the I / O scheduler

[0061] .name = "bpf_iosched_ops", / / Define the name

[0062] };

[0063] Among them, the __bpf_ops_iosched_ops structure described by.cfi_stubs is the core of the I / O scheduler, which defines various operations of I / O scheduling, including enqueueing I / O requests, dispatching I / O requests, and merging I / O requests, etc. The source code file block / bpf_dummy.c created in the system is an object under the block device. Block is the abbreviation of the Generic Block Layer in the Linux kernel, which is responsible for tasks such as I / O scheduling, request management, scheduler selection, and cache processing of block devices.

[0064] eBPF is known for its lightweight security and flexibility. Since eBPF programs directly operate on kernel objects, if they are allowed to pass without verification and inspection, it is likely to cause kernel data pollution and even system crashes, which does not conform to the original design intention of eBPF. Therefore, it is necessary to implement verification operations for the struct bpf_struct_ops object of the programmable I / O scheduler. These operations are used to verify when loading the eBPF scheduling instance to ensure that the program meets the security and correctness requirements. As Figure 2 shown, the struct bpf_struct_ops operation object for implementing the placeholder I / O scheduler in step S3 of this embodiment includes:

[0065] S3.1, implement a verifier (bpf_iosched_verifier_op) operation table. The verifier operation table is used to define a verifier for the legality of the eBPF program. The verifier is used to verify whether the eBPF program has unsafe memory access or infinite loops, and determines that the eBPF program verification fails to reject loading the I / O scheduler written by the user using the eBPF program when there is unsafe memory access or infinite loops;

[0066] S3.2, implement an initialization interface function (bpf_iosched_init). The initialization interface function is used to initialize the metadata BTF of the eBPF program so that the out-of-kernel programmable I / O scheduling instance can quickly access the kernel data structure and symbol information based on the information of the metadata BTF. Metadata BTF is a kernel type information format designed specifically for BTF programs and is used to describe kernel data structures to facilitate the eBPF program to quickly and correctly access these structures. Therefore, in the initialization operation of the bpf_struct_ops operation object, the determination and verification of the BTF format of the interface exposed by the I / O scheduler to the outside are completed;

[0067] S3.3, implement registration and deregistration interface functions. The registration and deregistration operations of the bpf_struct_ops object are used to register or deregister the eBPF program to a certain subsystem respectively. For the programmable I / O scheduler, the registration operation is to register to the general block layer after the out-of-kernel eBPF scheduling instance replaces the dummy I / O scheduler, and the deregistration operation is the opposite. Among them, the registration interface function (static int bpf_iosched_reg) is executed when the eBPF program is loaded and is used to register the placeholder I / O scheduler to the I / O scheduler linked list of the general block layer. Specifically:

[0068] static int bpf_iosched_reg(void kdata, struct bpf_link link),

[0069] Among them, void kdata is a pointer to kernel data, and struct bpf_link link is a connection structure between the BPF program and the kernel, used to manage the life cycle of the BPF program. It allows user-space programs to attach BPF programs to specific hook points (HOOK points) in the kernel and manage operations such as loading, unloading, and updating;

[0070] The unregistration interface function (static void bpf_iosched_unreg) is executed when the eBPF program is unregistered, and is used to remove the placeholder I / O scheduler from the I / O scheduler linked list of the general block layer. Specifically:

[0071] static void bpf_iosched_unreg(void kdata, struct bpf_link link);

[0072] S3.4, Implement the core operation interface of the I / O scheduler to execute the scheduling operation of the placeholder I / O scheduler and export it. Corresponding to the registration function, it is used to clean up related resources, that is, to remove the programmable I / O scheduler from the I / O scheduler linked list of the general block layer. Export the scheduling interface of the dummy I / O scheduler in the bpf_struct_ops object; A series of operations are defined and implemented in the above process, and these operations are all centered around being able to dynamically replace the dummy I / O scheduler. Here, it is necessary to explicitly export the scheduling interface of this scheduler in the bpf_struct_ops object so that eBPF can locate the kernel I / O scheduler to be replaced when loading the out-of-kernel implemented I / O scheduler.

[0073] Such as Figure 2 shown, in step S3 of this embodiment, 4) Implement the core operation interface of the I / O scheduler and export it. This attribute is described by.cfi_stubs. A complete I / O scheduler includes the most basic initialization operation, enqueue operation, merge operation, dequeue operation, completion operation, etc. Specifically, the scheduling operations of the placeholder I / O scheduler include:

[0074] a: Implement the scheduling queue initialization operation, including initializing the placeholder I / O scheduler and configuring the length and initial state of the scheduling request queue of the placeholder I / O scheduler;

[0075] b: Implement the enqueue operation of the scheduling queue, including monitoring the received I / O requests, inserting the received I / O requests into the scheduling request queue of the placeholder I / O scheduler, and the scheduling request queue adopts a first-in-first-out queue;

[0076] c: Implement the merge operation of the scheduling queue, including for the I / O requests in the scheduling request queue, judging whether there are multiple I / O requests with consecutive logical block addresses (LBAs) in each I / O request. If there are multiple I / O requests with consecutive logical block addresses (LBAs), then merge the multiple I / O requests with consecutive logical block addresses (LBAs) into one I / O request to improve the I / O processing efficiency;

[0077] d: Implement the dequeue operation of the scheduling queue, including using the first-in-first-out algorithm to take out the I / O requests in the scheduling request queue and dispatch them to the underlying storage medium;

[0078] e: Implement the completion operation of the scheduling queue, including notifying the upper layer of the I / O request after the storage medium processes the I / O request, and responding to the processing of the completion of the I / O request, including updating the I / O completion status of the placeholder I / O scheduler and updating the internal status of the placeholder I / O scheduler.

[0079] Through the above steps, the struct bpf_struct_ops operation object of the programmable I / O scheduler is implemented; in the previous step, the specific interfaces to be implemented by this operation object have been defined, and these interfaces provide an object-oriented way to manage, initialize, verify, and update the attributes of the programmable I / O scheduler. Therefore, this process is the key to the programmable I / O scheduler. For example, initializing the metadata of the eBPF program, verifying whether the members of the program are legal, registering and unregistering the I / O scheduler instances implemented by ebpf, checking and updating the kernel data structure, etc. are all completed in this step.

[0080] In step S4 of this embodiment, registering the struct bpf_struct_ops of the placeholder I / O scheduler into the eBPF subsystem of the kernel includes: registering all the metadata BTF of the placeholder I / O scheduler into the BPF subsystem of the kernel using the BPF program registration function interface; registering the structure of the placeholder I / O scheduler into the program list of the BPF subsystem of the kernel using the specified registration function interface. Specifically, in this embodiment, registering all the metadata BTF of the placeholder I / O scheduler into the BPF subsystem of the kernel using the BPF program registration function interface means registering all the metadata BTF of the placeholder I / O scheduler into the BPF subsystem of the kernel using the BPF program registration function interface register_btf_kfunc_id_set(); registering the structure of the placeholder I / O scheduler into the program list of the BPF subsystem of the kernel using the specified registration function interface means; registering the structure of the placeholder I / O scheduler into the program list of the BPF subsystem of the kernel using the specified registration function interface register_bpf_struct_ops(). The operation object is registered into the ebpf subsystem through the register_bpf_struct_ops interface. After the registration is completed, the HOOK point (hook point) of this I / O scheduler is exposed to the out-of-kernel user, and the out-of-kernel user can use the ebpf program to write a specific I / O scheduler and load it into the kernel to replace the initially implemented dummy I / O scheduler.

[0081] In addition, this embodiment also provides an implementation device for a programmable I / O scheduler, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the implementation method of the programmable I / O scheduler.

[0082] In addition, this embodiment also provides a computer-readable storage medium, in which a computer program or instruction is stored, and the computer program or instruction is programmed or configured to execute the implementation method of the programmable I / O scheduler through a processor.

[0083] In addition, this embodiment also provides a computer program product, including a computer program or instruction, and the computer program or instruction is programmed or configured to execute the implementation method of the programmable I / O scheduler through a processor.

[0084] Those skilled in the art should understand that the technical solutions provided by the present invention can be in the form of a method, a system, or a computer program product. Therefore, the present invention can be implemented in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can be in the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code. The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks. These computer program instructions can also be stored in a computer-readable memory capable of guiding a computer or other programmable data processing devices to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implement the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks. These computer program instructions can also be loaded onto a computer or other programmable data processing devices, such that a series of operation steps are executed on the computer or other programmable devices to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable devices provide steps for implementing the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0085] The above is only the preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art in this technical field, several improvements and refinements made without departing from the principle of the present invention should also be regarded as within the protection scope of the present invention.

Claims

1. A method for implementing a programmable I / O scheduler, characterized in that, It includes the following steps: S1. Create a placeholder I / O scheduler in the Linux general block layer as a shadow implementation of the programmable I / O scheduler; S2. Define the struct bpf_struct_ops operation object of the placeholder I / O scheduler. The struct bpf_struct_ops operation object is a structure in the Linux kernel used to support eBPF programs, and is used to provide an operation interface for the data structure of the placeholder I / O scheduler in the kernel; S3. Implement the struct bpf_struct_ops operation object of the placeholder I / O scheduler; S4. Register the struct bpf_struct_ops of the placeholder I / O scheduler into the eBPF subsystem of the kernel to expose the hook point of the placeholder I / O scheduler to users outside the kernel, so that the I / O scheduler written by users using eBPF programs located outside the kernel can be dynamically loaded into the kernel to replace the placeholder I / O scheduler.

2. The implementation method of the programmable I / O scheduler according to claim 1, characterized in that When creating the placeholder I / O scheduler in the Linux general block layer in step S1, it includes using the I / O scheduler to define the data structure struct elevator_type to create the placeholder I / O scheduler, and generating the set of operation interface functions, operation attributes, and names required for the I / O scheduler to define the data structure struct elevator_type, where the operation attributes only include read / write attributes.

3. The implementation method of the programmable I / O scheduler according to claim 1, wherein Defining the struct bpf_struct_ops operation object of the placeholder I / O scheduler in step S2 includes: S2.

1. Declare the placeholder I / O scheduler to be implemented in the kernel / bpf / bpf_struct_ops_types.h header file of the Linux system kernel; S2.

2. Create the block / bpf_dummy.c source file in the system. The block / bpf_dummy.c source file is used for the placeholder function of the BPF program loader. Use the struct bpf_struct_ops in the block / bpf_dummy.c source file to define the placeholder I / O scheduler to be implemented, and define in the struct bpf_struct_ops: the validator for the legality of the eBPF program, the initialization interface function, the registration and deregistration interface functions, the core operation interface of the I / O scheduler, and the name.

4. The implementation method of the programmable I / O scheduler according to claim 3, wherein Implementing the struct bpf_struct_ops operation object of the placeholder I / O scheduler in step S3 includes: S3.

1. Implement the validator operation table. The validator operation table is used to define the validator for the legality of the eBPF program. The validator is used to verify whether the eBPF program has unsafe memory access or infinite loops, and when there is unsafe memory access or infinite loops, it determines that the eBPF program verification fails to reject the loading of the I / O scheduler written by users using eBPF programs; S3.

2. Implement the initialization interface function, which is used to initialize the metadata BTF of the eBPF program, so that the out-of-kernel programmable I / O scheduling instance can quickly access the kernel data structure and symbol information based on the information in the metadata BTF; S3.

3. Implement the registration and deregistration interface functions. The registration interface function is executed when the eBPF program is loaded and is used to register the placeholder I / O scheduler into the I / O scheduler linked list of the general block layer. The deregistration interface function is executed when the eBPF program is deregistered and is used to remove the placeholder I / O scheduler from the I / O scheduler linked list of the general block layer; S3.

4. Implement the I / O scheduler core operation interface to execute the scheduling operation of the placeholder I / O scheduler and export it.

5. The implementation method of the programmable I / O scheduler according to claim 4, characterized in that The scheduling operations of the placeholder I / O scheduler include: a: Implement the scheduling queue initialization operation, including initializing the placeholder I / O scheduler and setting the length and initial state of the scheduling request queue of the placeholder I / O scheduler; b: Implement the enqueue operation of the scheduling queue, including monitoring the received I / O requests, inserting the received I / O requests into the scheduling request queue of the placeholder I / O scheduler, and the scheduling request queue adopts a first-in-first-out queue; c: Implement the merge operation of the scheduling queue, including for the I / O requests in the scheduling request queue, judging whether there are multiple I / O requests with consecutive logical block addresses (LBAs) in each I / O request. If there are multiple I / O requests with consecutive logical block addresses (LBAs), then merge the multiple I / O requests with consecutive logical block addresses (LBAs) into one I / O request to improve the I / O processing efficiency; d: Implement the dequeue operation of the scheduling queue, including using the first-in-first-out algorithm to take out the I / O requests in the scheduling request queue and dispatch them to the underlying storage medium; e: Implement the completion operation of the scheduling queue, including notifying the upper layer of the I / O requests completed by the storage medium and responding to the processing of the completion of the I / O requests, including updating the I / O completion status of the placeholder I / O scheduler and updating the internal state of the placeholder I / O scheduler.

6. The implementation method of the programmable I / O scheduler according to claim 4, wherein In step S4, registering the struct bpf_struct_ops of the placeholder I / O scheduler into the eBPF subsystem of the kernel includes: using the BPF program registration function interface to register all the metadata BTF of the placeholder I / O scheduler into the BPF subsystem of the kernel; using the specified registration function interface to register the structure of the placeholder I / O scheduler into the program list of the BPF subsystem of the kernel.

7. The implementation method of the programmable I / O scheduler according to claim 6, characterized in that The registration of all metadata BTFs of the placeholder I / O scheduler into the BPF subsystem of the kernel using the BPF program registration function interface means registering all metadata BTFs of the placeholder I / O scheduler into the BPF subsystem of the kernel using the BPF program registration function interface register_btf_kfunc_id_set(); the registration of the structure of the placeholder I / O scheduler into the program list of the BPF subsystem of the kernel using the specified registration function interface means registering the structure of the placeholder I / O scheduler into the program list of the BPF subsystem of the kernel using the specified registration function interface register_bpf_struct_ops().

8. An implementation device of a programmable I / O scheduler, comprising a microprocessor and a memory connected to each other, characterized in that, The microprocessor is programmed or configured to execute the implementation method of the programmable I / O scheduler according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instruction is programmed or configured to execute the implementation method of the programmable I / O scheduler according to any one of claims 1 to 7 through a processor.

10. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instruction is programmed or configured to execute the implementation method of the programmable I / O scheduler according to any one of claims 1 to 7 through a processor.

Images