Implementation method and device of programmable I / O scheduler, medium and product

Abstract

The invention discloses an implementation method and device of a programmable I / O (Input / Output) scheduler, a medium and a product. The method comprises the following steps: creating a placeholder I / O scheduler as a shadow implementation of the programmable I / O scheduler; and defining and implementing a struct bpfstructops operation object of the placeholder I / O scheduler, and registering the struct bpfstructops operation object in an eBPF subsystem of the kernel to expose a hook point of the placeholder I / O scheduler to a user outside the kernel, so that the user outside the kernel can dynamically load the I / O scheduler written by an eBPF program into the kernel to replace the placeholder I / O scheduler. The invention aims to realize a programmable I / O (Input / Output) scheduler to meet the requirements of a user on I / O capability in a specific scene and customize an I / O scheduling strategy which conforms to own bottom hardware and upper business characteristics online, so that the user can customize and expand the I / O scheduler online, flexibly and conveniently, and the required development cost and period are greatly saved.

Description

Technical Field

[0001] The present invention relates to the field of operating systems, and particularly to a method, device, medium and product for implementing a programmable I / O scheduler. Background Art

[0002] The I / O scheduler is crucial for the Linux operating system and storage performance. Especially in a multi-tasking and complex storage environment, an efficient I / O scheduler can greatly improve the efficiency of the storage system and optimize the service level of system services by adjusting the scheduling policy and optimizing disk utilization. On the one hand, with the rapid development of wafer process and integrated circuit technology, the underlying storage components are rapidly iterating. On the other hand, with the rise of new technologies such as cloud computing and AI, the complexity and variability of upper-layer business loads are becoming increasingly obvious. The combination of these two aspects poses new requirements for the storage efficiency of the existing Linux system, especially for the I / O scheduler. For example, the current I / O scheduler is difficult to maximize the capabilities of emerging diverse storage devices such as solid-state drives (SSDs) and non-volatile memory (NVMe) devices; another example is that the current I / O scheduler design mostly faces general scenarios and lacks consideration for specific scenarios such as hybrid deployment scenarios and low-latency scenarios. If there is a method or system that can allow users to conveniently, quickly and securely customize an I / O scheduling policy that conforms to the characteristics of their underlying hardware and upper-layer services online, it will be a great improvement for both the storage efficiency of the Linux system and the user's business experience.

[0003] Current I / O schedulers in the Linux general block layer, such as Deadline, BFQ (Budget Fair Queuing), etc., are designed for general scenarios and can meet the I / O requirements of most scenarios. However, if you want to maximize I / O efficiency or perform storage QOS (Quality of Service) for specific scenarios, you need to customize the I / O scheduler separately; the process generally includes the following steps: 1) Obtain the Linux kernel source code and build a kernel module development environment; 2) Develop a specific I / O scheduler; 3) Compile & debug the newly developed I / O scheduler; 4) Stop the current business system and replace it with the new I / O scheduler. From the above process, it can be seen that whether developing or testing a new I / O scheduler to meet business requirements requires downtime and the process is very cumbersome and challenging. Even more, the above process may not succeed the first time, and often requires repeated downtime, development and debugging to meet the requirements. Summary of the Invention

[0004] Technical problems to be solved by the present invention: In view of the above problems in the prior art, an implementation method, device, medium and product of a programmable I / O scheduler are provided. The present invention aims to implement a programmable I / O scheduler to meet the I / O capacity requirements of specific user scenarios, customize an I / O scheduling policy that conforms to the characteristics of its underlying hardware and upper-layer services online, enabling users to customize and expand the I / O scheduler online, flexibly and conveniently, and greatly saving the required development costs and cycles.

[0005] To solve the above technical problems, the technical solution adopted by the present invention is as follows: An implementation method of a programmable I / O scheduler, comprising the following steps: S1, create a placeholder I / O scheduler in the Linux general block layer as a shadow implementation of the programmable I / O scheduler; S2, define the struct bpf_struct_ops operation object of the placeholder I / O scheduler, where the struct bpf_struct_ops operation object is a structure in the Linux kernel used to support eBPF programs, and is used to provide an operation interface for the data structure of the placeholder I / O scheduler in the kernel; S3, implement the struct bpf_struct_ops operation object of the placeholder I / O scheduler; S4, register the struct bpf_struct_ops of the placeholder I / O scheduler into the eBPF subsystem of the kernel to expose the hook point of the placeholder I / O scheduler to users outside the kernel, so that the I / O scheduler written by users outside the kernel using eBPF programs can be dynamically loaded into the kernel to replace the placeholder I / O scheduler.

[0006] Optionally, when creating the placeholder I / O scheduler in the Linux general block layer in step S1, it includes using the I / O scheduler to define the data structure struct elevator_type to create the placeholder I / O scheduler, and generating the set of operation interface functions, operation attributes and names required for the I / O scheduler definition data structure struct elevator_type, where the operation attributes only include read / write attributes.

[0007] Optionally, defining the struct bpf_struct_ops operation object of the placeholder I / O scheduler in step S2 includes: S2.1, declare the placeholder I / O scheduler to be implemented in the kernel / bpf / bpf_struct_ops_types.h header file of the Linux system kernel; S2.2. Create a source code file named block / bpf_dummy.c in the system. The block / bpf_dummy.c source code file is used for the placeholder function of the BPF program loader. In the block / bpf_dummy.c source code file, use the structure struct bpf_struct_ops to define the placeholder I / O scheduler to be implemented, and in the structure struct bpf_struct_ops, define: a validator for the legality of the eBPF program, an initialization interface function, registration and deregistration interface functions, a core operation interface for the I / O scheduler, and a name.

[0008] Optionally, the operation object of struct bpf_struct_ops for implementing the placeholder I / O scheduler in step S3 includes: S3.1. Implement a validator operation table, which is used to define a validator for the legality of the eBPF program. The validator is used to verify whether the eBPF program has unsafe memory access or an infinite loop, and when there is unsafe memory access or an infinite loop, it determines that the eBPF program verification fails to reject the loading of the I / O scheduler written by the user using the eBPF program. S3.2. Implement an initialization interface function, which is used to initialize the metadata BTF of the eBPF program, so that the out-of-kernel programmable I / O scheduling instance can quickly access the kernel data structure and symbol information based on the information in the metadata BTF. S3.3. Implement registration and deregistration interface functions. The registration interface function is executed when the eBPF program is loaded and is used to register the placeholder I / O scheduler to the I / O scheduler linked list of the general block layer. The deregistration interface function is executed when the eBPF program is deregistered and is used to remove the placeholder I / O scheduler from the I / O scheduler linked list of the general block layer. S3.4. Implement a core operation interface for the I / O scheduler to execute the scheduling operation of the placeholder I / O scheduler and export it.

[0009] Optionally, the scheduling operations of the placeholder I / O scheduler include: a: Implement a scheduling queue initialization operation, including initializing the placeholder I / O scheduler and configuring the length and initial state of the scheduling request queue of the placeholder I / O scheduler. b: Implement an enqueue operation for the scheduling queue, including monitoring the received I / O requests, inserting the received I / O requests into the scheduling request queue of the placeholder I / O scheduler, and the scheduling request queue uses a first-in-first-out queue. c: Implement the merging operation of the scheduling queue, including, for the I / O requests in the scheduling request queue, determining whether there are multiple I / O requests with consecutive logical block addresses (LBAs) among the individual I / O requests. If there are multiple I / O requests with consecutive LBAs, merge the multiple I / O requests with consecutive LBAs into one I / O request to improve the I / O processing efficiency; d: Implement the dequeue operation of the scheduling queue, including using the first-in, first-out (FIFO) algorithm to retrieve the I / O requests from the scheduling request queue and dispatch them to the underlying storage medium; e: Implement the completion operation of the scheduling queue, including notifying the upper layer of the I / O request after the storage medium processes the I / O request and responding to the processing of the completion of the I / O request, including updating the I / O completion status of the placeholder I / O scheduler and updating the internal state of the placeholder I / O scheduler.

[0010] Optionally, registering the struct bpf_struct_ops of the placeholder I / O scheduler into the eBPF subsystem of the kernel in step S4 includes: registering all the metadata BTF of the placeholder I / O scheduler into the BPF subsystem of the kernel using the BPF program registration function interface; registering the structure of the placeholder I / O scheduler into the program list of the BPF subsystem of the kernel using the specified registration function interface.

[0011] Optionally, registering all the metadata BTF of the placeholder I / O scheduler into the BPF subsystem of the kernel using the BPF program registration function interface means registering all the metadata BTF of the placeholder I / O scheduler into the BPF subsystem of the kernel using the register_btf_kfunc_id_set() BPF program registration function interface; registering the structure of the placeholder I / O scheduler into the program list of the BPF subsystem of the kernel using the specified registration function interface means registering the structure of the placeholder I / O scheduler into the program list of the BPF subsystem of the kernel using the register_bpf_struct_ops() specified registration function interface.

[0012] In addition, the present invention also provides an implementation device for a programmable I / O scheduler, including a microprocessor and a memory connected to each other, where the microprocessor is programmed or configured to execute the implementation method of the programmable I / O scheduler.

[0013] In addition, the present invention also provides a computer-readable storage medium storing a computer program or instruction, where the computer program or instruction is programmed or configured to be executed by a processor to execute the implementation method of the programmable I / O scheduler.

[0014] In addition, the present invention also provides a computer program product, including a computer program or instruction, which is programmed or configured to execute the implementation method of the programmable I / O scheduler through a processor.

[0015] Compared with the prior art, the present invention can mainly achieve the following beneficial effects: (1) Strong practicability. The programmable I / O scheduler implemented by the present invention enables users to conveniently, quickly, and securely customize I / O scheduling policies that conform to their underlying hardware and upper-layer service characteristics online. (2) Originality of the implementation method. The implementation method based on eBPF has low coupling and strong customizability. Most importantly, it has very low invasiveness to the Linux kernel, and an I / O scheduler instance can be implemented online without modifying the kernel source code. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a basic flowchart of the method of the embodiment of the present invention.

[0017] Figure 2 It is a flowchart of the operation object of struct bpf_struct_op implemented in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] The present invention proposes an implementation method and device for a programmable I / O scheduler. An I / O scheduler framework is implemented through eBPF technology, enabling users to customize and expand the I / O scheduler online, flexibly, and conveniently, and greatly saving the required development cost and cycle. Among them, eBPF (Extended Berkeley Packet Filter) is a technology for executing user-defined code in the Linux kernel, which enables users to dynamically load and execute programs in the kernel space without modifying the kernel source code. The emergence of eBPF provides feasibility for this requirement. In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0019] As Figure 1 shown, the present embodiment provides an implementation method for a programmable I / O scheduler, including the following steps: S1, create a placeholder I / O scheduler (dummy I / O scheduler) in the Linux general block layer as a shadow implementation of the programmable I / O scheduler; S2, define the struct bpf_struct_ops operation object of the placeholder I / O scheduler. The struct bpf_struct_ops operation object is a structure in the Linux kernel used to support eBPF programs, and is used to provide an operation interface for the data structure of the placeholder I / O scheduler in the kernel; S3. Implement the struct bpf_struct_ops operation object of the placeholder I / O scheduler; S4. Register the struct bpf_struct_ops of the placeholder I / O scheduler to the eBPF subsystem of the kernel to expose the hook points of the placeholder I / O scheduler to users outside the kernel, so that the I / O scheduler written by users using eBPF programs outside the kernel can be dynamically loaded into the kernel to replace the placeholder I / O scheduler. Among them, the HOOK point refers to a specific position where dynamic insertion can be performed, allowing developers or system modules to insert custom code before or after a specific operation. It is usually used in subsystems such as networking, I / O, and scheduling to extend kernel functions or implement specific operations. The implementation method of the programmable I / O scheduler in this embodiment combines the placeholder I / O scheduler and the I / O scheduler written using eBPF programs to implement a programmable I / O scheduler, integrating the security, flexibility, and scalability of eBPF, meeting the I / O capability requirements of users in specific scenarios, online customizing I / O scheduling policies that conform to the characteristics of their underlying hardware and upper-layer services, enabling users to customize and expand the I / O scheduler online, flexibly, and conveniently, and greatly saving the required development costs and cycles.

[0020] In step S1 of this embodiment, a placeholder I / O scheduler (dummy I / O scheduler) is created in the Linux general block layer as a shadow implementation of the programmable I / O scheduler; the I / O scheduler of the linux system is a core component of the block general block layer, responsible for operations such as enqueueing, arranging, and dispatching all I / O requests flowing through the general block layer. Therefore, it is necessary to first implement a simple dummy scheduler in the general block layer. The so-called dummy scheduler refers to one that has basic operation interfaces and attributes, but the implementation can be the most basic, mainly serving as a placeholder, and the I / O scheduling instance implemented based on the eBPF program outside the kernel can dynamically replace it. When creating a placeholder I / O scheduler (dummy I / O scheduler) in the Linux general block layer in step S1 of this embodiment, it includes using the I / O scheduler definition data structure struct elevator_type to create the placeholder I / O scheduler, and generating the set of operation interface functions, operation attributes, and names required for the I / O scheduler definition data structure struct elevator_type, where the operation attributes only include read / write attributes.

[0021] The core member definitions of the I / O scheduler definition data structure struct elevator_type are as follows: struct elevator_type { struct elevator_mq_ops ops; / / The set of operation interface functions of the I / O scheduler struct elv_fs_entry elevator_attrs; / / Attributes for interacting with user space const char elevator_name; / / The name of the scheduler struct module elevator_owner; / / Pointer to the kernel module where the I / O scheduler is located struct list_head list; / / Linked to the global scheduler type list }; To implement a dummy I / O scheduler, it is actually to implement the interfaces and attributes required by the structure. In this embodiment, the interfaces and attributes required by the structure include: implementing the set of operation interface functions of the I / O scheduler, which is described by the function struct elevator_mq_ops ops and includes the initialization operation of the scheduler, allocating necessary data structures, scheduling of I / O requests, etc. Implementing the operation attributes of the dummy scheduler, which are described by the struct elv_fs_entry structure and are used for interacting with user space; the dummy scheduler is only implemented as a shadow of a programmable I / O scheduler, and only simple read and write attributes need to be implemented. Defining the name of the dummy scheduler, the name of the I / O scheduler is used to uniquely identify a scheduler. As an optional implementation, in order to improve recognition, it is named "bpf_dummy_iosched" in this embodiment.

[0022] Define the struct bpf_struct_ops operation object of the dummy I / O scheduler in step S2. The struct bpf_struct_ops operation object is a type of eBPF program that can be used to provide an operation interface for specific kernel data structures. This type of eBPF program is used in conjunction with the structured operation object (struct bpf_struct_ops) in the kernel to clarify how to perform operations on specific kernel data structures; by defining the struct bpf_struct_ops operation object of the I / O scheduler, it indicates to the eBPF subsystem that a HOOK point for the I / O scheduler will be implemented, as well as the specific operations and interfaces to be implemented at this HOOK point.

[0023] In step S2 of this embodiment, the operation object of the struct bpf_struct_ops of the placeholder I / O scheduler includes: S2.1, Declare the placeholder I / O scheduler to be implemented (i.e., "bpf_dummy_iosched") in the kernel / bpf / bpf_struct_ops_types.h header file of the Linux system kernel. The statement is as follows: BPF_STRUCT_OPS_TYPE(bpf_dummy_iosched); S2.2, Create a block / bpf_dummy.c source file in the system. The block / bpf_dummy.c source file is used for the placeholder function of the BPF program loader. Use the struct bpf_struct_ops structure to define the placeholder I / O scheduler to be implemented in the block / bpf_dummy.c source file. And in the struct bpf_struct_ops structure, a validator for the eBPF program's legality, an initialization interface function, registration and deregistration interface functions, the core operation interface of the I / O scheduler, and a name are defined. For example, in this embodiment, the definition in the struct bpf_struct_ops structure is as follows: struct bpf_struct_ops bpf_bpf_dummy_iosched = { .verifier_ops =&bpf_iosched_verifier_ops, / / Define the validator for the eBPF program's legality .init = bpf_iosched_init, / / Define the initialization interface function .reg = bpf_iosched_reg, / / Define the registration interface function .unreg = bpf_iosched_unreg, / / Define the deregistration interface function .cfi_stubs =&__bpf_ops_iosched_ops, / / Define the core operation interface of the I / O scheduler .name = "bpf_iosched_ops", / / Define the name }; The __bpf_ops_iosched_ops structure described by.cfi_stubs is the core of the I / O scheduler, which defines various operations of I / O scheduling, including enqueueing I / O requests, dispatching I / O requests, and merging I / O requests. The source code file block / bpf_dummy.c created in the system is an object under the block device. Block is the abbreviation of the Generic Block Layer in the Linux kernel, which is responsible for tasks such as I / O scheduling, request management, scheduler selection, and cache processing of block devices.

[0024] eBPF is known for its lightweight security and flexibility. Since ebpf programs directly operate on kernel objects, if they are directly allowed to pass without being verified and checked, it is easy to cause kernel data pollution and even system crashes, which does not conform to the original design intention of eBPF. Therefore, it is necessary to implement verification operations for the struct bpf_struct_ops object of the programmable I / O scheduler. These operations are used to verify when loading the eBPF scheduling instance to ensure that the program meets the security and correctness requirements. As Figure 2 shown, the struct bpf_struct_ops operation object for implementing the placeholder I / O scheduler in step S3 of this embodiment includes: S3.1, implement a verifier (bpf_iosched_verifier_op) operation table, which is used to define a verifier for the legality of the eBPF program. The verifier is used to verify whether the eBPF program has unsafe memory access or infinite loops, and determines that the eBPF program verification fails to reject loading the I / O scheduler written by the user using the eBPF program when there are unsafe memory access or infinite loops; S3.2, implement an initialization interface function (bpf_iosched_init), which is used to initialize the metadata BTF of the eBPF program, so that the out-of-kernel programmable I / O scheduling instance can quickly access the kernel data structure and symbol information based on the information of the metadata BTF; Metadata BTF is a kernel type information format designed specifically for BTF programs, which is used to describe kernel data structures to facilitate ebpf programs to quickly and correctly access these structures. Therefore, in the initialization operation of the bpf_struct_ops operation object, the BTF format determination and verification of the interface exposed by the I / O scheduler are completed; S3.3. Implement the registration and deregistration interface functions. The registration and deregistration operations of the bpf_struct_ops object are used to register or deregister the eBPF program into a certain subsystem respectively. For the programmable I / O scheduler, the registration operation is to register it into the general block layer after replacing the dummy I / O scheduler with the out-of-kernel eBPF scheduling instance, and the deregistration operation is the opposite. The registration interface function (static int bpf_iosched_reg) is executed when the eBPF program is loaded, and is used to register the placeholder I / O scheduler into the I / O scheduler linked list of the general block layer. Specifically: static int bpf_iosched_reg(void kdata, struct bpf_link link), where void kdata is a pointer to the kernel data, and struct bpf_link link is the connection structure between the BPF program and the kernel, which is used to manage the life cycle of the BPF program. It allows the user-space program to attach the BPF program to a specific hook point (HOOK point) of the kernel and manage operations such as its loading, unloading, and updating; The deregistration interface function (static void bpf_iosched_unreg) is executed when the eBPF program is deregistered, and is used to remove the placeholder I / O scheduler from the I / O scheduler linked list of the general block layer. Specifically: static void bpf_iosched_unreg(void kdata, struct bpf_link link); S3.4. Implement the core operation interface of the I / O scheduler to execute the scheduling operation of the placeholder I / O scheduler and export it. Corresponding to the registration function, it is used to clean up relevant resources, that is, to remove the programmable I / O scheduler from the I / O scheduler linked list of the general block layer. Export the scheduling interface of the dummy I / O scheduler in the bpf_struct_ops object; In the above process, a series of operations are defined and implemented, and these operations are all centered around being able to dynamically replace the dummy I / O scheduler. Here, it is necessary to explicitly export the scheduling interface of this scheduler in the bpf_struct_ops object so that eBPF can locate the kernel I / O scheduler to be replaced when loading the out-of-kernel implemented I / O scheduler.

[0025] Such as Figure 2As shown, in step S3 of this embodiment, 4) implement and export the core operation interface of the I / O scheduler. This attribute is described by.cfi_stubs. A complete I / O scheduler includes basic initialization operations, enqueue operations, merge operations, dequeue operations, completion operations, etc. Specifically, the scheduling operations of the placeholder I / O scheduler include: a: Implement the scheduling queue initialization operation, including initializing the placeholder I / O scheduler and configuring the length and initial state of the scheduling request queue of the placeholder I / O scheduler; b: Implement the enqueue operation of the scheduling queue, including monitoring the received I / O requests, inserting the received I / O requests into the scheduling request queue of the placeholder I / O scheduler, and the scheduling request queue adopts a first-in-first-out queue; c: Implement the merge operation of the scheduling queue, including for the I / O requests in the scheduling request queue, determining whether there are multiple I / O requests with consecutive logical block addresses (LBAs) in each I / O request. If there are multiple I / O requests with consecutive logical block addresses (LBAs), then merge the multiple I / O requests with consecutive logical block addresses (LBAs) into one I / O request to improve the I / O processing efficiency; d: Implement the dequeue operation of the scheduling queue, including using the first-in-first-out algorithm to take out the I / O requests in the scheduling request queue and dispatch them to the underlying storage medium; e: Implement the completion operation of the scheduling queue, including notifying the upper layer of the I / O request after the storage medium processes the I / O request and responding to the processing of the completion of the I / O request, including updating the I / O completion status of the placeholder I / O scheduler and updating the internal state of the placeholder I / O scheduler.

[0026] Through the above steps, the struct bpf_struct_ops operation object of the programmable I / O scheduler is implemented; the interfaces that the operation object specifically needs to implement have been defined in the previous step. These interfaces provide an object-oriented way to manage, initialize, verify, and update the attributes of the programmable I / O scheduler. Therefore, this process is the key to the programmable I / O scheduler. For example, initializing the metadata of the eBPF program, verifying whether the members of the program are legal, registering and unregistering the I / O scheduler instances implemented by ebpf, checking and updating the kernel data structure, etc. are all completed in this step.

[0027] In step S4 of this embodiment, registering the struct bpf_struct_ops of the placeholder I / O scheduler into the eBPF subsystem of the kernel includes: registering all the metadata BTF of the placeholder I / O scheduler into the BPF subsystem of the kernel using the BPF program registration function interface; registering the structure of the placeholder I / O scheduler into the program list of the BPF subsystem of the kernel using the specified registration function interface. Specifically, in this embodiment, registering all the metadata BTF of the placeholder I / O scheduler into the BPF subsystem of the kernel using the BPF program registration function interface means registering all the metadata BTF of the placeholder I / O scheduler into the BPF subsystem of the kernel using the BPF program registration function interface register_btf_kfunc_id_set(); registering the structure of the placeholder I / O scheduler into the program list of the BPF subsystem of the kernel using the specified registration function interface means; registering the structure of the placeholder I / O scheduler into the program list of the BPF subsystem of the kernel using the specified registration function interface register_bpf_struct_ops(). The operation object is registered into the ebpf subsystem through the register_bpf_struct_ops interface. After the registration is completed, the HOOK point of the I / O scheduler is exposed to the out-of-kernel user. The out-of-kernel user can then write a specific I / O scheduler using the ebpf program and load it into the kernel to replace the initially implemented dummy I / O scheduler.

[0028] In addition, this embodiment also provides an implementation device for a programmable I / O scheduler, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the implementation method of the programmable I / O scheduler.

[0029] In addition, this embodiment also provides a computer-readable storage medium, in which a computer program or instruction is stored, and the computer program or instruction is programmed or configured to execute the implementation method of the programmable I / O scheduler through a processor.

[0030] In addition, this embodiment also provides a computer program product, including a computer program or instruction, and the computer program or instruction is programmed or configured to execute the implementation method of the programmable I / O scheduler through a processor.

[0031] Those skilled in the art should understand that the technical solutions provided by the present invention can be in the form of a method, a system, or a computer program product. Therefore, the present invention can be implemented in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can be in the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, as well as the combination of processes and / or blocks in the flowchart and / or block diagram. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Therefore, the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0032] The above are only the preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art in this technical field, several improvements and refinements made without departing from the principle of the present invention should also be regarded as within the protection scope of the present invention.

Claims

1. A method for implementing a programmable I / O scheduler, characterized in that: The steps include: S1, create a placeholder I / O scheduler in the Linux common block layer as a shadow implementation of the programmable I / O scheduler; S2, define a struct bpf_struct_ops operation object of the placeholder I / O scheduler, wherein the struct bpf_struct_ops operation object is a structure in the Linux kernel for supporting eBPF programs, and is used to provide an operation interface for the data structure of the placeholder I / O scheduler in the kernel; S3, implement the struct bpf_struct_ops operation object of the placeholder I / O scheduler; S4, register the struct bpf_struct_ops of the placeholder I / O scheduler into the eBPF subsystem of the kernel to expose the hook point of the placeholder I / O scheduler to users outside the kernel, so that the I / O scheduler written by the user outside the kernel using the eBPF program can be dynamically loaded into the kernel to replace the placeholder I / O scheduler.

2. The method for implementing a programmable I / O scheduler according to claim 1, wherein: When creating a placeholder I / O scheduler in the Linux general block layer in step S1, it includes using the I / O scheduler definition data structure structelevator_type to create the placeholder I / O scheduler, and generating the operation interface function set, operation attributes and name required for the I / O scheduler definition data structure structelevator_type, where the operation attributes only include read and write attributes.

3. The method for implementing a programmable I / O scheduler according to claim 1, wherein: The struct bpf_struct_ops operation object defining the placeholder I / O scheduler in step S2 includes: S2.1, declare the placeholder I / O scheduler to be implemented in the kernel / bpf / bpf_struct_ops_types.h header file of the Linux kernel; S2.2, create a block / bpf_dummy.c source code file in the system. The block / bpf_dummy.c source code file is used for the placeholder function of the BPF program loader. The block / bpf_dummy.c source code file uses the structure structbpf_struct_ops to define the placeholder I / O scheduler to be implemented, and the structure struct bpf_struct_ops defines: a verifier of the legitimacy of the eBPF program, an initialization interface function, a registration and deregistration interface function, and an I / O scheduler core operation interface and name.

4. The method for implementing a programmable I / O scheduler according to claim 3, wherein: The struct bpf_struct_ops operation object that implements the placeholder I / O scheduler in step S3 includes: S3.1, implement a validator operation table, the validator operation table is used to define a validator for the legitimacy of the eBPF program, the validator is used to verify whether the eBPF program has unsafe memory access or infinite loop, and when there is unsafe memory access or infinite loop, determine that the eBPF program verification fails to refuse to load the I / O scheduler written by the user using the eBPF program; S3.2, implement an initialization interface function, which is used to initialize the metadata BTF of the eBPF program so that the off-core programmable I / O scheduling instance can quickly access the kernel's data structure and symbol information based on the metadata BTF information; S3.3, implement the registration and deregistration interface functions, wherein the registration interface function is executed when the eBPF program is loaded, and is used to register the placeholder I / O scheduler to the I / O scheduler linked list of the general block layer; the deregistration interface function is executed when the eBPF program is deregistered, and is used to remove the placeholder I / O scheduler from the I / O scheduler linked list of the general block layer; S3.4, implement the I / O scheduler core operation interface for executing the scheduling operation of the placeholder I / O scheduler and exporting it.

5. The method for implementing a programmable I / O scheduler according to claim 4, characterized in that: The scheduling operation of the placeholder I / O scheduler includes: a: Implement the scheduling queue initialization operation, including initializing the placeholder I / O scheduler and setting the length and initial state of the scheduling request queue of the placeholder I / O scheduler; b: Implement the queue entry operation of the scheduling queue, including monitoring the received I / O requests and inserting the received I / O requests into the scheduling request queue of the placeholder I / O scheduler, and the scheduling request queue adopts a first-in-first-out queue; c: Implementing the merging operation of the scheduling queue, including determining whether there are multiple I / O requests with connected logical block addresses LBA in each I / O request for the I / O requests in the scheduling request queue, and if there are multiple I / O requests with connected logical block addresses LBA, merging the multiple I / O requests with connected logical block addresses LBA into one I / O request to improve I / O processing efficiency; d: Implement the dequeue operation of the scheduling queue, including taking out the I / O requests in the scheduling request queue using the first-in-first-out algorithm and dispatching them to the underlying storage medium; e: Implement the completion operation of the scheduling queue, including notifying the upper layer of the I / O request completed by the storage medium, and responding to the processing of the I / O request completion, including the I / O completion status update of the placeholder I / O scheduler, and the internal status update of the placeholder I / O scheduler.

6. The method for implementing a programmable I / O scheduler according to claim 4, wherein: Registering the struct bpf_struct_ops of the placeholder I / O scheduler into the eBPF subsystem of the kernel in step S4 includes: using the BPF program registration function interface to register all metadata BTFs of the placeholder I / O scheduler into the BPF subsystem of the kernel; and using the specified registration function interface to register the structure of the placeholder I / O scheduler into the program list in the BPF subsystem of the kernel.

7. The method for implementing a programmable I / O scheduler according to claim 6, characterized in that: The use of the BPF program registration function interface to register all metadata BTFs of the placeholder I / O scheduler into the BPF subsystem of the kernel refers to using the BPF program registration function interface register_btf_kfunc_id_set() to register all metadata BTFs of the placeholder I / O scheduler into the BPF subsystem of the kernel; using the specified registration function interface to register the structure of the placeholder I / O scheduler into the program list in the BPF subsystem of the kernel refers to using the specified registration function interface register_bpf_struct_ops() to register the structure of the placeholder I / O scheduler into the program list in the BPF subsystem of the kernel.

8. A device for implementing a programmable I / O scheduler, comprising a microprocessor and a memory connected to each other, characterized in that: The microprocessor is programmed or configured to execute the implementation method of the programmable I / O scheduler according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program or instruction stored therein, characterized in that: The computer program or instruction is programmed or configured to execute the implementation method of the programmable I / O scheduler according to any one of claims 1 to 7 through a processor.

10. A computer program product comprising a computer program or instructions, characterized in that The computer program or instruction is programmed or configured to execute the implementation method of the programmable I / O scheduler according to any one of claims 1 to 7 through a processor.

Images