Scheduling control method and device for disk IO request, computer equipment and medium

Abstract

The invention provides a scheduling control method and device for a disk IO request, computer equipment and a medium. The scheduling control method comprises the steps that a service request queue used for accessing a disk is created; receiving disk IO requests of a plurality of target services; distributing the disk IO request to a corresponding service request queue according to the service priority information; calculating the number of residual processing requests of the service request queue; when the number of the remaining processing requests is greater than the preset remaining number, determining a service request queue and calculating the number of first requests that can be submitted to a kernel; triggering a disk IO request of the service request queue according to the first request quantity and submitting the disk IO request to a kernel; processing the disk IO request through the kernel, and executing service callback on the processed disk IO request through the user space; according to the technical scheme provided by the embodiment of the invention, the system stability can be greatly improved on the premise of giving full play to the IO performance of the disk by reasonably carrying out scheduling control on the IO request of the disk.

Description

Technical Field

[0001] This invention relates to the field of disk scheduling technology, and in particular to a method, apparatus, computer device, and medium for scheduling and controlling disk I / O requests. Background Technology

[0002] With the rapid development and innovation of information technology, various industries have also experienced rapid growth. The information explosion has generated massive amounts of data, requiring high-speed analysis at low cost to extract value. Large-scale storage systems are fundamental, and disk access control is the most critical module within these systems. One reason is that disks are currently the hardware bottleneck of mainstream computers; improving disk access speed and maximizing disk input / output (IO) performance can effectively improve overall system performance. Another reason is that different business operations have different requirements, and effective control of disk access is essential for providing stable service. However, current disk scheduling algorithms cannot control access at the business level. In big data systems, large-scale and high-concurrency access often occurs, easily leading to disk thrashing, which can severely disrupt the system, causing business interruptions and significantly reducing system stability. Summary of the Invention

[0003] The purpose of this invention is to at least solve one of the technical problems existing in the prior art, and to provide a method, apparatus, computer equipment and medium for scheduling and controlling disk I / O requests. By reasonably scheduling and controlling disk I / O requests under the premise of considering business priorities, the pressure on the disk caused by a large number of disk I / O requests can be reduced, and the system stability can be greatly improved while giving full play to the disk I / O performance.

[0004] In a first aspect, embodiments of the present invention provide a method for scheduling and controlling disk I / O requests, including:

[0005] Create a service request queue for accessing the disk, wherein the service request queue includes a priority request queue and a normal request queue, and the priority of the priority request queue is greater than the priority of the normal request queue;

[0006] Receive disk I / O requests for multiple target services, wherein the disk I / O requests include service priority information reflecting the priority of the target services;

[0007] The disk I / O requests are distributed to the corresponding service request queues based on the service priority information.

[0008] Calculate the number of remaining processing requests in the business request queue;

[0009] When the number of remaining processing requests is greater than the preset remaining number, determine the queue of business requests to be processed and calculate the first number of requests that can be submitted to the kernel.

[0010] Based on the first number of requests, the disk I / O requests of the business request queue are submitted to the kernel;

[0011] The kernel processes the disk I / O requests, and the user space executes business callbacks for the completed disk I / O requests.

[0012] The disk I / O request scheduling and control method provided by the embodiments of the present invention has at least the following beneficial effects: By creating priority request queues and ordinary request queues, different priorities can be assigned to different service request queues. When disk I / O requests for each target service are received, the priority of the disk I / O request can be determined according to the service priority information. Then, the disk I / O request is distributed to the corresponding service request queue according to the service priority information, which can fully consider the scheduling and control at the service level. Then, the remaining number of processing requests in the service request queue is calculated. When the remaining number of processing requests is greater than the preset remaining number, the service request queue to be processed is determined and the first number of requests that can be submitted to the kernel is calculated. Then, according to the first number of requests... The number of requests triggers the submission of disk I / O requests from the business request queue to the kernel. This allows for flexible adjustment of the number of disk I / O requests submitted to the kernel, significantly reducing disk jitter. Even under high concurrency, the system can ensure normal operation of the business. After the disk I / O requests are submitted to the kernel in an orderly manner, the kernel processes them and executes business callbacks and returns results through user space. This can efficiently complete the processing of business requests. By reasonably scheduling and controlling disk I / O requests while considering business priorities, the pressure on the disk caused by a large number of disk I / O requests can be reduced. This can greatly improve system stability while fully utilizing disk I / O performance.

[0013] In the above-described method for scheduling and controlling disk I / O requests, the step of triggering the submission of disk I / O requests from the service request queue to the kernel based on the first request quantity includes:

[0014] Determine whether the number of the first requests exceeds half of the preset maximum kernel processing capacity;

[0015] If the number of the first requests does not exceed half of the preset maximum kernel processing capacity, the number of the first requests is returned and the disk I / O requests of the business request queue are submitted to the kernel.

[0016] The aforementioned method for scheduling and controlling disk I / O requests also includes:

[0017] When the number of the first requests exceeds half of the preset maximum kernel processing capacity, the current request submission process exits to wait for the next disk I / O request to trigger the business request queue to be submitted to the kernel.

[0018] In the above-described method for scheduling and controlling disk I / O requests, the step of distributing the disk I / O requests to the corresponding service request queues based on the service priority information includes:

[0019] The disk I / O requests are categorized by service priority type based on the service priority information.

[0020] When the service priority type of the disk I / O request is determined to be the first priority type, the disk I / O request is distributed to the priority request queue;

[0021] When the service priority type of the disk I / O request is determined to be the second priority type, the disk I / O request is distributed to the normal request queue.

[0022] In the above-described method for scheduling and controlling disk I / O requests, after distributing the disk I / O requests to the corresponding service request queues according to the service priority information, the scheduling and control method further includes:

[0023] The disk I / O requests in the service request queue are sorted from high to low priority based on the service priority information, so that high-priority disk I / O requests are placed at the front of the service request queue.

[0024] In the above-described method for scheduling and controlling disk I / O requests, the step of submitting the disk I / O request that triggers the service request queue to the kernel includes:

[0025] When the number of remaining processing requests is greater than the number of first requests, the disk I / O requests in the business request queue are submitted to the kernel according to the number of first requests.

[0026] Unsubmitted disk I / O requests in the business request queue are cached in a preset thread pool to wait for the next submission to the kernel.

[0027] In the above-described method for scheduling and controlling disk I / O requests, when the disk I / O request is processed by the kernel, the scheduling and control method further includes:

[0028] Determine the current running state of the kernel;

[0029] When the current running state of the kernel is determined to be abnormal, the currently processed disk I / O request is recorded and the preset exception handling program is executed.

[0030] After a preset exception handling time, the disk I / O request is processed again.

[0031] Secondly, embodiments of the present invention provide a scheduling and control device for disk I / O requests, comprising:

[0032] The request queue management module is used to create a business request queue for accessing the disk. The business request queue includes a priority request queue and a normal request queue, wherein the priority of the priority request queue is greater than the priority of the normal request queue.

[0033] The disk I / O request module is used to receive disk I / O requests from multiple target services, wherein the disk I / O requests include service priority information reflecting the priority of the target services;

[0034] The disk I / O request module is further configured to distribute the disk I / O request to the corresponding service request queue according to the service priority information;

[0035] The disk I / O proxy module is also used to calculate the number of remaining processing requests in the business request queue;

[0036] The disk I / O proxy module is also used to determine the queue of business requests to be processed and calculate the first number of requests that can be submitted to the kernel when the number of remaining processing requests is greater than the preset remaining number.

[0037] An IO request submission module is used to trigger the submission of disk IO requests from the business request queue to the kernel based on the first number of requests;

[0038] The processing module is used to process the disk I / O request through the kernel and execute business callbacks on the processed disk I / O request through user space.

[0039] The disk I / O request scheduling and control device provided by the embodiments of the present invention has at least the following beneficial effects: By creating priority request queues and ordinary request queues, different priorities can be assigned to different service request queues. When receiving disk I / O requests for each target service, the priority of the disk I / O request can be determined according to the service priority information. Then, the disk I / O request is distributed to the corresponding service request queue according to the service priority information, which can fully consider the scheduling control at the service level. Then, the remaining number of processing requests in the service request queue is calculated. When the remaining number of processing requests is greater than the preset remaining number, the service request queue to be processed is determined and the first number of requests that can be submitted to the kernel is calculated. Then, according to the first number of requests... The number of requests triggers the submission of disk I / O requests from the business request queue to the kernel. This allows for flexible adjustment of the number of disk I / O requests submitted to the kernel, significantly reducing disk jitter. Even under high concurrency, the system can ensure normal operation of the business. After the disk I / O requests are submitted to the kernel in an orderly manner, the kernel processes them and executes business callbacks and returns results through user space. This can efficiently complete the processing of business requests. By reasonably scheduling and controlling disk I / O requests while considering business priorities, the pressure on the disk caused by a large number of disk I / O requests can be reduced. This can greatly improve system stability while fully utilizing disk I / O performance.

[0040] Thirdly, embodiments of the present invention provide a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the scheduling control method described in the first aspect embodiment above.

[0041] The computer device provided by the embodiments of the present invention has at least the following beneficial effects: By creating priority request queues and ordinary request queues, different priorities can be assigned to different service request queues. When disk I / O requests for each target service are received, the priority of the disk I / O request can be determined according to the service priority information. Then, the disk I / O request is distributed to the corresponding service request queue according to the service priority information, which can fully consider the scheduling control at the service level. Then, the remaining number of processing requests in the service request queue is calculated. When the remaining number of processing requests is greater than the preset remaining number, the service request queue to be processed is determined and the first number of requests that can be submitted to the kernel is calculated. Then, based on the first number of requests... Disk I / O requests that trigger the business request queue are submitted to the kernel. This allows for flexible adjustment of the submitted disk I / O requests, greatly reducing disk jitter and ensuring normal business operation even under high concurrency. After the disk I / O requests are submitted to the kernel in an orderly manner, the kernel processes them and executes business callbacks and returns results through user space. This efficiently completes business request processing. By considering business priorities and reasonably scheduling and controlling disk I / O requests, the pressure on the disk caused by a large number of disk I / O requests can be reduced, and system stability can be greatly improved while fully utilizing disk I / O performance.

[0042] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the scheduling and control method described in the first aspect of the embodiments above.

[0043] The computer-readable storage medium provided according to embodiments of the present invention has at least the following beneficial effects: By creating priority request queues and ordinary request queues, different priorities can be assigned to different service request queues. When disk I / O requests for each target service are received, the priority of the disk I / O request can be determined according to the service priority information. Then, the disk I / O request is distributed to the corresponding service request queue according to the service priority information, which can fully consider the scheduling control at the service level. Then, the remaining number of processing requests in the service request queue is calculated. When the remaining number of processing requests is greater than the preset remaining number, the service request queue to be processed is determined and the first number of requests that can be submitted to the kernel is calculated. Then, according to the first request... The number of disk I / O requests triggered by the business request queue is submitted to the kernel. This allows for flexible adjustment of the number of disk I / O requests submitted to the kernel, greatly reducing disk jitter. Even under high concurrency, the normal operation of the business can be guaranteed. After the disk I / O requests are submitted to the kernel in an orderly manner, the kernel processes the disk I / O requests and executes business callbacks on the processed disk I / O requests in user space and returns the results. This can efficiently complete the processing of business requests. By reasonably scheduling and controlling the disk I / O requests while considering business priorities, the pressure on the disk caused by a large number of disk I / O requests can be reduced. This can greatly improve system stability while fully utilizing disk I / O performance.

[0044] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the description and the drawings. Attached Figure Description

[0045] The present invention will be further described below with reference to the accompanying drawings and embodiments;

[0046] Figure 1 This is a flowchart of the disk I / O request scheduling and control method provided in Embodiment 1 of the present invention.

[0047] Figure 2 This is a flowchart of the disk I / O request scheduling and control method provided in Embodiment 2 of the present invention;

[0048] Figure 3 This is a flowchart of the disk I / O request scheduling and control method provided in Embodiment 3 of the present invention;

[0049] Figure 4 This is a flowchart of the disk I / O request scheduling and control method provided in Embodiment 4 of the present invention;

[0050] Figure 5 This is a flowchart of the disk I / O request scheduling and control method provided in Embodiment 5 of the present invention;

[0051] Figure 6 This is a flowchart illustrating the specific implementation of the disk I / O request scheduling and control method provided in Embodiment Six of the present invention.

[0052] Figure 7 This is a schematic diagram of the structure of the computer device provided in Embodiment 7 of the present invention. Detailed Implementation

[0053] This section will describe in detail specific embodiments of the present invention. Preferred embodiments of the present invention are shown in the accompanying drawings. The purpose of the drawings is to supplement the textual description with graphics, so that people can intuitively and vividly understand each technical feature and overall technical solution of the present invention, but they should not be construed as limiting the scope of protection of the present invention.

[0054] It should be understood that in the description of the embodiments of the present invention, the use of terms such as "first" and "second" is only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of technical features indicated, or implicitly indicating the order of the technical features indicated. "At least one" means one or more; "more than" means two or more; "greater than," "less than," and "exceeding" are understood to exclude the stated number; "above," "below," and "within" are understood to include the stated number; "several" means one or more, unless otherwise explicitly defined. "And / or" describes the relationship between related objects, indicating that three relationships can exist. It can be understood that A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. Where A and B can be singular or plural.

[0055] Furthermore, unless otherwise explicitly specified and limited, the term "connection / linkage" should be interpreted broadly. For example, it can be a fixed or movable connection, a detachable or non-detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or a connection capable of communication; it can be a direct connection or an indirect connection via an intermediate medium. It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown in the flowchart. It should also be noted that the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0056] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0057] like Figure 1 As shown, an embodiment of the first aspect of the present invention provides a method for scheduling and controlling disk I / O requests, including but not limited to steps S110 to S170:

[0058] Step S110: Create a service request queue for accessing the disk, wherein the service request queue includes a priority request queue and a normal request queue, and the priority of the priority request queue is greater than the priority of the normal request queue;

[0059] It's important to note that to handle high-concurrency business access demands, two business request queues can be pre-created to cache disk I / O requests. These queues are designated as a priority request queue and a regular request queue, with the priority queue having a higher priority than the regular request queue. Essentially, the priority queue stores high-priority business requests, while the regular request queue stores lower-priority requests. All disk I / O requests can access the disk through these two queues.

[0060] Step S120: Receive disk I / O requests from multiple target services, wherein the disk I / O requests include service priority information reflecting the priority of the target services;

[0061] Once disk I / O requests for each target service are received, the service priority information corresponding to each target service can be obtained, thereby determining the priority of the target service, that is, determining the priority of the corresponding disk I / O request.

[0062] Step S130: Distribute disk I / O requests to the corresponding business request queues according to business priority information;

[0063] Based on business priority information, the priority of disk I / O requests can be determined, and these requests can then be distributed to appropriate business request queues. For example, disk I / O requests for high-priority businesses can be placed in a priority request queue, while those for low-priority businesses can be placed in a regular request queue. In subsequent request processing, disk I / O requests in the priority request queue can be processed first, followed by those in the regular request queue.

[0064] Step S140: Calculate the number of remaining processing requests in the business request queue;

[0065] Step S150: When the number of remaining processing requests is greater than the preset remaining number, determine the queue of business requests to be processed and calculate the number of first requests that can be submitted to the kernel;

[0066] It should be noted that by determining whether the number of remaining processing requests in the business request queue is greater than a preset remaining number, it can be determined whether there are any unprocessed disk I / O requests in the business request queue. When the number of remaining processing requests is greater than the preset remaining number, it indicates that there are still unprocessed disk I / O requests in the business request queue. Therefore, a queue of business requests to be processed is determined, and the number of first requests that can be submitted to the kernel is calculated. It can be understood that the number of first requests is less than or equal to the number of remaining processing requests. In one embodiment, the number of first requests can be determined based on the number of remaining processing requests and the actual processing status of the kernel. If the actual processing status of the kernel is relatively busy, the number of first requests can be relatively small, prioritizing the processing of some disk I / O requests.

[0067] In addition, during the actual request processing, you can choose the appropriate business request queue according to the actual needs, that is, you can choose the priority request queue or the normal request queue.

[0068] In one embodiment, the remaining quantity is preset to 0. When the remaining number of processing requests is greater than 0, the queue of business requests to be processed is determined and the number of first requests that can be submitted to the kernel is calculated. When the remaining number of processing requests is less than or equal to 0, the process ends.

[0069] Step S160: Based on the first number of requests, trigger the submission of disk I / O requests from the business request queue to the kernel;

[0070] The number of disk I / O requests that can be submitted to the kernel can be determined based on the number of the first request. This can then trigger the submission of disk I / O requests from the business request queue to the kernel, facilitating subsequent processing of the disk I / O requests.

[0071] Step S170: The kernel processes the disk I / O request and executes the business callback for the completed disk I / O request in user space.

[0072] Once a disk I / O request is submitted to the kernel, the kernel can process the disk I / O request and notify the user space after processing. Upon receiving the notification, the user space can execute the corresponding business callback function for the disk I / O request and return the result, thus efficiently completing the business request processing.

[0073] In one embodiment, the kernel can issue a notification after completing a disk I / O request, then execute a Job callback function on the completed disk I / O request through user space, and submit any unprocessed disk I / O requests in the business request queue to the kernel.

[0074] The disk I / O request scheduling and control method provided in the first aspect embodiment above can assign different priorities to different service request queues by creating priority request queues and ordinary request queues. When receiving disk I / O requests for each target service, the priority of the disk I / O request can be determined according to the service priority information. Then, the disk I / O request is distributed to the corresponding service request queue according to the service priority information, which can fully consider the scheduling and control at the service level. Then, the remaining number of processing requests in the service request queue is calculated. When the remaining number of processing requests is greater than the preset remaining number, the service request queue to be processed is determined and the first number of requests that can be submitted to the kernel is calculated. Then, the kernel is triggered according to the first number of requests. Disk I / O requests submitted to the kernel from the business request queue can be flexibly adjusted, greatly reducing disk jitter. Even under high concurrency, the normal operation of the business can be guaranteed. After the disk I / O requests are submitted to the kernel in an orderly manner, the kernel processes the disk I / O requests and executes business callbacks on the processed disk I / O requests in user space and returns the results. This can efficiently complete the processing of business requests. By reasonably scheduling and controlling disk I / O requests while considering business priorities, the pressure on the disk caused by a large number of disk I / O requests can be reduced. While making full use of disk I / O performance, the system stability can be greatly improved.

[0075] It is understandable that by effectively scheduling and controlling disk I / O requests, the system stability under high concurrency access can be greatly improved, which is especially suitable for big data system application scenarios that generate a large number of concurrent access requests to the disk.

[0076] like Figure 2 As shown, in the above-mentioned disk I / O request scheduling and control method, step S160 triggers the submission of disk I / O requests from the service request queue to the kernel based on the first request quantity, including but not limited to steps S210 and S220:

[0077] Step S210: Determine whether the number of the first requests exceeds half of the preset maximum number of kernels that can process them;

[0078] Step S220: When the number of the first request does not exceed half of the preset maximum kernel processing capacity, return the number of the first request and trigger the disk I / O request of the business request queue to be submitted to the kernel.

[0079] In this embodiment, the preset maximum kernel processing capacity represents the maximum number of requests that the kernel can process. First, it is determined whether the number of first requests that can be submitted to the kernel exceeds the preset maximum kernel processing capacity. If the number of first requests does not exceed half of the preset maximum kernel processing capacity, it is considered that the kernel can efficiently process disk I / O requests at this time. Then, the number of first requests is returned and the disk I / O requests of the business request queue are triggered to be submitted to the kernel, which can optimize kernel processing performance and greatly improve system stability.

[0080] The above-mentioned method for scheduling and controlling disk I / O requests also includes the following steps:

[0081] When the number of first requests exceeds half of the preset maximum kernel processing capacity, the current request submission process exits to wait for the next disk I / O request to trigger the business request queue to be submitted to the kernel.

[0082] In this embodiment, if the number of first requests exceeds half of the preset maximum kernel processing capacity, the current request submission process is exited, and the process waits for the next disk I / O request to trigger the business request queue to be submitted to the kernel. In other words, the submission of disk I / O requests to the kernel is temporarily suspended to avoid excessive disk I / O requests putting pressure on the disk and can greatly reduce the occurrence of disk jitter.

[0083] like Figure 3 As shown, in the above-mentioned disk I / O request scheduling and control method, step S130 distributes disk I / O requests to the corresponding service request queues according to service priority information, including but not limited to steps S310 to S330:

[0084] Step S310: Classify disk I / O requests by business priority type according to business priority information;

[0085] Step S320: When the service priority type of the disk I / O request is determined to be the first priority type, the disk I / O request is distributed to the priority request queue;

[0086] Step S330: When the business priority type of the disk I / O request is determined to be the second priority type, the disk I / O request is distributed to the normal request queue.

[0087] In this embodiment, the priority of the corresponding disk I / O request can be determined based on the business priority information, thereby classifying the disk I / O request into business priority types. When the business priority type of the disk I / O request is determined to be the first priority type, it means that the disk I / O request is a high-priority business request, and the disk I / O request is distributed to the priority request queue. When the business priority type of the disk I / O request is determined to be the second priority type, it means that the disk I / O request is a low-priority business request, and the disk I / O request is distributed to the normal request queue. By dividing disk I / O requests of different priorities into priority request queues and normal request queues, the scheduling control at the business level can be fully considered, and the system requirements can be better adapted.

[0088] In one embodiment, the service priority information can be a priority value. When the priority value of the disk I / O request is greater than or equal to a preset value, the service priority type of the disk I / O request is determined to be a first priority type. When the priority value of the disk I / O request is less than the preset value, the service priority type of the disk I / O request is determined to be a second priority type.

[0089] In the above-mentioned disk I / O request scheduling and control method, after step S130 distributes the disk I / O requests to the corresponding service request queues according to service priority information, the scheduling and control method further includes the following steps:

[0090] The disk I / O requests in the business request queue are sorted from high to low priority based on business priority information, so that high-priority disk I / O requests are placed at the front of the business request queue.

[0091] In this embodiment, after disk I / O requests are distributed to the service request queue, the disk I / O requests in the service request queue are sorted from high to low priority according to the service priority information. The sorted service request queue can be understood as a priority queue. Since high-priority disk I / O requests are placed at the front of the service request queue, high-priority disk I / O requests can be submitted to the kernel for processing first to meet the service requirements.

[0092] like Figure 4 As shown, in the above-described disk I / O request scheduling and control method, step S220 triggers the submission of the disk I / O request from the service request queue to the kernel, including but not limited to steps S410 and S420:

[0093] Step S410: When the number of remaining processing requests is greater than the number of first requests, submit the disk I / O requests in the business request queue to the kernel according to the number of first requests;

[0094] Step S420: Cache unsubmitted disk I / O requests in the business request queue to a preset thread pool to wait for the next submission to the kernel.

[0095] In this embodiment, since there is a limit to the number of requests submitted to the kernel each time, when the number of remaining processing requests is greater than the first number of requests, a portion of the disk I / O requests in the business request queue are submitted to the kernel according to the first number of requests, and the unsubmitted disk I / O requests in the business request queue are cached in a preset thread pool, which can effectively improve the data read and write speed. After the kernel finishes processing the disk I / O requests, the disk I / O requests in the thread pool are submitted to the kernel, which can accelerate data access efficiency and effectively improve performance.

[0096] In one embodiment, uncommitted disk I / O requests in the business request queue can be cached in the Committer thread pool.

[0097] like Figure 5 As shown, in the above-described disk I / O request scheduling and control method, when the disk I / O request is processed by the kernel in step S170, the scheduling and control method also includes, but is not limited to, steps S510 to S530:

[0098] Step S510: Determine the current running state of the kernel;

[0099] Step S520: When it is determined that the current running state of the kernel is an abnormal state, record the currently processed disk I / O request and execute the preset exception handler;

[0100] Step S530: After the preset exception handling time, process the disk I / O request again.

[0101] In this embodiment, considering that kernel operation may be abnormal when the kernel processes disk I / O requests, the current running state of the kernel can be determined in real time. When it is determined that the current running state of the kernel is abnormal, the currently processed disk I / O request is recorded and a preset exception handler is executed. The kernel operation is debugged in time to restore it to a normal state. After the preset exception handling time of the exception handler is executed, the kernel has recovered to a normal state. Since the abnormal state will affect the normal processing of the currently processed disk I / O request, the recorded disk I / O request is reprocessed. This can ensure the normal operation of the business and reduce the situation where the system becomes unresponsive and interrupts the business.

[0102] In one embodiment, the abnormal state of the kernel may be caused by illegal instructions, memory access errors, or deadlocks.

[0103] like Figure 6As shown, in one embodiment, upon receiving a disk I / O request, the priority of the disk I / O request is first determined. High-priority service requests are distributed to the priority request queue, while other low-priority service requests are distributed to the normal request queue. Then, the remaining number of processing requests in the service request queue (priority request queue or normal request queue) is calculated, and it is determined whether the remaining number of processing requests is greater than 0. If the remaining number of processing requests is less than or equal to 0, the process ends. If the remaining number of processing requests is greater than 0, the service request queue to be processed is determined, and the number of first requests that can be submitted to the kernel is calculated. It is determined whether the number of first requests exceeds half of the preset maximum kernel processing capacity. If it does, the current request submission process exits. If it does not exceed, a disk I / O request is triggered to be submitted to the kernel. The kernel processes the disk I / O request and, after processing, notifies the user space. Upon receiving the notification, the user space executes a service callback.

[0104] Secondly, embodiments of the present invention provide a disk I / O request scheduling and control device, including a request queue management module, a disk I / O request module, a disk I / O proxy module, an I / O request submission module, and a processing module. The request queue management module is used to create a service request queue for accessing the disk. The service request queue includes a priority request queue and a normal request queue, with the priority of the priority request queue being higher than the priority of the normal request queue. The disk I / O request module is used to receive disk I / O requests from multiple target services. The disk I / O requests include service priority information reflecting the priority of the target services. The disk I / O request module is also used to distribute the disk I / O requests to the corresponding service request queues according to the service priority information. The disk I / O proxy module is also used to calculate the remaining number of processing requests in the service request queues. The disk I / O proxy module is also used to determine the service request queues to be processed and calculate the first number of requests that can be submitted to the kernel when the remaining number of processing requests is greater than a preset remaining number. The I / O request submission module is used to trigger the submission of disk I / O requests from the service request queues to the kernel according to the first number of requests. The processing module is used to process the disk I / O requests through the kernel and execute service callbacks on the processed disk I / O requests through user space.

[0105] The disk I / O request scheduling and control device provided in this embodiment of the invention can assign different priorities to different service request queues by creating priority request queues and ordinary request queues. When receiving disk I / O requests for each target service, the priority of the disk I / O request can be determined according to the service priority information. Then, the disk I / O request is distributed to the corresponding service request queue according to the service priority information, which can fully consider the scheduling and control at the service level. Then, the remaining number of processing requests in the service request queue is calculated. When the remaining number of processing requests is greater than the preset remaining number, the service request queue to be processed is determined and the first number of requests that can be submitted to the kernel is calculated. Then, the disk I / O requests of the service request queue are triggered to be submitted to the kernel according to the first number of requests. The disk I / O requests submitted to the kernel can be flexibly adjusted, greatly reducing disk jitter. Even under the condition of high concurrency access, the normal operation of the service can be guaranteed. After the disk I / O requests are submitted to the kernel in an orderly manner, the kernel processes the disk I / O requests and executes the service callback and returns the result through the user space for the processed disk I / O requests. The service request processing can be completed efficiently. Under the premise of considering the service priority, by reasonably scheduling and controlling the disk I / O requests, the pressure on the disk caused by a large number of disk I / O requests can be reduced. Under the premise of fully utilizing the disk I / O performance, the system stability can be greatly improved.

[0106] In one embodiment, the processing module includes a kernel module and an IO request processing module. The kernel module is used to process disk IO requests and access the disk. After processing all disk IO requests, it notifies the IO request processing module. The IO request processing module is used to receive the notification issued by the kernel after processing the disk IO requests and to execute business callbacks for the processed disk IO requests.

[0107] In the aforementioned disk I / O request scheduling and control device, the submission of disk I / O requests from the service request queue to the kernel is triggered based on the first request quantity, including:

[0108] Determine if the number of the first request exceeds half of the preset maximum kernel processing capacity;

[0109] If the number of the first request does not exceed half of the preset maximum kernel processing capacity, return the number of the first request and trigger the submission of disk I / O requests from the business request queue to the kernel.

[0110] The aforementioned disk I / O request scheduling and control device also includes a wait-to-submit module, which is used for:

[0111] When the number of first requests exceeds half of the preset maximum kernel processing capacity, the current request submission process exits to wait for the next disk I / O request to trigger the business request queue to be submitted to the kernel.

[0112] In the aforementioned disk I / O request scheduling and control device, the disk I / O requests are distributed to the corresponding service request queues according to service priority information, including:

[0113] Disk I / O requests are categorized by business priority type based on business priority information;

[0114] When the business priority type of a disk I / O request is determined to be the first priority type, the disk I / O request is distributed to the priority request queue;

[0115] When the business priority type of a disk I / O request is determined to be the second priority type, the disk I / O request is distributed to the normal request queue.

[0116] The aforementioned disk I / O request scheduling and control device also includes a queue sorting module, which is used for:

[0117] The disk I / O requests in the business request queue are sorted from high to low priority based on business priority information, so that high-priority disk I / O requests are placed at the front of the business request queue.

[0118] In the aforementioned disk I / O request scheduling and control device, the disk I / O request that triggers the service request queue is submitted to the kernel, including:

[0119] When the number of remaining processing requests is greater than the number of first requests, the disk I / O requests in the business request queue are submitted to the kernel according to the number of first requests.

[0120] Unsubmitted disk I / O requests in the business request queue are cached in a predefined thread pool to wait for the next submission to the kernel.

[0121] In the aforementioned disk I / O request scheduling and control device, the processing module is further configured to:

[0122] Determine the current running state of the kernel;

[0123] When the kernel's current running state is determined to be abnormal, the currently processed disk I / O request is recorded and the preset exception handler is executed.

[0124] After the preset exception handling time, the disk I / O request will be processed again.

[0125] It should be noted that the specific implementation method and corresponding technical effects of the disk I / O request scheduling and control device in this embodiment of the invention can be referred to the specific implementation method and corresponding technical effects of the disk I / O request scheduling and control method described above.

[0126] like Figure 7As shown, a third aspect embodiment of the present invention provides a computer device 700, including a memory 710, a processor 720, and a computer program stored in the memory 710 and executable on the processor 720, wherein the processor 720 and the memory 710 can be connected via a bus or other means. Figure 7 The diagram illustrates an example of a bus connection, where the processor, when executing a computer program, implements the scheduling control method described in the first aspect above, for example, implementing the method described above. Figures 1 to 6 The methods and steps in the text.

[0127] The computer device provided in this embodiment of the invention can assign different priorities to different business request queues by creating priority request queues and ordinary request queues. When receiving disk I / O requests for various target businesses, the priority of the disk I / O requests can be determined according to business priority information. Then, the disk I / O requests are distributed to the corresponding business request queues according to the business priority information, which can fully consider the scheduling control at the business level. Then, the remaining number of processing requests in the business request queue is calculated. When the remaining number of processing requests is greater than the preset remaining number, the business request queue to be processed is determined and the first number of requests that can be submitted to the kernel is calculated. Then, the disk I / O requests of the business request queue are triggered to be submitted to the kernel according to the first number of requests. The disk I / O requests submitted to the kernel can be flexibly adjusted, greatly reducing disk jitter. Even under high concurrency access, the normal operation of the business can be guaranteed. After the disk I / O requests are submitted to the kernel in an orderly manner, the kernel processes the disk I / O requests and executes business callbacks and returns results through user space for the processed disk I / O requests. Business request processing can be completed efficiently. Under the premise of considering business priority, by reasonably scheduling and controlling the disk I / O requests, the pressure on the disk caused by a large number of disk I / O requests can be reduced. Under the premise of fully utilizing the disk I / O performance, the system stability can be greatly improved.

[0128] A fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions that can be used to cause a computer to perform the scheduling and control method of the first aspect embodiment above, for example, to perform the above-described... Figures 1 to 6The method and steps described are as follows. By creating priority request queues and normal request queues, different priorities can be assigned to different business request queues. When disk I / O requests for various target businesses are received, the priority of the disk I / O requests can be determined based on business priority information. Then, the disk I / O requests are distributed to the corresponding business request queues according to the business priority information, which can fully consider the scheduling control at the business level. Then, the remaining number of processing requests in the business request queue is calculated. When the number of remaining processing requests is greater than the preset remaining number, the business request queue to be processed is determined and the number of first requests that can be submitted to the kernel is calculated. Then, the disk I / O requests of the business request queue are triggered to be submitted to the kernel according to the number of first requests. This can flexibly adjust the disk I / O requests submitted to the kernel, greatly reducing disk jitter. Even under high concurrency access, the normal operation of the business can be guaranteed. After the disk I / O requests are submitted to the kernel in an orderly manner, the kernel processes the disk I / O requests and executes business callbacks and returns results through user space for the processed disk I / O requests. This can efficiently complete the processing of business requests. Under the premise of considering business priorities, reasonable scheduling control of disk I / O requests can reduce the pressure on the disk caused by a large number of disk I / O requests. Under the premise of fully utilizing disk I / O performance, the system stability can be greatly improved.

[0129] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which may include computer storage media or non-transitory media and communication media or transient media. As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc DVD or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0130] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A method for scheduling and controlling disk I / O requests, characterized in that, include: Create a service request queue for accessing the disk, wherein the service request queue includes a priority request queue and a normal request queue, and the priority of the priority request queue is greater than the priority of the normal request queue; Receive disk I / O requests for multiple target services, wherein the disk I / O requests include service priority information reflecting the priority of the target services; The disk I / O requests are distributed to the corresponding service request queues based on the service priority information. Calculate the number of remaining processing requests in the business request queue; When the number of remaining processing requests is greater than the preset remaining number, determine the queue of business requests to be processed and calculate the first number of requests that can be submitted to the kernel. Based on the first number of requests, the disk I / O requests of the business request queue are submitted to the kernel; The kernel processes the disk I / O requests, and the user space executes business callbacks for the completed disk I / O requests.

2. The scheduling and control method according to claim 1, characterized in that, The step of triggering the submission of disk I / O requests from the service request queue to the kernel based on the first request quantity includes: Determine whether the number of the first requests exceeds half of the preset maximum kernel processing capacity; If the number of the first requests does not exceed half of the preset maximum kernel processing capacity, the number of the first requests is returned and the disk I / O requests of the business request queue are submitted to the kernel.

3. The scheduling and control method according to claim 2, characterized in that, Also includes: When the number of the first requests exceeds half of the preset maximum kernel processing capacity, the current request submission process exits to wait for the next disk I / O request to trigger the business request queue to be submitted to the kernel.

4. The scheduling and control method according to claim 1, characterized in that, The step of distributing the disk I / O request to the corresponding service request queue according to the service priority information includes: The disk I / O requests are categorized by service priority type based on the service priority information. When the service priority type of the disk I / O request is determined to be the first priority type, the disk I / O request is distributed to the priority request queue; When the service priority type of the disk I / O request is determined to be the second priority type, the disk I / O request is distributed to the normal request queue.

5. The scheduling and control method according to claim 1, characterized in that, After distributing the disk I / O requests to the corresponding service request queues according to the service priority information, the scheduling control method further includes: The disk I / O requests in the service request queue are sorted from high to low priority based on the service priority information, so that high-priority disk I / O requests are placed at the front of the service request queue.

6. The scheduling and control method according to claim 2, characterized in that, The submission of the disk I / O request that triggers the service request queue to the kernel includes: When the number of remaining processing requests is greater than the number of first requests, the disk I / O requests in the business request queue are submitted to the kernel according to the number of first requests. Unsubmitted disk I / O requests in the business request queue are cached in a preset thread pool to wait for the next submission to the kernel.

7. The scheduling and control method according to claim 1, characterized in that, When processing the disk I / O request through the kernel, the scheduling control method further includes: Determine the current running state of the kernel; When the current running state of the kernel is determined to be abnormal, the currently processed disk I / O request is recorded and the preset exception handling program is executed. After a preset exception handling time, the disk I / O request is processed again.

8. A scheduling and control device for disk I / O requests, characterized in that, include: The request queue management module is used to create a business request queue for accessing the disk. The business request queue includes a priority request queue and a normal request queue, wherein the priority of the priority request queue is greater than the priority of the normal request queue. The disk I / O request module is used to receive disk I / O requests from multiple target services, wherein the disk I / O requests include service priority information reflecting the priority of the target services; The disk I / O request module is further configured to distribute the disk I / O request to the corresponding service request queue according to the service priority information; The disk I / O proxy module is also used to calculate the number of remaining processing requests in the business request queue; The disk I / O proxy module is also used to determine the queue of business requests to be processed and calculate the first number of requests that can be submitted to the kernel when the number of remaining processing requests is greater than the preset remaining number. An IO request submission module is used to trigger the submission of disk IO requests from the business request queue to the kernel based on the first number of requests; The processing module is used to process the disk I / O request through the kernel and execute business callbacks on the processed disk I / O request through user space.

9. A computer device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the scheduling control method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the scheduling and control method as described in any one of claims 1 to 7.

Images