Database request scheduling method and device and computer equipment

Abstract

The invention discloses a database request scheduling method and device and computer equipment. The method comprises the following steps: receiving a database request, and determining the type of the database request; the priority of the database request is determined according to the type of the database request, the priority of the database priority volume write request in the database request is the highest, and the priority volume at least comprises a redo log file; and under the condition that the database load exceeds a preset threshold value, scheduling the database request according to the priority. According to the method and the device, the technical problem of relatively low database log writing efficiency caused by relatively high database redo log writing time delay in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a database request scheduling method, device and computer equipment. Background Art

[0002] Data packet scheduling technology is the process of determining when a data packet can be scheduled and in what order to schedule the data packet. For database request scheduling methods, priority scheduling technology in related technologies is rarely used in storage systems. Storage systems mainly process requests to read and write data (Input / Output, referred to as I / O requests) in the storage system in the form of First Input First Output (FIFO). Especially in database scenarios, storage systems do not have a method specifically for database requests (for example, database log write requests) to perform priority scheduling.

[0003] At the same time, in the scenario of mixed database load (for example, online transaction processing (OLTP) + online analytical processing (OLAP)), the concurrency of database redo log (redo log) write requests is not high for the storage system. On the contrary, the concurrency of other data read and write requests is relatively large. According to the traditional strict priority scheduling or weighted polling scheduling, the latency of database log write requests cannot be effectively reduced, resulting in low efficiency of database log writing. Summary of the invention

[0004] The embodiments of the present application provide a database request scheduling method, apparatus, and computer device to at least solve the technical problem in the related art that the database redo log writing delay is high, resulting in low database log writing efficiency.

[0005] According to one aspect of an embodiment of the present application, a database request scheduling method is provided, including: receiving a database request and determining the type of the database request; determining the priority of the database request according to the type of the database request, wherein a database priority volume write request has the highest priority among the database requests, and the priority volume includes at least: a redo log file; when the database load exceeds a preset threshold, scheduling the database request according to the priority.

[0006] Optionally, the priority of the database request is determined according to the type of the database request, including: obtaining the type of the database request; when the database request is a priority volume write request, transferring the database request to a high priority queue; when the database request is a read request, transferring the database request to a medium priority queue; when the database request is a non-priority volume write request, transferring the database request to a low priority queue, wherein the scheduling priorities of the high priority queue, the medium priority queue, and the low priority queue decrease in sequence.

[0007] Optionally, scheduling database requests according to priority includes: polling a first variable and a second variable, wherein the first variable is used to control the dequeueing of requests in a high priority queue, indicating the number of high priority queue scheduling intervals, and the second variable is used to control the dequeueing of requests in a low priority queue; when the value of the first variable is zero, it is determined to schedule the dequeueing of requests in the high priority queue, and when the value of the second variable is zero, it is determined to schedule the dequeueing of requests in the low priority queue.

[0008] Optionally, polling the first variable and the second variable includes: detecting a polling instruction; and when the polling instruction is detected, reducing the values ​​of the first variable and the second variable by one.

[0009] Optionally, the method further includes: obtaining initial values ​​of the first variable and the second variable, wherein the initial values ​​of the first variable and the second variable are both zero; when it is detected that the database load exceeds a preset threshold, the value of the second variable is increased by one.

[0010] Optionally, the method also includes: obtaining a preset sliding window; obtaining the load of all threads in the database except the main thread within the preset sliding window every preset period; and determining the average of the loads of all threads except the main thread within the preset sliding window as the database load.

[0011] Optionally, the method also includes: receiving database requests and determining the priority of the database requests; when the database requests received within a preset time period are all low priority requests or all high priority requests, scheduling the database requests according to a first-in-first-out strategy, wherein the scheduling priority of the low priority requests is lower than the scheduling priority of the high priority requests.

[0012] According to another aspect of an embodiment of the present application, a database request scheduling device is also provided, including: a receiving module, used to receive a database request and determine the type of the database request; a determining module, used to determine the priority of the database request according to the type of the database request, wherein the database priority volume write request has the highest priority among the database requests, and the priority volume at least includes: a redo log file; a scheduling module, used to schedule the database request according to the priority when the database load exceeds a preset threshold.

[0013] According to another aspect of the embodiment of the present application, a computer device is provided, including: a memory and a processor, wherein the memory is used to store program instructions; and the processor is connected to the memory and is used to execute the above-mentioned database request scheduling method.

[0014] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned database request scheduling method by running the computer program.

[0015] According to another aspect of the embodiments of the present application, a computer program product is provided, including computer instructions, which implement the above-mentioned database request scheduling method when executed by a processor.

[0016] In an embodiment of the present application, a database request is received and the type of the database request is determined; the priority of the database request is determined according to the type of the database request, wherein the database priority volume write request has the highest priority among the database requests, and the priority volume includes at least: a redo log file; when the database load exceeds a preset threshold, the database requests are prioritized according to the type according to the method of scheduling database requests according to priority, and then the database requests are scheduled according to priority according to the database load, wherein the database priority volume write request has the highest priority among the database requests, and the priority volume includes at least: a redo log file, and the database requests for the redo log are scheduled preferentially, thereby achieving the purpose of reducing the write latency of the redo log, thereby achieving the technical effect of improving the write efficiency of the database log, and further solving the technical problem in the related art that the write latency of the database redo log is high, resulting in low write efficiency of the database log. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0018] Figure 1 It is a hardware structure block diagram of a computer terminal for implementing a database request scheduling method according to an embodiment of the present application;

[0019] Figure 2 is a flowchart of a database request scheduling method according to an embodiment of the present application;

[0020] Figure 3 is a schematic diagram of an SPDK architecture according to an embodiment of the present application;

[0021] Figure 4is an I / O processing flow chart according to an embodiment of the present application;

[0022] Figure 5 is a thread distribution diagram according to an embodiment of the present application;

[0023] Figure 6 is a schematic diagram of a time sliding window according to an embodiment of the present application;

[0024] Figure 7 is a priority scheduling schematic diagram according to an embodiment of the present application;

[0025] Figure 8 is another priority scheduling schematic diagram according to an embodiment of the present application;

[0026] Fig. 9 It is a structural diagram of a database request scheduling device according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] The information collected in the embodiments of the present application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or reject automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered.

[0030] In order to better understand the embodiments of the present application, the technical terms involved in the embodiments of the present application are explained as follows:

[0031] Storage performance development kit (SPDK): An open source high-performance storage development framework for implementing high-performance, low-latency storage applications. This is an open source high-performance storage development framework designed for implementing low-latency, high-throughput storage applications, especially for NVMe (Non-Volatile Memory Express)-based solid-state storage devices.

[0032] I / O: Refers to requests to read and write data in a storage system.

[0033] in-flight I / O: I / O request being processed.

[0034] Online Transaction Processing (OLTP): Also known as transaction-oriented processing, it is mainly used to quickly respond to user interactive transaction processing, such as bank transactions, order processing, etc., and is characterized by frequent read and write operations and low latency.

[0035] Online Analytical Processing (OLAP) is a computer processing technology used to analyze and query large-scale data sets. It is mainly used for data analysis and query, especially for large-scale data sets. It is characterized by more reading and less writing and complex query.

[0036] First Input First Output (FIFO) is a scheduling strategy that processes requests in the order of "first come, first served".

[0037] Priority scheduling technology in the related art is rarely used in storage systems, and storage systems mainly process I / O requests in the form of FIFO (first-in-first-out). Especially in database scenarios, the storage system does not have a method specifically for priority scheduling of database requests (for example, database log write requests). The concurrency of database redo log (redo log) write requests is not high for the storage system. On the contrary, the concurrency of other data read and write requests is relatively large. According to traditional strict priority scheduling or weighted polling scheduling, the latency of database log write requests cannot be effectively reduced, resulting in low efficiency in writing database logs. Therefore, there is a technical problem in the related art that the database redo log write latency is high, resulting in low efficiency in writing database logs. In order to solve the problems existing in the related art, an embodiment of the present application provides a database request scheduling method, which can be run on Figure 1 In the computer terminal shown, the computer terminal is explained below.

[0038] The database request scheduling method embodiment provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a database request scheduling method. Figure 1 As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0039] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0040] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the database request scheduling method in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned database request scheduling method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0041] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0042] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0043] It should be noted that, in some optional embodiments, the above Figure 1 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0044] In the above operating environment, an embodiment of the present application provides an embodiment of a database request scheduling method. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] Figure 2 is a flowchart of a database request scheduling method according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0046] Step S202, receiving a database request and determining the type of the database request;

[0047] Step S204, determining the priority of the database request according to the type of the database request, wherein the database priority volume write request has the highest priority among the database requests, and the priority volume at least includes: a redo log file.

[0048] Step S206: When the database load exceeds a preset threshold, database requests are scheduled according to priority.

[0049] Through the above steps S202 to S206, the writing efficiency of the database log is improved; by adopting receiving a database request and determining the type of the database request; determining the priority of the database request according to the type of the database request, wherein the database priority volume write request has the highest priority in the database request, and the priority volume at least includes: redo log file; when the database load exceeds the preset threshold, the database request is prioritized according to the type according to the method of scheduling the database request according to the priority, and then the database request is scheduled according to the priority according to the database load, the database priority volume write request has the highest priority in the database request, and the priority volume at least includes: redo log file, and the database request of the redo log is scheduled first, thereby achieving the purpose of reducing the writing delay of the redo log, thereby achieving the technical effect of improving the writing efficiency of the database log, and then solving the technical problem of the high writing delay of the database redo log in the related technology, resulting in low writing efficiency of the database log. This method can ensure that the writing of key data such as the database redo log file is prioritized under high load conditions, effectively prevent data loss or inconsistency, and is particularly suitable for industries such as finance and telecommunications that have extremely high requirements for data security and consistency. The following is a detailed description.

[0050] In the technical solution provided in step S202 of the above database request scheduling method, high-performance all-flash distributed block storage software (e.g., zStorage) developed based on the SPDK framework can be used. The architecture design of SPDK is intended to maximize storage performance and reduce latency. By implementing I / O processing in user space, using polling mode, supporting RDMA and shared memory, etc., storage applications can bypass the bottleneck of the operating system kernel and interact directly with the hardware, thereby achieving higher data throughput and lower latency. Figure 3 As shown in the figure, a schematic diagram of SPDK architecture is presented, which uses the polling mechanism. Poller is a user-defined function, including timed Poller and non-timed Poller. Timed Poller is triggered once at a specified interval; non-timed Poller is triggered by polling all the time (triggering hundreds of thousands of times per second in zStorage). Poller is a key component in SPDK used to poll device drivers to check whether new I / O requests have arrived. The Poller mechanism is one of the keys to SPDK's high performance and low latency. It can quickly respond to I / O events at the millisecond or microsecond level. Core0 is the main thread, and Core1-N represents child threads, which are responsible for initializing the entire SPDK environment, setting Set configuration parameters and initialize some basic services. Reactor represents the scheduler (Core0-N corresponds to Reactor0-N respectively); IOdrive refers to the device driver in SPDK, which is responsible for communicating with specific storage devices, converting I / O requests from Poller into operations that the device can understand, and then executing these operations and returning the results to Poller. Each Reactor can schedule multiple Pollers to check the status of IOdrive through a polling mechanism. When an I / O request is detected, Poller passes the request to IOdrive, which is responsible for sending the request to the specific storage device and reporting the result to Poller after completing the I / O operation.

[0051] When determining the type of database request, you can categorize the request by volume type. zStorage provides an interface to set the priority of the volume and store the redo log files in the high-priority volume.

[0052] In the technical solution provided in step S204, there are multiple methods for determining the priority of a database request according to the type of database request. For example, the priority of a database request can be determined in the following ways: obtaining the type of database request; when the database request is a priority volume write request, transferring the database request to a high priority queue; when the database request is a read request, transferring the database request to a medium priority queue; when the database request is a non-priority volume write request, transferring the database request to a low priority queue, wherein the scheduling priorities of the high priority queue, the medium priority queue, and the low priority queue are successively reduced. This hierarchical request processing mechanism can effectively balance different types of database requests, avoid key operations being blocked by low priority requests, improve the overall response speed of the database and user experience, and is suitable for various Internet services that need to process a large number of read and write requests.

[0053] The following is a specific embodiment:

[0054] like Figure 4 As shown, an I / O processing flow chart is provided. Client RPC (remote procedure call) receives a database request (e.g., I / O request) sent by a module (e.g., Front End module) on the I / O path in zStorage used for processing I / O requests sent by the user host, and sends the I / O request to a module (e.g., Chunk Server) on the I / O path used to implement distributed consistency of data and disk placement. After receiving the I / O request, the Chunk Server uses the IO identification module to distinguish the type of I / O request. If the request is a write request from a high-priority volume (e.g., database redo log, i.e., the above-mentioned priority volume write request), the request will be directly placed in the high-priority queue to ensure that it can be processed quickly. The read request is placed in the medium-priority queue, and the non-priority volume write request enters the low-priority queue. At the same time, a priority scheduler (non-scheduled poller) is added to perform priority scheduling based on load perception. After the I / O request is scheduled out of the queue, it enters the distributed consistency algorithm PAFT module to achieve consistency and is finally persisted in the storage device.

[0055] In the above steps, when a database request is received and the priority of the database request is determined, if the database requests received within a preset time period (for example, 1 minute) are all low priority requests or all high priority requests, the database requests are scheduled according to a first-in-first-out strategy (for example, FIFO), wherein the scheduling priority of the low priority request is lower than the scheduling priority of the high priority request.

[0056] In the technical solution provided in step S206, scheduling database requests according to priority can be achieved in a variety of ways, for example: polling the first variable and the second variable, wherein the first variable is used to control the dequeueing of requests in the high priority queue, indicating the number of high priority queue scheduling intervals, and the second variable is used to control the dequeueing of requests in the low priority queue; when the value of the first variable is zero, the dequeueing of requests in the high priority queue is determined, and when the value of the second variable is zero, the dequeueing of requests in the low priority queue is determined. By dynamically adjusting the values ​​of the first variable and the second variable, the processing frequency of high priority and low priority requests can be flexibly controlled to ensure that the write operation of key data can be processed in a timely manner under high load conditions, while avoiding long waiting times for low priority requests, which is suitable for scenarios that require real-time data processing in a high concurrency environment, such as online trading systems.

[0057] It should be noted that in the above steps, the first variable and the second variable are polled in the following manner: detecting a polling instruction; when a polling instruction is detected, the values ​​of the first variable and the second variable are both reduced by one. This mechanism can ensure the fairness and efficiency of request scheduling, control the order of request processing by reducing the variable value, and avoid unnecessary waste of resources. It is particularly suitable for dynamic system environments that require real-time adjustment of resource allocation strategies.

[0058] It is also necessary to obtain the initial values ​​of the first variable and the second variable, wherein the initial values ​​of the first variable and the second variable are both zero; when it is detected that the database load exceeds the preset threshold, the value of the second variable increases by one; the priority of request processing can be dynamically adjusted according to the actual load of the database. When the system load is high, the scheduling interval of the low-priority queue is appropriately increased to reduce its occupation of system resources, thereby improving the processing speed of key operations, which is suitable for high-performance server clusters that process large-scale data. And obtain a preset sliding window; obtain the load of all threads in the database except the main thread in the preset sliding window every preset period; determine the average value of the load of all threads except the main thread in the preset sliding window as the database load. By using the sliding window to calculate the load average, the real-time load of the database can be more accurately reflected, avoiding the degradation of system performance caused by short-term high load, and is suitable for scenarios that require real-time monitoring and adjustment of loads, such as resource scheduling of cloud computing platforms.

[0059] In order to better explain the database request scheduling method proposed in the embodiment of the present application, the present application also proposes a specific embodiment:

[0060] When the system load is small (less I / O or system resource utilization does not exceed a preset threshold (e.g., 80%)), the system can respond to requests very quickly without priority scheduling. Therefore, a load sensing module is designed to sense the system load. When the database load exceeds the preset threshold, database requests are scheduled according to priority. The system load can be calculated in the following way:

[0061] 1) Calculation benchmark: The system load is reflected by in-flight I / O (hereinafter referred to as ifio). IO_LOAD represents the pressure of an I / O request (i.e., the above-mentioned database request) on the system. Since the I / O requests received by the system vary in size, they are estimated uniformly based on 4K. It is stipulated that when I / O size = 4KB, I / O_LOAD = 1. For I / Os of 512B to 8KB, they are regarded as a 4K I / O (the storage system will align small I / Os), i.e., IO_LOAD = 1. For I / Os of 8K to 128KB, IO_LOAD is converted according to IO_SIZE / 8K. For example, IO_LOAD of an I / O of 128KB is 128KB / 8K = 16, i.e., IO_LOAD of an I / O of 128KB is 16.

[0062] 2) Calculation of overall system load:

[0063] like Figure 5 As shown in the figure, a thread distribution diagram is provided. Core0 is the main thread and is not responsible for I / O scheduling. Except for core0, each thread on each core has a priority scheduler. The scheduler records the instantaneous load of the thread before each work. The main thread (Core0) counts the ifio of all working threads, that is, ifio = ifio-1 + ifio-2 + ifio-3 + ... + ifio-n. The main thread maintains a timer poller to periodically check and summarize the load of all working threads in the system at intervals of 1ms, which ensures the real-time and accuracy of the load information. Sliding window (such as Figure 6 The introduction of the window size is an effective mechanism for smooth load evaluation. The window size is set to 10, which means that when calculating the current system load, the main thread will consider the average number of in-flight I / Os (I / O requests being processed) at the past 10 time points (that is, within 10ms).

[0064] The sliding window maintained by the main thread can reflect the load changes of the system. When the system load is more serious than the last time, increase the number of scheduling intervals. As the number of scheduling intervals increases, the number of I / O requests scheduled out of the queue will gradually decrease, thereby gradually reducing the system load. When the system load is lower than the set threshold (the system is relatively idle), set the number of scheduling intervals to 0. The benefits of using the sliding window average are as follows: Enhanced stability: By taking the average value, it is possible to avoid making excessive scheduling decisions due to instantaneous high load at a certain point in time, ensuring the stability and reliability of the scheduling strategy. Noise filtering: In actual systems, the load may have short-term fluctuations, which may be caused by temporary behavior of the system and may not necessarily reflect the long-term load status of the system. The sliding window average can filter out these short-term noises and obtain a more accurate load trend. Balance between response delay and accuracy: The window size of 10 time points finds a balance between response speed and load accuracy. If the window size is set too small, although it can respond to load changes more quickly, it may cause frequent fluctuations in the scheduling strategy; if the window size is set too large, it may be insensitive to the real-time response of the system load. Through this mechanism, the main thread can more accurately perceive the system load status and adjust the scheduling strategy according to the real-time load situation. When the system load is high, the delay of high-priority I / O (such as database redo log write requests) can be reduced through priority scheduling. When the load is low, the I / O processing speed can be increased by reducing the number of scheduling intervals to ensure efficient operation of the system.

[0065] When the load reaches a preset threshold (for example, the current database request exceeds the maximum threshold that the system can handle or the system resource utilization is greater than 80%), the database request is scheduled according to priority: the first variable (for example, idle_high) and the second variable (for example, idle_low) are polled to record whether I / O can be scheduled.

[0066] idle_high is used to control the dequeueing of requests in the high priority queue and indicates the number of scheduling intervals of the high priority queue.

[0067] idle_low is used to control the dequeueing of requests in the low-priority queue and indicates the number of scheduling intervals of the low-priority queue.

[0068] The priority scheduler (non-scheduled poller) will decrement the above two variables (i.e., the first variable and the second variable) by 1 each time it polls. When any one of them becomes 0, it means that an I / O request can be scheduled out of the corresponding queue (i.e., when the value of the first variable is zero, the database request in the high priority queue is scheduled out of the queue, and when the value of the second variable is zero, the request in the low priority queue is scheduled out of the queue), and the corresponding variable is reset at the same time. If there is no I / O request in the queue, the variable will remain at 0 until there is an I / O request in the queue and it is dequeued.

[0069] For example, idle_high = 0, idle_low = 1 means that the priority scheduler schedules a high-priority I / O request (i.e., database request) out of the queue every time (zero interval), and schedules a low-priority I / O out of the queue once in an interval. The number of scheduling intervals changes with the system load. The specific strategy is as follows:

[0070] 1) Initially, the scheduling interval times of all queues are 0;

[0071] 2) When it is detected that the system load is greater than the preset threshold, the number of low priority queue scheduling intervals increases by one. When the system load is still greater than the preset threshold during the next detection, the number of low priority queue scheduling intervals continues to increase by one; that is, as long as the system load is found to be greater than the preset threshold during detection, the number of low priority queue scheduling intervals will increase by one on the original basis.

[0072] 3) At the same time, an upper limit is set for the scheduling interval times of all queues. After the upper limit is reached, the scheduling interval times will no longer increase.

[0073] 4) When a detection finds that the system load is less than the preset threshold, the scheduling interval number is set to 0.

[0074] like Figure 7 As shown in the figure, a priority scheduling diagram is provided, which describes the dequeue sequence of database requests when the requests in the two queues are not empty and the default values ​​of the scheduling interval parameters are idle_high = 0 and idle_low = 1. Yellow indicates high priority requests and gray indicates low priority requests. Control the dequeue of requests according to the scheduling strategy:

[0075] During the first scheduling, the high priority scheduling interval is 0, so a high priority request is scheduled out of the queue; the low priority queue scheduling interval is 1, the low priority request is not out of the queue, and the parameter idle_low is reduced by 1 and set to 0. The queued sequence is: high; during the second scheduling, the high priority scheduling interval is 0, so a high priority request is scheduled out of the queue; the low priority queue scheduling interval is 0, a low priority request is scheduled out of the queue, and the parameter idle_low is reset to the default value of 1. The queued sequence is: high -> high -> low; during the third scheduling, the high priority scheduling interval is 0, so a high priority request is scheduled out of the queue; the low priority queue scheduling interval is 1, the low priority request is not out of the queue, and the parameter idle_low is reduced by 1 and set to 0. The queued sequence is: high -> high -> low -> high; during the fourth scheduling, the high priority scheduling interval is 0, so a high priority request is scheduled out of the queue; the low priority queue scheduling interval is 0, a low priority request is scheduled out of the queue, and the parameter idle_low is reset to the default value of 1. The dequeued sequence is: high->high->low->high->high->high->low.

[0076] After the four dispatches are completed, the queue dequeue sequence is: high->high->low->high->high->high->low.

[0077] like Figure 8 As shown, another priority scheduling schematic diagram is provided, where idle_high=1, idle_low=2 indicates that the priority scheduler schedules a high priority request out of the queue once, and schedules a low priority request out of the queue twice. Figure 8 Indicates that when the default values ​​of the scheduling interval parameters are idle_high=1 and idle_low=2, the requested dequeue sequence is: In the first scheduling, the scheduling intervals of both queues are >0, so no request is dequeued in this scheduling. Both parameters are reduced by 1. The dequeue sequence is empty; In the second scheduling, the high priority scheduling interval is 0, so a high priority request is scheduled to be dequeued, and the parameter idle_high is reset to the default value of 1, the low priority queue scheduling interval is 1, and the low priority request is not dequeued, and the parameter idle_low is reduced by 1 and set to 0. The dequeued sequence is: high; In the third scheduling, the high priority scheduling interval is 1, and the high priority request is not dequeued, and the parameter idle_high is reduced by one and set to 0; The low priority queue scheduling interval is 0, and a low priority request is scheduled to be dequeued, and the parameter idle_low is reset to the default value of 2. The dequeued sequence is: high->low. Each subsequent scheduling is still carried out according to the above scheduling strategy. After the final six schedulings are completed, the requested dequeue sequence is: high->low->high->high->low.

[0078] The scheduling algorithms in the related technology are: First-In-First-Out (FIFO): insert data packets into the queue based on the order of arrival; the data packets at the head of the queue are dequeued first, which cannot guarantee fairness and does not meet the differentiated service. The FIFO scheduling mechanism cannot distinguish different types of I / O requests, which means that even when processing high-priority database log write requests, it may be blocked by low-priority read requests or other write requests, affecting the submission speed of database transactions. In the database mixed load (OLAP+OLTP) scenario, FIFO may not be able to effectively reduce the latency of redo log write requests, because the concurrency of redo log write requests is smaller than that of other data read and write requests, but it is extremely critical in transaction processing; Strict Priority (SP): There are multiple queues, each with a priority. In the enqueue phase, data packets are mapped to queues. If the queue is not full, they are allowed to enqueue. In the dequeue phase, if the high-priority queue is not empty, the high-priority queue element is selected first to dequeue. However, in the database mixed load scenario, non-high priority I / O requests may be blocked for a long time, resulting in uneven resource allocation, and some requests waiting for too long, affecting the response time of the overall system. When processing high priority requests, the SP scheduling algorithm does not consider the current load status of the system, which may result in the processing advantage of high priority requests not being obvious when the system load is light, and the restrictions on low priority requests being too strict when the load is heavy; Weighted Round Robin (WRR): Add weights to each queue to achieve differentiated scheduling. WRR achieves differentiated scheduling by setting weights for different queues, but in the database mixed load scenario, if the weights are fixed, it may not be able to effectively cope with the changes in scheduling requirements caused by changes in system load, especially when processing a large number of I / O requests. The complexity of dynamically calculating and adjusting the weights of each queue is high, which may affect the scheduling efficiency.

[0079] The database request scheduling method provided in the above embodiment of the present application solves the problem of high latency of log write requests in mixed load scenarios by giving priority to database redo log write requests, ensuring the rapid submission of database transactions and improving the overall performance of the database. Through the load sensing module, the system can dynamically adjust the scheduling strategy according to the real-time load situation. When the system load is light, FIFO processing is adopted to reduce unnecessary priority scheduling overhead; when the load is serious, priority scheduling is adopted to reduce the latency of key requests. By setting the number of scheduling intervals to control the processing frequency of non-high priority requests, even in the priority scheduling mode, low priority requests are avoided from being blocked for a long time, the fairness of scheduling is enhanced, and the priority scheduling algorithm is applied to the database scenario, and the I / O is prioritized in the storage system, and the database redo log write request is given priority, and the database redo log write latency is reduced in the scenario of large concurrent mixed load of the database.

[0080] Fig. 9 : is a structural diagram of a database request scheduling according to an embodiment of the present application, the device includes:

[0081] The receiving module 902 is used to receive a database request and determine the type of the database request.

[0082] The determination module 904 is used to determine the priority of the database request according to the type of the database request, wherein the database priority volume write request has the highest priority among the database requests, and the priority volume at least includes: a redo log file.

[0083] The determination module 904 is also used to determine the priority of the database request according to the type of the database request: obtain the type of the database request; when the database request is a priority volume write request, transfer the database request to the high priority queue; when the database request is a read request, transfer the database request to the medium priority queue; when the database request is a non-priority volume write request, transfer the database request to the low priority queue, wherein the scheduling priorities of the high priority queue, the medium priority queue and the low priority queue decrease in sequence.

[0084] The scheduling module 906 is used to schedule database requests according to priority when the database load exceeds a preset threshold.

[0085] The scheduling module 906 is also used to schedule database requests according to priority: polling the first variable and the second variable, wherein the first variable is used to control the dequeueing of requests in the high priority queue, indicating the number of high priority queue scheduling intervals, and the second variable is used to control the dequeueing of requests in the low priority queue; when the value of the first variable is zero, it is determined to schedule the dequeueing of requests in the high priority queue, and when the value of the second variable is zero, it is determined to schedule the dequeueing of requests in the low priority queue.

[0086] The database request scheduling device improves the writing efficiency of the database log; by receiving a database request and determining the type of the database request; determining the priority of the database request according to the type of the database request, wherein the database priority volume write request has the highest priority in the database request, and the priority volume at least includes: a redo log file; when the database load exceeds a preset threshold, the database request is prioritized according to the type according to the method of scheduling the database request according to the priority, and then the database request is scheduled according to the priority according to the database load, wherein the database priority volume write request has the highest priority in the database request, and the priority volume at least includes: a redo log file, and the database request of the redo log is scheduled preferentially, thereby achieving the purpose of reducing the writing delay of the redo log, thereby achieving the technical effect of improving the writing efficiency of the database log, and further solving the technical problem in the related art that the writing delay of the database redo log is high, resulting in low writing efficiency of the database log.

[0087] It should be noted that Figure 7 The database request scheduling device shown is used to execute Figure 2 The database request scheduling method shown, therefore the relevant explanations in the above database request scheduling method are also applicable to the database request scheduling device, and will not be repeated here.

[0088] The embodiment of the present application also provides a computer device, including: a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory and is used to execute the above database request scheduling method. For example, a database request is received and the type of the database request is determined; the priority of the database request is determined according to the type of the database request, wherein the database priority volume write request has the highest priority in the database request, and the priority volume at least includes: a redo log file; when the database load exceeds a preset threshold, the database request is scheduled according to the priority.

[0089] The embodiment of the present application also provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned database request scheduling method by running the computer program. For example, a database request is received, and the type of the database request is determined; the priority of the database request is determined according to the type of the database request, wherein the database priority volume write request has the highest priority in the database request, and the priority volume at least includes: redo log files; when the database load exceeds a preset threshold, the database request is scheduled according to the priority.

[0090] The embodiment of the present application also provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the database request scheduling method in the present application are implemented. For example, a database request is received, and the type of the database request is determined; the priority of the database request is determined according to the type of the database request, wherein the database priority volume write request has the highest priority in the database request, and the priority volume at least includes: a redo log file; when the database load exceeds a preset threshold, the database request is scheduled according to the priority.

[0091] The embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the database request scheduling method in the present application. For example, receiving a database request and determining the type of the database request; determining the priority of the database request according to the type of the database request, wherein the database priority volume write request has the highest priority among the database requests, and the priority volume at least includes: redo log files; when the database load exceeds a preset threshold, scheduling the database request according to the priority.

[0092] The embodiment of the present application also provides a computer program, which, when executed by a processor, implements the steps of the database request scheduling method in the present application. For example, receiving a database request and determining the type of the database request; determining the priority of the database request according to the type of the database request, wherein the database priority volume write request has the highest priority among the database requests, and the priority volume at least includes: redo log files; when the database load exceeds a preset threshold, scheduling the database request according to the priority.

[0093] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0094] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0095] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0096] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0097] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0098] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0099] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A database request scheduling method, characterized in that: include: receiving a database request and determining a type of the database request; Determining the priority of the database request according to the type of the database request, wherein the database priority volume write request has the highest priority among the database requests, and the priority volume at least includes: a redo log file; When the database load exceeds a preset threshold, the database request is scheduled according to the priority.

2. The method according to claim 1, characterized in that Determining the priority of the database request according to the type of the database request includes: Obtaining the type of the database request; In the case where the database request is a priority volume write request, transferring the database request to a high priority queue; When the database request is a read request, the database request is transferred to a medium priority queue; In the case that the database request is a non-priority volume write request, the database request is transmitted to a low priority queue, wherein the scheduling priorities of the high priority queue, the medium priority queue and the low priority queue are sequentially decreased.

3. The method according to claim 1, characterized in that Scheduling the database request according to the priority level includes: Polling a first variable and a second variable, wherein the first variable is used to control the dequeueing of requests in a high priority queue, indicating the number of scheduling intervals of the high priority queue, and the second variable is used to control the dequeueing of requests in a low priority queue; When the value of the first variable is zero, it is determined that the requests in the high priority queue are scheduled to be dequeued, and when the value of the second variable is zero, it is determined that the requests in the low priority queue are scheduled to be dequeued.

4. The method according to claim 3, characterized in that Polling the first variable and the second variable includes: Detect polling instructions; When the polling instruction is detected, the values ​​of the first variable and the second variable are both reduced by one.

5. The method according to claim 3, characterized in that: The method further comprises: Obtaining initial values ​​of the first variable and the second variable, wherein the initial values ​​of the first variable and the second variable are both zero; When it is detected that the database load exceeds a preset threshold, the value of the second variable is increased by one.

6. The method according to claim 1, characterized in that The method further comprises: Get the preset sliding window; Obtaining the load of all threads in the database except the main thread within the preset sliding window at every preset period; The average value of the loads of all threads except the main thread within the preset sliding window is determined as the database load.

7. The method according to claim 1, characterized in that The method further comprises: receiving a received database request and determining a priority of the database request; When all database requests received within a preset time period are low priority requests or all are high priority requests, the database requests are scheduled according to a first-in-first-out strategy, wherein the scheduling priority of the low priority requests is lower than the scheduling priority of the high priority requests.

8. A database request scheduling device, characterized in that: include: A receiving module, used for receiving a database request and determining the type of the database request; A determination module, configured to determine the priority of the database request according to the type of the database request, wherein the database priority volume write request has the highest priority among the database requests, and the priority volume at least includes: a redo log file; The scheduling module is used to schedule the database request according to the priority when the database load exceeds a preset threshold.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory is used to store program instructions; The processor is connected to the memory, and is used to execute the database request scheduling method described in any one of claims 1 to 7.

10. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the database request scheduling method described in any one of claims 1 to 7 is implemented.

Images