Epoll-based task scheduling method and system
Through the epoll-based task scheduling method, the problem that the prior art cannot efficiently handle a large number of concurrent events, and the task scheduling effect with higher performance and lower resource consumption is achieved.
Patent Information
- Application Number
- CN202411511470.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-28
- Publication Date
- 2025-02-11
AI Technical Summary
Existing task scheduling methods cannot handle a large number of concurrent events efficiently and inexpensively.
Using the epoll-based task scheduling method, by initializing the task scheduling entity, create IO read event list, IO write event list, timer queue, task queue and thread pool, and realize IO listening and timer functions through the epoll timeout mechanism. The callback function responds to IO events and timers, realizing IO events and timers to drive task execution.
When handling large number of concurrent connections, it provides higher performance and lower resource consumption, improving the response ability of task scheduling to events and the efficiency of event concurrent processing.
Smart Images
Figure CN119415256A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of computer information processing technology, and more specifically, to a task scheduling method and system based on epoll. Background Art
[0002] Task scheduling refers to arranging and managing the execution of various tasks in a program according to a certain strategy and sequence. Its significance is to improve program operation efficiency, optimize resource allocation, ensure priority execution of key tasks, rationally utilize system resources, improve the overall performance and response speed of the program, meet user needs, and improve user experience.
[0003] IO events and timers are the two most common task scheduling methods. Task scheduling based on IO events and timers can meet the functional requirements in most scenarios. Among them, the task scheduling method based on IO events mainly relies on external events (such as user input, network requests, etc.) to trigger program execution. The advantage of this method is that the program responds quickly and can handle various events in a timely manner. The task scheduling method based on timers executes the program at a predetermined time interval. The advantage of this method is that the program execution is stable and can ensure that the task is completed at a specific time point. However, neither of these two methods can handle a large number of concurrent events efficiently and with low consumption. Summary of the invention
[0004] In view of the above defects or improvement needs of the prior art, the present application provides a task scheduling method and system based on epoll, which aims to solve the technical problem that the current task scheduling method cannot handle a large number of concurrent events efficiently and with low consumption.
[0005] To achieve the above objectives, in a first aspect, the present application provides a task scheduling method based on epoll, the task scheduling method comprising the following steps:
[0006] Initialize the task scheduling entity, and create the IO read event list, IO write event list, timer queue, task queue and thread pool inside the entity;
[0007] According to actual needs, add IO read events to the IO read event list, add IO write events to the IO write event list, add timers to the timer queue, and add simple tasks to the task queue;
[0008] Start the main thread loop, implement IO monitoring and timer functions through the epoll timeout mechanism in the main thread, respond to IO events and timers through callback functions, thereby implementing IO event and timer driven task execution;
[0009] The execution of simple tasks is achieved through the thread pool mechanism.
[0010] Preferably, the IO monitoring and timer functions are implemented in the main thread through the timeout mechanism of epoll, and the IO events and timers are responded to through the callback function, which specifically includes the following steps:
[0011] S1. Get the value t of the minimum timer in the timer queue;
[0012] S2, monitor epoll events, and set the timeout to t;
[0013] S3. Execute the triggered IO read event and IO write event within the timeout period;
[0014] S4. When the timeout period is reached, the event of the smallest timer in the timer queue is triggered to execute; after the execution is completed, the smallest timer is deleted from the timer queue and the process returns to step S1.
[0015] Preferably, in step S3, the triggered IO read event and IO write event are executed, specifically:
[0016] If the IO read event is detected to be readable, the corresponding callback function is called to implement the execution of the IO read event; if the IO write event is detected to be writable, the corresponding callback function is called to implement the execution of the IO write event.
[0017] Preferably, in step S4, the event triggering execution of the minimum timer in the timer queue is specifically: executing the callback function corresponding to the minimum timer in the timer queue to realize the event execution of the minimum timer.
[0018] Preferably, the corresponding callback function is specifically: when an IO read event is added to an IO read event list, a corresponding callback function is added at the same time to execute the IO read event; when an IO write event is added to an IO write event list, a corresponding callback function is added at the same time to execute the IO write event; when a timer is added to a timer queue, a corresponding callback function is added at the same time to execute the timer event.
[0019] Preferably, the execution of simple tasks is achieved through a thread pool mechanism, specifically:
[0020] The threads in the thread pool wait for thread signals; after receiving the thread signals, the thread pool takes out simple tasks from the task queue in sequence and executes them, and deletes the simple tasks that have been executed until the task queue is empty.
[0021] In a second aspect, the present application provides a task scheduling system based on epoll, the task scheduling system comprising the following units:
[0022] The initialization unit is used to initialize the task scheduling entity and create an IO read event list, an IO write event list, a timer queue, a task queue and a thread pool within the entity;
[0023] An event adding unit, used to add an IO read event to an IO read event list, add an IO write event to an IO write event list, add a timer to a timer queue, and add a simple task to a task queue according to actual needs;
[0024] The main thread running unit is used to start the main thread loop, listen to IO read and write events in the main thread, implement the timer function through the epoll timeout mechanism, and respond to IO events and timers through callback functions, thereby realizing IO event and timer driven task execution;
[0025] The thread pool running unit is used to implement the execution of simple tasks through the thread pool mechanism.
[0026] In a third aspect, the present application provides an electronic device, comprising: a memory for storing programs; a processor for executing the programs stored in the memory, when the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.
[0027] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.
[0028] In a fifth aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.
[0029] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art:
[0030] (1) In the task scheduling scheme of the present application, the timeout mechanism of epoll is used for monitoring to realize IO monitoring and timer functions, and the callback function is used to respond to IO events and timers, thereby realizing IO event and timer-driven task execution based on epoll. When processing a large number of concurrent connections, the method of the present application provides higher performance and lower resource consumption.
[0031] (2) The task scheduling solution of the present application has a thread pool function, which can conveniently hand over time-consuming processing to the thread pool by adding simple tasks, thereby improving the task scheduling's ability to respond to events and the efficiency of concurrent event processing.
[0032] (3) The task scheduling scheme of this application can call basic API methods, which is convenient for adding and deleting IO monitoring, timers, simple tasks, etc. at any time. This technical scheme is highly versatile and has certain practical value. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 It is a structural diagram of epoll-based task scheduling provided in an embodiment of the present application.
[0034] Figure 2 This is a workflow diagram of the epoll-based task scheduling system provided in an embodiment of the present application.
[0035] Figure 3 This is a flowchart of read task creation and read task deletion provided by an embodiment of the present application.
[0036] Figure 4 This is a flowchart of writing task creation and writing task deletion provided by an embodiment of the present application.
[0037] Figure 5 This is a flowchart of scheduled task creation and scheduled task deletion provided in an embodiment of the present application.
[0038] Figure 6 This is a flowchart of task creation and task deletion provided in an embodiment of the present application.
[0039] Figure 7 It is a flow chart of the internal processing loop of the framework operation provided in the embodiment of the present application.
[0040] Figure 8 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0042] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0043] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple timers refer to two or more timers, etc.; multiple IO read events refer to two or more IO read events, etc.
[0044] First, the technical terms involved in the embodiments of the present application are introduced.
[0045] epoll: An I / O event notification mechanism in the Linux kernel, which is used to monitor whether I / O events such as readable, writable, or abnormal have occurred on multiple file descriptors.
[0046] Next, the technical solutions provided in the embodiments of the present application are introduced.
[0047] (1) Figure 1 As shown in the figure, the epoll-based task scheduling framework is composed of an IO read event list, an IO write event list, a timer queue, a task queue, and a thread pool.
[0048] (2) Figure 2 As shown in the figure, the task scheduling framework initializes the entity, then adds IO read events, IO write events, timers and simple tasks to the corresponding queues or lists as needed, and then starts listening to implement IO event and timer event driver execution. This is achieved through the following sub-steps:
[0049] (21) The task scheduling framework provides basic API methods to implement operations on the framework. The framework entity is initialized through thread_init, event monitoring and execution is started through thread_run, and the framework entity is released through thread_exit. The read event task entity for a certain IO is added or deleted through read_add or read_del, and the write event task entity for a certain IO is added or deleted through write_add or write_del. A timer task entity is added or deleted through timer_add or timer_del, and a simple task entity is added or deleted through task_add or task_del.
[0050] (22) Figure 3 As shown, the read_add operation first creates a read task entity, adds the entity to the epoll event, and then adds the entity to the read event list; the read_del operation deletes the read task entity from the epoll event and deletes the entity from the read event list.
[0051] (23) Figure 4As shown, the write_add operation first creates a write task entity, adds the entity to the epoll event, and then adds the entity to the write event list; the write_del operation deletes the write task entity from the epoll event and deletes the entity from the write event list.
[0052] (24) Figure 5 As shown, the timer_add operation first creates a timer task entity and then adds the entity to the timer queue; the timer_del operation deletes the timer task entity from the timer queue.
[0053] (25) Figure 6 As shown, the task_add operation first creates a simple task entity and then adds the entity to the task queue; the task_del operation deletes the simple task entity from the task queue.
[0054] (3) Figure 7 As shown in the figure, the thread_run operation starts the main thread loop, monitors IO read and write events in the main thread, implements the timer through the epoll timeout mechanism, and responds to IO events and timer events through callback functions. Specifically:
[0055] After the framework starts listening and executing, it sets the listening timeout according to the minimum value of the timer queue.
[0056] Within the timeout period, when a read-listening IO is readable, the callback function added during read_add will be called, thereby processing the IO read event; when a write-listening IO is writable, the callback function added during write_add will be called, thereby processing the IO write event.
[0057] When the timeout period is reached, the framework removes the member with the smallest time from the timer queue and executes the callback function of the member.
[0058] The framework resets the listen timeout to the minimum value of the timer queue.
[0059] (4) Figure 7 As shown, the sub-threads in the thread pool wait for thread signals. When the main thread adds a simple task, it sends a thread signal to notify the thread pool to start taking out simple tasks from the task queue and executing them. After the execution is completed, the simple task entity is deleted. When the threads in the thread pool have completed the execution of all simple tasks in the task queue, the threads all wait for new thread signals.
[0060] This application also implements a task scheduling system based on epoll.
[0061] It should be understood that the above-mentioned system is used to execute the methods in the above-mentioned embodiments. The implementation principles and technical effects of the corresponding units in the system are similar to those described in the above-mentioned methods. The working process of the system can refer to the corresponding process in the above-mentioned method and will not be repeated here.
[0062] Based on the method in the above embodiment, the embodiment of the present application provides an electronic device, such as Figure 8 As shown, the electronic device may include: a processor, a communications interface, a memory, and a communication bus, wherein the processor, the communications interface, and the memory communicate with each other via the communication bus. The processor may call the logic instructions in the memory to execute the method in the above embodiment.
[0063] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.
[0064] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.
[0065] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.
[0066] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.
[0067] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.
[0068] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a storage medium or transmitted through the storage medium. The computer instructions may be transmitted from a website site, a computer, a server or a data center to another website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.
[0069] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.
[0070] The above content is easily understood by those skilled in the art. The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the protection scope of the present application.
Claims
1. A task scheduling method based on epoll, characterized in that: The task scheduling method comprises the following steps: Initialize the task scheduling entity, and create the IO read event list, IO write event list, timer queue, task queue and thread pool inside the entity; According to actual needs, add IO read events to the IO read event list, add IO write events to the IO write event list, add timers to the timer queue, and add simple tasks to the task queue; Start the main thread loop, implement IO monitoring and timer functions through the epoll timeout mechanism in the main thread, respond to IO events and timers through callback functions, thereby implementing IO event and timer driven task execution; The execution of simple tasks is achieved through the thread pool mechanism.
2. The task scheduling method according to claim 1, characterized in that: In the main thread, the IO monitoring and timer functions are implemented through the epoll timeout mechanism, and the IO events and timers are responded to through the callback function. The specific steps include: S1. Get the value t of the minimum timer in the timer queue; S2, monitor epoll events, and set the timeout to t; S3. Execute the triggered IO read event and IO write event within the timeout period; S4. When the timeout period is reached, the event of the smallest timer in the timer queue is triggered to execute; after the execution is completed, the smallest timer is deleted from the timer queue and the process returns to step S1.
3. The task scheduling method according to claim 2, characterized in that: In step S3, the triggered IO read event and IO write event are executed, specifically: If the IO read event is detected to be readable, the corresponding callback function is called to implement the execution of the IO read event; if the IO write event is detected to be writable, the corresponding callback function is called to implement the execution of the IO write event.
4. The task scheduling method according to claim 2, characterized in that: In the step S4, the event triggering execution of the minimum timer in the timer queue is specifically: executing the callback function corresponding to the minimum timer in the timer queue to realize the event execution of the minimum timer.
5. The task scheduling method according to claim 3 or 4, characterized in that: The corresponding callback function is specifically: when an IO read event is added to the IO read event list, a corresponding callback function is added at the same time to execute the IO read event; when an IO write event is added to the IO write event list, a corresponding callback function is added at the same time to execute the IO write event; When a timer is added to the timer queue, a corresponding callback function is also added to execute the event of the timer.
6. The task scheduling method according to claim 1, characterized in that: The execution of simple tasks is achieved through the thread pool mechanism, specifically: The threads in the thread pool wait for thread signals; after receiving the thread signals, the thread pool takes out simple tasks from the task queue in sequence and executes them, and deletes the simple tasks that have been executed until the task queue is empty.
7. A task scheduling system based on epoll, characterized in that: The task scheduling system includes the following units: The initialization unit is used to initialize the task scheduling entity and create an IO read event list, an IO write event list, a timer queue, a task queue and a thread pool within the entity; An event adding unit, used to add an IO read event to an IO read event list, add an IO write event to an IO write event list, add a timer to a timer queue, and add a simple task to a task queue according to actual needs; The main thread running unit is used to start the main thread loop, listen to IO read and write events in the main thread, implement the timer function through the epoll timeout mechanism, and respond to IO events and timers through callback functions, thereby realizing IO event and timer driven task execution; The thread pool running unit is used to implement the execution of simple tasks through the thread pool mechanism.
8. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the program stored in the memory. When the program stored in the memory is executed, the processor is configured to execute the method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein when the computer program runs on a processor, the processor is enabled to execute the method according to any one of claims 1 to 6.
10. A computer program product, characterized in that When the computer program product runs on a processor, the processor is caused to execute the method according to any one of claims 1 to 6.