Operating system scheduling algorithm optimization method and device

Abstract

The invention relates to the related technical field of operating systems, and particularly provides an operating system scheduling algorithm optimization method and device, a development container mixed distribution management module is set to be used for identifying an operating container as an online task and an offline task, an online task waiting queue and an offline task waiting queue are developed, the online task waiting queue is identified as a high-priority queue, and the offline task waiting queue is identified as a high-priority queue; the offline task queue is identified as a low-priority queue; and optimizing a CPU scheduling algorithm, preferentially processing the high-priority waiting queue, and processing the tasks in the low-priority waiting queue after the task processing times of the high-priority waiting queue exceed system parameters of the number of online tasks processed at one time. Compared with the prior art, the method has the advantages that reasonable scheduling of the online tasks and the offline tasks in the mixed distribution scene can be improved, the overall utilization rate of resources is increased, and the cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field related to operating systems, and specifically provides a method and device for optimizing an operating system scheduling algorithm. Background Art

[0002] With the vigorous development of my country's digital economy, the cloud migration and cloud use plan continues to advance, the cloud computing industry is also developing rapidly, the technology and ecosystem are becoming more and more perfect, and cloud native technology has developed from the conceptual stage to the stage of widespread application. Different industries are increasingly accepting cloud native, and business needs are no longer limited to simple cloud migration. They are more concerned about how to make full use of cloud computing resources and maximize their benefits.

[0003] However, in actual applications, cloud-native applications face the challenge of uneven temporal and spatial distribution of business access volume. In order to more effectively address this challenge, cloud-native tasks can be divided into two categories: online tasks and offline tasks. Online tasks require real-time or near real-time responses and have extremely high requirements for service level agreements (SLAs), such as online shopping, search engines, social media, and real-time data stream analysis. These tasks are very sensitive to latency and require fast processing of user requests. Offline tasks do not require real-time processing and can tolerate certain delays, such as batch data processing, big data analysis, machine learning training, and regular report generation.

[0004] In cloud-native business scenarios, in order to more effectively utilize operating system resources, online tasks and offline tasks are often deployed together on the same infrastructure to improve resource utilization and reduce costs. This hybrid deployment method requires sophisticated task scheduling, resource allocation, and isolation management, especially the reasonable arrangement of offline tasks, so as to improve overall resource utilization efficiency and reduce operating costs.

[0005] How to optimize the resource scheduling of online and offline tasks in a cloud-native hybrid deployment environment, effectively process offline tasks, and make full use of operating system resources to reduce operating costs is an urgent problem to be solved by technicians in this field. Summary of the invention

[0006] The present invention aims at solving the above-mentioned deficiencies of the prior art and provides a highly practical method for optimizing an operating system scheduling algorithm.

[0007] A further technical task of the present invention is to provide a reasonably designed, safe and applicable operating system scheduling algorithm optimization device.

[0008] The technical solution adopted by the present invention to solve its technical problem is: An operating system scheduling algorithm optimization method is provided, wherein a container mixed distribution management module is set up to identify running containers as online tasks and offline tasks, and online task and offline task waiting queues are developed, wherein the online task waiting queue is identified as a high priority queue, and the offline task queue is identified as a low priority queue; Optimize the CPU scheduling algorithm and give priority to processing high-priority waiting queues. When the number of times the high-priority waiting queue tasks are processed exceeds the system parameter for the number of online tasks to be processed at one time, the tasks in the low-priority waiting queues will be processed.

[0009] Furthermore, the container hybrid management module runs applications based on container technology in a cloud-native scenario. When users run real-time services and non-real-time services together, a hybrid scenario is generated. Provides an API for identifying online and offline tasks. After the container is started, call the API to identify the container as an online or offline task.

[0010] Furthermore, the API interface parameters are parameter name, input parameter, output parameter, parameter type and parameter meaning; When the API receives a processing request, it modifies the cpu.qos_level field under the cgroup according to the containerid. If the container is identified as an offline task, the cpu.qos_level field is set to -1; if the container is identified as an online task, the cpu.qos_level field is set to 0.

[0011] Furthermore, the YAML configuration file is modified to configure whether the container is an online container or an offline container. When the container is started, the API for identifying online and offline tasks is called to complete the online and offline identification of the container.

[0012] Furthermore, in the task waiting queue, in the operating system kernel state, a new system parameter for the number of online tasks processed at one time is added to support the configuration of the number of online tasks processed at one time, which is used to identify the maximum number of online tasks processed at one time.

[0013] Furthermore, online task and offline task waiting queues are developed in the kernel state of the operating system. The online task waiting queue stores online tasks with a high priority; the offline task waiting queue stores offline tasks with a low priority. Modify the logic of the operating system running queue to obtain tasks. If the obtained task is an offline task, put it into the offline task waiting queue; if the obtained task is an online task, put it into the online task waiting queue.

[0014] Furthermore, the CPU scheduling algorithm is optimized and the logic of getting tasks in the running queue is modified as follows: If the hybrid scenario feature is enabled, tasks are obtained from the online task waiting queue. After the online tasks are obtained, the CPU processes the tasks. When the number of online waiting queue task processing times exceeds the system parameter for the number of online tasks processed at one time, the tasks in the offline task waiting queue will be processed.

[0015] An operating system scheduling algorithm optimization device includes: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to execute an operating system scheduling algorithm optimization method.

[0016] Compared with the prior art, the method and device for optimizing the scheduling algorithm of an operating system of the present invention have the following outstanding beneficial effects: The present invention provides an operating system scheduling algorithm optimization method that supports cloud native hybrid scenarios. Through this method, it supports identifying cloud native container tasks as online or offline, supports online tasks and offline tasks waiting in scheduling queues, supports online task priority scheduling, and supports offline task scheduling suppression, thereby improving the reasonable scheduling of online tasks and offline tasks in hybrid scenarios, improving the overall utilization of resources, and reducing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Attached Figure 1 The present invention is a flowchart of an operating system scheduling algorithm optimization method. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] A best embodiment is given below: like Figure 1As shown, in this embodiment, an operating system scheduling algorithm optimization method is provided, a container mixed management module is developed to identify running containers as online tasks and offline tasks, and online task and offline task waiting queues are developed, the online task waiting queue is identified as a high priority queue, and the offline task queue is identified as a low priority queue; Optimize the CPU scheduling algorithm, give priority to processing high-priority waiting queues, and process tasks in low-priority waiting queues when the number of high-priority waiting queue tasks processed exceeds the system parameter for the number of online tasks processed at one time. This ensures the timely processing of online tasks and the processing of offline tasks.

[0021] The container hybrid management module runs applications based on container technology in cloud native scenarios. When users run real-time and non-real-time businesses together, a hybrid scenario is created.

[0022] In the mixed deployment scenario, a container mixed deployment management module is developed in user mode to support the function of identifying online and offline tasks.

[0023] Among them, the container mixed distribution management module structure is: Resource management: Containers apply for resources through Request and Limit. The colocation resource model achieves optimal resource allocation through dynamic resource overselling and resource isolation technology.

[0024] Scheduler: The scheduler is responsible for allocating containers to appropriate nodes for running. In a hybrid scenario, the scheduler will comprehensively consider the characteristics of the tasks running in the container, such as online services mainly use CPU resources, while offline services mainly use disk and I / O resources.

[0025] Service quality management: In a hybrid scenario, the service level of the online business container can be set to Guaranteed to guarantee its resource usage, while the service level of the offline business container can be set to Best-Effort. When resources are tight, the Best-Effort container will be killed first to ensure the stability of the online business.

[0026] Identify online and offline tasks: Supports the function of identifying online and offline tasks, and provides an API for identifying online and offline tasks. After the container is started, call the API to identify the container as an online or offline task.

[0027] How containers operate in a hybrid scenario: Container startup: A container is described by an image, which is a lightweight, executable, independent software package. When starting a container, specify whether the current container is an online or offline task.

[0028] Resource isolation: When running containers, it supports isolation and resource allocation control between containers. Each container has its own independent file system, network interface, and process space, and does not affect each other.

[0029] Dynamic resource allocation: In a hybrid scenario, container resource allocation is dynamic. When the online business peaks, the system automatically adjusts resource allocation and gives offline business resources to online business.

[0030] Service discovery and communication: Service discovery and communication between containers are achieved through the service discovery mechanism. In a mixed deployment scenario, different types of containers can discover and communicate with each other and work together.

[0031] Provides an API for identifying online and offline tasks. After the container is started, call the API to identify the container as an online or offline task.

[0032] API interface parameters include parameter name, input parameter, output parameter, parameter type and parameter meaning; When the parameter name is containerid, the input / output parameter is IN, the parameter type is int, and the parameter means the container ID; When the parameter name is offline, the input / output parameter is IN, the parameter type is int, and the parameter means online task and offline task. The online task is represented by "0" and the offline task is represented by "-1". When the parameter name is ret, the input / output parameter is OUT, the parameter type is int, and the parameter means whether the return is successful.

[0033] When the API receives a processing request, it modifies the cpu.qos_level field under the cgroup according to the containerid. If the container is identified as an offline task, the cpu.qos_level field is set to -1; if the container is identified as an online task, the cpu.qos_level field is set to 0. After that, the API returns the processing request.

[0034] Specific container process online and offline configuration examples: / / Specify the container as offline echo -1> / sys / fs / cgroup / cpu / docker / container number / cpu.qos_level / / Specify the container as online echo 0> / sys / fs / cgroup / cpu / docker / container number / cpu.qos_level By modifying the YAML configuration file, you can configure whether the container is an online container or an offline container. When the container is started, you can call the API that identifies the online and offline tasks to complete the online and offline identification of the container.

[0035] The YAML format example is: apiVersion: v1 kind: Pod metadata: name: nginx annotations: preemptable: "false" # true means the service is offline, false means the service is online, the default value is false spec: containers: - name: nginx image: nginx resources: limits: memory: "200Mi" cpu: "1" requests: memory: "200Mi" cpu: "1" In the task waiting queue, modify the operating system kernel to support mixed deployment scenario features. By enabling the QOS feature, support online and offline task queues and support mixed deployment scenarios for container processes.

[0036] Operating system kernel code sample: / / Operating system kconfig configuration, support mixed distribution CONFIG_QOS_SCHED=y CONFIG_QOS_SCHED_PRIO_LB=y / / Add offline task queue to the running queue structrq { … structlist_headcfs_offline_tasks; / / Offline task queue structlist_headcfs_tasks; / / Online task queue … In the operating system kernel state, a new system parameter for the number of online tasks processed at one time is added to support the configuration of the number of online tasks processed at one time, which is used to identify the maximum number of online tasks that can be processed at one time.

[0037] In the kernel state of the operating system, online task and offline task waiting queues are developed. The online task waiting queue stores online tasks with a high priority; the offline task waiting queue stores offline tasks with a low priority.

[0038] Modify the logic of the operating system running queue to obtain tasks. If the obtained task is an offline task, put it into the offline task waiting queue; if the obtained task is an online task, put it into the online task waiting queue.

[0039] Code sample: / / Assign online and offline scheduling tasks static __always_inline voidadjust_rq_cfs_tasks(void (*list_op)(structlist_head *, structlist_head *), structrq *rq, structsched_entity *se) structtask_group *tg = task_group(task_of(se)); if (qos_sched_enabled()&&sysctl_sched_prio_load_balance_enabled&& is_offline_level(tg->qos_level)) / / When the task is offline, add the task to the offline queue; otherwise, add it to the online queue (*list_op)(&se->group_node,&rq->cfs_offline_tasks); Else (*list_op)(&se->group_node,&rq->cfs_tasks); / / Task queue processing static voidaccount_entity_enqueue(structcfs_rq *cfs_rq, structsched_entity *se) / / Handle online and offline tasks when starting QOS configuration #ifdef CONFIG_QOS_SCHED_PRIO_LB / / Assign online and offline scheduling tasks adjust_rq_cfs_tasks(list_add, rq, se); #else list_add(&se->group_node,&rq->cfs_tasks); #endif Optimize the CPU scheduling algorithm and modify the logic of getting tasks in the running queue to: If the hybrid scenario feature is enabled, tasks are obtained from the online task waiting queue. After the online tasks are obtained, the CPU processes the tasks.

[0040] When the number of online waiting queue task processing times exceeds the system parameter for the number of online tasks processed at one time, the tasks in the offline task waiting queue will be processed.

[0041] This ensures timely processing of online tasks while also processing offline tasks.

[0042] Code example: / / Processing task logic static int processing task (structlb_env *env) / / Prioritize the tasks in the online queue structlist_head *tasks =&env->src_rq->cfs_tasks; again: while (!list_empty(tasks)) { env->loop++; if (qos_sched_enabled()&&sysctl_sched_prio_load_balance_enabled&& env->imbalance>0) { if (env->loop>env->loop_max) env->loop = 0; loop++; if (loop == 1) { / / If the online task queue task has been completed, process the offline task tasks =&env->src_rq->cfs_offline_tasks; goto again.

[0043] Based on the above method, an operating system scheduling algorithm optimization device in this embodiment includes: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to execute an operating system scheduling algorithm optimization method.

[0044] The above-mentioned specific implementations are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the above-mentioned specific implementations. Any technical solutions that conform to the above-mentioned specific implementations of the present invention and any appropriate changes or substitutions made by ordinary technicians in the relevant technical field shall fall within the patent protection scope of the present invention.

[0045] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for optimizing an operating system scheduling algorithm, characterized in that: Set up a mixed deployment management module for developing containers to identify running containers as online tasks and offline tasks, develop online task and offline task waiting queues, identify the online task waiting queue as a high priority queue, and the offline task queue as a low priority queue; Optimize the CPU scheduling algorithm and give priority to processing high-priority waiting queues. When the number of times the high-priority waiting queue tasks are processed exceeds the system parameter for the number of online tasks to be processed at one time, the tasks in the low-priority waiting queues will be processed.

2. The method for optimizing an operating system scheduling algorithm according to claim 1, characterized in that: The container hybrid management module runs applications based on container technology in a cloud-native scenario. When users run real-time services and non-real-time services together, a hybrid scenario is generated. Provides an API for identifying online and offline tasks. After the container is started, call the API to identify the container as an online or offline task.

3. The method for optimizing an operating system scheduling algorithm according to claim 2, characterized in that: The API interface parameters are parameter name, input parameter, output parameter, parameter type and parameter meaning; When the API receives a processing request, it modifies the cpu.qos_level field under the cgroup according to the containerid. If the container is identified as an offline task, the cpu.qos_level field is set to -1; if the container is identified as an online task, the cpu.qos_level field is set to 0.

4. The method for optimizing an operating system scheduling algorithm according to claim 3, characterized in that: Modify the YAML configuration file to configure whether the container is an online container or an offline container. When the container is started, call the API for identifying online and offline tasks to complete the online and offline identification of the container.

5. The method for optimizing an operating system scheduling algorithm according to claim 4, characterized in that: In the task waiting queue, in the operating system kernel state, a new system parameter for the number of online tasks processed at one time is added to support the configuration of the number of online tasks processed at one time, which is used to identify the maximum number of online tasks that can be processed at one time.

6. The method for optimizing an operating system scheduling algorithm according to claim 5, characterized in that: Develop online task and offline task waiting queues in the operating system kernel state. The online task waiting queue stores online tasks with high priority; the offline task waiting queue stores offline tasks with low priority. Modify the logic of the operating system running queue to obtain tasks. If the obtained task is an offline task, put it into the offline task waiting queue; if the obtained task is an online task, put it into the online task waiting queue.

7. The method for optimizing an operating system scheduling algorithm according to claim 6, characterized in that: Optimize the CPU scheduling algorithm and modify the logic of getting tasks in the running queue to: If the hybrid scenario feature is enabled, tasks are obtained from the online task waiting queue. After the online tasks are obtained, the CPU processes the tasks. When the number of online waiting queue task processing times exceeds the system parameter for the number of online tasks processed at one time, the tasks in the offline task waiting queue will be processed.

8. An operating system scheduling algorithm optimization device, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 7.

Images