Self-adaptive process scheduling method and system based on real-time state perception

Abstract

The invention discloses a self-adaptive process scheduling method and system based on real-time state perception, and the method comprises the steps: firstly carrying out the low-overhead real-time monitoring of a system operation state through an extended Berkley data packet filter technology, and collecting multi-dimensional original data; secondly, identifying a current system load mode by using a pre-trained gradient boosting decision tree model, and performing stable prediction on a load mode of a next decision period in combination with a historical window mechanism based on time weighted voting so as to smooth instantaneous fluctuation; then, querying a'load mode-target scheduler 'mapping table which is constructed offline in advance according to a prediction result to determine a target scheduling strategy; and finally, dynamically and safely switching to a target scheduling strategy through the extensible scheduler class framework, and realizing self-adaptive adjustment of system scheduling. According to the method, macroscopic strategy selection and microcosmic scheduling decision are decoupled, high real-time inference overhead is avoided, safety and stability of the system are guaranteed by utilizing a kernel native mechanism, and the system operation efficiency under diversified loads is remarkably improved while low performance overhead, high safety and easy deployment are kept.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer operating system, and in particular, relates to a real-time state-aware adaptive process scheduling method and system. BACKGROUND

[0002] Process scheduling is one of the core functions of modern operating systems, which is responsible for managing and allocating central processor computing resources, and its efficiency directly affects the system throughput, task response time and ultimately the user experience. Traditional operating system schedulers, such as the Completely Fair Scheduler, usually use a fixed set of algorithms based on universal heuristic rules. Such schedulers perform well in general computing scenarios, but as hardware architectures (such as heterogeneous central processors with large cores) and software applications (such as interactive applications, batch processing, and stream computing) become increasingly complex and diverse, their limitations become increasingly apparent. Static scheduling strategies are difficult to adapt to the dynamic workload patterns of application programs at runtime, resulting in low resource utilization, high response delay or poor user experience in specific scenarios.

[0003] To address this challenge, research in the field can be divided into two technical routes. The first is to design a dedicated scheduling strategy for a specific application scenario. Although such specialized schedulers have superior performance, they are costly to develop and their optimization effect is limited to specific scenarios, making it difficult to adapt to the mixed load environment commonly seen in modern computing. The second is to introduce artificial intelligence technology to assist in scheduling decisions. Although such intelligent schedulers have some adaptive ability, they usually face two major challenges: first, the overhead of model reasoning is large, which may offset the optimization effect of artificial intelligence technology in the kernel scheduling scenario where performance is extremely sensitive, causing performance degradation; second, directly modifying the kernel scheduling logic introduces significant system complexity and stability risks, making deployment difficult.

[0004] Therefore, there is an urgent need for an intelligent scheduling solution with low overhead, high adaptability, easy deployment and effective user experience improvement to solve the adaptability problem of traditional static schedulers in dynamic and complex load environments. SUMMARY

[0005] The present application overcomes the above-mentioned shortcomings of the prior art and provides a real-time state-aware adaptive scheduling method, system and computer readable storage medium, aiming to solve the problems of insufficient adaptability, high performance overhead and difficult deployment of existing operating system schedulers when facing dynamic and heterogeneous complex workloads.

[0006] To achieve the above-mentioned purpose, the first aspect of the present application provides a real-time state-aware adaptive process scheduling method, comprising the following steps:

[0007] S1: Real-time collection of system state data. Real-time collection of information reflecting the operating system runtime load condition is achieved through extended Berkeley packet filter technology.

[0008] S2: Identify the current system load pattern. The system state raw data is preprocessed and feature engineered to generate a structured feature vector, and a pre-trained gradient boosting decision tree classification model is used to analyze the feature vector to obtain the current load pattern label and store it in the pattern label pool.

[0009] S3: Predict the system load pattern of the next decision cycle. According to the historical pattern label data in the pattern label pool, weighted voting is performed using a fixed-size historical window and exponential decay weight to predict the load pattern of the next system operation.

[0010] S4: Select the target scheduling strategy. According to the predicted load pattern, query the preset load pattern-target scheduling strategy mapping table to retrieve and determine the corresponding target scheduling strategy.

[0011] S5: Perform scheduling strategy dynamic switching. Compare the target scheduling strategy with the current active scheduling strategy of the system, if they are not consistent, then through the extensible scheduler class technology, dynamically unload the current scheduling strategy and load and apply the target scheduling strategy during the operation system runtime, thereby realizing real-time adaptive adjustment of the scheduling strategy.

[0012] In step S1, real-time collection of system state uses extended Berkeley packet filter technology, and the bpftrace toolset is used to write and load the extended Berkeley packet filter program for low-overhead, non-intrusive data collection in the operating system kernel space. The specific steps are as follows:

[0013] S1-1: First, implement state monitoring in the kernel space through extended Berkeley packet filter technology. This process involves writing, deploying, and executing the extended Berkeley packet filter program, and the collected system operation indicators cover six dimensions: central processor, memory, disk, process, scheduling, and network, including the indicators shown in Table 1:

[0014] Table 1 System state data indicator table

[0015]

[0016]

[0017] To collect each indicator as shown in Table 1, the user state control program needs to load the pre-written extended Berkeley packet filter program into the kernel and dynamically attach it to a series of preset event points in the kernel, including:

[0018] Tracepoints: sched:sched_switch for monitoring task switch, sched:sched_wakeup for monitoring task wakeup, sched:sched_migrate_task for monitoring task migration; block:block_rq_issue and block:block_rq_complete for monitoring disk I / O requests; sock:inet_sock_set_state, tcp:tcp_destroy_sock, tcp:tcp_retransmit_skb for monitoring TCP connection status and events; and irq:softirq_entry, irq:softirq_exit, irq:irq_handler_entry / exit for monitoring interrupt handling.

[0019] Kernel probes: __mutex_lock_slowpath, mutex_lock, mutex_unlock for monitoring lock contention, holding and blocking; tcp_sendmsg, tcp_recvmsg, netif_receive_skb, ip_rcv for monitoring key nodes in network packet sending and receiving procedures; and input_event_to_user for monitoring user input events.

[0020] When any of the attached event points is triggered by the kernel execution flow, the corresponding extended Berkeley packet filter program is executed in the kernel context. To reduce the impact of data collection on system performance, a part of data statistics and aggregation operations are directly completed in the kernel mode. This process utilizes the extended Berkeley packet filter map provided by the extended Berkeley packet filter as an efficient data exchange structure between the kernel mode and the user mode. The aggregation operations executed in the kernel mode specifically include event counting, value distribution statistics (such as histogram) and calculation of basic statistical quantities (maximum value, minimum value, average value). In this way, the original, high-frequency event flow is preprocessed into a structured, low-frequency statistical data flow, which is then read by the user-mode monitoring program from the extended Berkeley packet filter map.

[0021] Secondly, the extended Berkeley packet filter probe is not easy to directly collect or process information with high cost. The user space monitoring program reads and parses the information in the virtual file system provided by the operating system (such as / proc and / sys file systems) to obtain supplementary system state data. The specific collection contents include CPU utilization percentage calculation, global memory state acquisition, disk queue length reading, and process-related state statistics. In particular, the monitoring program can also be configured to identify processes with specific attributes, such as processes using graphics processors or processes currently possessing user input focus, by executing external commands or scripts, and perform specialized statistics on the resource usage of these processes.

[0022] S1-2: Kernel data collection and aggregation. When the kernel execution flow triggers any of the above event points, the corresponding extended Berkeley packet filter program is safely executed in the kernel state. The program directly accesses the kernel data structure related to the event to obtain the original performance indicators. To reduce data processing and transmission overhead, the extended Berkeley packet filter program uses extended Berkeley packet filter maps in the kernel for real-time aggregation. The extended Berkeley packet filter map types and aggregation operations used include: using hash map BPF_MAP_TYPE_HASH to count the frequency of occurrence of various events; using histogram map BPF_MAP_TYPE_HISTOGRAM to record the logarithmic distribution of key event delays; for delay data, directly calculate its maximum value, minimum value, cumulative value and count value in the kernel for calculating the average value in the user state. The aggregated structured data is read by the user state monitoring program through the extended Berkeley packet filter map, thereby completing the data collection.

[0023] Among them, in step S2, the collected data is converted into a load pattern label that can be used for decision making. The specific steps of conversion are as follows:

[0024] S2-1: Data preprocessing and feature engineering. The user state monitoring program reads the aggregated data in the extended Berkeley packet filter map at a fixed time interval to form a time window. For the data of each time window, feature engineering calculation is performed to calculate the key quantile value (50th percentile, 90th percentile, 95th percentile) and Gini coefficient.

[0025] S2-2: Feature vector construction and classification. All the statistics calculated in the previous step, together with the aggregated data read directly from the extended Berkeley packet filter map, are combined to construct a structured feature vector with dimension 150. The feature vector is input to a pre-trained gradient boosting decision tree classification model, and the hyperparameters are tuned by Bayesian optimization. The output of the model is the load pattern label for the current time window. The label is stored in a first-in-first-out queue, i.e., the pattern label pool, along with the current timestamp.

[0026] In step S3, when a new load pattern label identification result is generated in step S2, step S3 enters a new decision cycle, and the system load pattern in the next decision cycle is predicted using a history window mechanism based on time-weighted voting. Due to the minimum time interval restriction that allows switching in the scheduling strategy switching stage, the impact of the classification model's false prediction will be further amplified. To solve this problem, this step smoothes the instantaneous identification result by historical data to improve the stability of the prediction. The recording length of the historical data is defined as N, and at time τ i , the identified load pattern is defined as L i , the identification result (L i , τ i ) is defined as r i , the time weight of the result r i is defined as W(r i ), t cur is the current timestamp, and the time decay factor is defined as α. The specific steps of prediction are as follows:

[0027] S3-1: Maintain the history record window. The pattern label pool is used as a fixed time length history record window. When a new label enters, if the window is full, the oldest label is removed.

[0028] S3-2: Calculate the time weight. Calculate the time weight W(r i ) for each identification result (L i , τ i ) in the history record window, and the calculation formula is:

[0029]

[0030] S3-3: Weighted voting. Group all records in the history window according to their load patterns L i . For each group, accumulate the weights W(r i ) of all records in the group to get the weighted total score of the load pattern. Finally, select the load pattern with the highest weighted total score as the final prediction of the system's running state in the next stage.

[0031] In step S4, the target scheduling strategy is selected. Since the devices to which the method is applied can have different hardware resources and system environments, the same load mode can have different matching target scheduling strategies on different devices. Therefore, a "load mode-scheduler" mapping table needs to be constructed offline before running the method. In runtime, this step will select the target scheduling strategy according to the pre-constructed mapping table. The implementation process is as follows:

[0032] S4-1: In an offline state, first define a scheduler library, predefine possible load modes, and for each scheduler in the scheduler library, run performance tests under all pre-defined load modes. According to the performance indicators (such as game frame rate, compilation time) collected by the tests, calculate the comprehensive performance score of each scheduler under the load mode using pre-set evaluation criteria. The evaluation criteria include absolute numerical score and stability score. Average the obtained scores to get the comprehensive performance score.

[0033] The absolute numerical score is calculated by an improved Sigmoid function, with the formula being:

[0034]

[0035] where S abs (x) is the absolute numerical score, x is the performance indicator value, S max is the pre-set full score, θ is the performance threshold, f is the scaling factor, k v is the value coefficient, and γ v is the value index. The ± in the formula is used to adapt to positive and negative performance indicators.

[0036] The stability score is calculated according to the variance of the performance indicator value, with the formula being:

[0037]

[0038] where S stability is the stability score, S max is the pre-set full score, σ is the variance of the performance indicator value, k s is the stability coefficient, and γ s is the stability index.

[0039] S4-2: Based on the calculated comprehensive performance score, find the scheduler with the highest score for each load mode, and store this correspondence as a "load mode-scheduler" mapping table.

[0040] S4-3: Runtime query. At runtime, the scheduling agent uses the predicted load pattern determined by S3 (which corresponds exactly to the predefined load pattern set in S4-1) as a key to query the mapping table constructed in S4-2, thereby obtaining the corresponding target scheduling strategy.

[0041] In step S5, the scheduling strategy dynamic switching is performed. When the target scheduling strategy determined by S4 is inconsistent with the currently active scheduling strategy of the system, the execution module of the scheduling agent issues an instruction to the kernel to safely offload the currently active scheduler extension Berkeley packet filter program through the interface of the extensible scheduler class, and load, verify and attach a new target scheduler extension Berkeley packet filter program.

[0042] To prevent the scheduler from frequently switching due to rapid oscillation of the load between different patterns, after each successful strategy switching, the system will enter a cooling period. During the cooling period, even if the decision module outputs a new target strategy, the execution module will ignore the switching request.

[0043] The second aspect of the present application provides an adaptive scheduling system based on real-time state perception, which comprises a memory and one or more processors, the memory stores executable code, and the one or more processors execute the executable code to perform the method of any one of the above aspects.

[0044] The third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to perform the method of any one of the above aspects.

[0045] The adaptive process scheduling method and system based on real-time state perception of the present application aim to solve the problems of poor adaptability, high overhead and difficult deployment of the traditional operating system scheduler when facing dynamic and complex workloads. The method first performs low-overhead real-time monitoring of the system running state through the extended Berkeley packet filter technology, and collects multi-dimensional raw data; secondly, it uses a pre-trained gradient boosting decision tree model to identify the current system load pattern, and combines a history window mechanism based on time-weighted voting to stably predict the load pattern of the next decision period to smooth transient fluctuations; then, according to the prediction result, it queries the pre-offline constructed "load pattern-target scheduler" mapping table to determine the target scheduling strategy; finally, it dynamically and safely switches to the target scheduling strategy through the extensible scheduler class framework to realize the adaptive adjustment of the system scheduling. The present application decouples the macro strategy selection and the micro scheduling decision, avoids the high real-time inference overhead, and uses the native kernel mechanism to ensure the safety and stability of the system, while maintaining low performance overhead, high security and easy deployment, and significantly improving the system running efficiency under diversified loads.

[0046] The advantages of the present application are: by introducing a real-time load-aware module based on an extended Berkeley packet filter, low-overhead, kernel-level monitoring of the macroscopic running state of the system is achieved. Combined with a history window mechanism based on time-weighted voting, the present application can accurately and stably identify and predict complex and variable workload patterns, effectively smoothing transient fluctuations and noise interference. On this basis, using the kernel's native extensible scheduler class framework, the system is dynamically switched to the currently optimal scheduling strategy. The above design decouples the macroscopic strategy selection and the microscopic single scheduling decision, avoiding the high real-time inference overhead of traditional intelligent schedulers, and using the extended Berkeley packet filter verifier and automatic rollback mechanism to ensure the safety and stability of the system. This design ensures that the present application can achieve high adaptability and high performance under different hardware environments and mixed workloads, while maintaining low performance overhead, high safety and easy deployment, significantly improving the overall running efficiency of the system. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a flowchart of the method of the present application.

[0048] Figure 2 is a detailed flowchart of the predicted system load pattern according to the present application.

[0049] Figure 3 is a confusion matrix diagram of the load pattern classification model according to the present application on the test set.

[0050] Figure 4 is a confusion matrix diagram of the load pattern classification model according to the present application on the unseen device.

[0051] Figure 5 is a line chart of the time-weighted voting algorithm for the correct prediction time improvement effect.

[0052] Figure 6 is a detailed flowchart of the offline construction of the "load pattern-target scheduler" mapping table according to the present application.

[0053] Figure 7 is a detailed flowchart of the execution of the dynamic switching of the scheduling strategy.

[0054] Figure 8 is a system schematic diagram of the present application. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application will be described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments.

[0056] Embodiment 1

[0057] The embodiment provides a specific implementation of an adaptive scheduling method based on real-time state perception. The overall framework of the method is shown in Figure 1 The core idea is to build a closed-loop "perception-decision-execution" system, which continuously monitors the operating system state, intelligently identifies the current workload mode, and selects and switches to the most suitable strategy for the current mode from a preset scheduling strategy library, including the following steps:

[0058] S1: Real-time acquisition of system state data. This step uses the extended Berkeley packet filter technology to perform low-overhead, non-intrusive data acquisition in the operating system kernel space. In the specific implementation, the bpftrace toolset is used to write and load the extended Berkeley packet filter program. The specific steps of acquisition are as follows:

[0059] S1-1: First, implement state monitoring in the kernel space through the extended Berkeley packet filter technology. This process involves writing, deploying, and executing the extended Berkeley packet filter program. In a preferred embodiment, the collected system running indicators cover six dimensions: central processor, memory, disk, process, scheduling, and network. Specifically, the indicators can include those shown in Table 1:

[0060] Table 1 System state data indicator table

[0061]

[0062] To collect the indicators shown in Table 1, the embodiment loads the pre-written extended Berkeley packet filter program into the kernel through the user-mode control program, and dynamically attaches it to a series of preset event points in the kernel. The event points can specifically include but are not limited to the following examples:

[0063] Tracepoint: A stable and open event tracking interface in the kernel. The tracepoint used in this embodiment can include: sched:sched_switch for monitoring task switching, sched:sched_wakeup for monitoring task wake-up, sched:sched_migrate_task for monitoring task migration; block:block_rq_issue and block:block_rq_complete for monitoring disk I / O requests; sock:inet_sock_set_state, tcp:tcp_destroy_sock, and tcp:tcp_retransmit_skb for monitoring TCP connection status and events; and irq:softirq_entry, irq:softirq_exit, and irq:irq_handler_entry / exit for monitoring interrupt handling.

[0064] Kernel probes: dynamic probes placed at the entry or exit of kernel functions. The probes used in this embodiment can include: __mutex_lock_slowpath, mutex_lock, mutex_unlock for monitoring lock contention, holding and blocking conditions; tcp_sendmsg, tcp_recvmsg, netif_receive_skb, ip_rcv for monitoring key nodes in network packet sending and receiving processes; and input_event_to_user for monitoring user input events.

[0065] When the kernel execution flow triggers any of the above-attached event points, the corresponding extended Berkeley packet filter program is executed in the kernel context. To reduce the impact of data collection on system performance, a part of the data statistics and aggregation operations are directly completed in the kernel state. This process uses the extended Berkeley packet filter map provided by the extended Berkeley packet filter as an efficient data exchange structure between the kernel state and the user state. The aggregation operations performed in the kernel state specifically include event counting, numerical distribution statistics (such as histograms), and calculation of basic statistical quantities (maximum, minimum, and average). In this way, the original, high-frequency event stream is preprocessed into a structured, low-frequency statistical data stream, which is then read by the user-space monitoring program from the extended Berkeley packet filter map.

[0066] Secondly, an auxiliary user-space monitoring program is used to obtain information that is not easily directly collected or is costly to process by the extended Berkeley packet filter probes. The user-space monitoring program obtains supplementary system state data by reading and analyzing information in the virtual file system (such as the / proc and / sys file systems) provided by the operating system. The specific collection contents include percentage calculation of CPU utilization, global memory state acquisition, disk queue length reading, and process-related state statistics. In particular, the monitoring program can also be configured to identify processes with specific properties, such as processes that are using a graphics processor or processes that currently have user input focus, by executing external commands or scripts, and to perform special statistics on the resource usage of these processes.

[0067] S1-2: Kernel-state data collection and aggregation. When the kernel execution flow triggers any of the above-attached event points, the corresponding extended Berkeley packet filter program is safely executed in the kernel state. The program directly accesses the kernel data structures related to the events to obtain the original performance indicators. To reduce the data processing and transmission overhead, the extended Berkeley packet filter program uses the extended Berkeley packet filter map in the kernel for real-time aggregation.

[0068] In the embodiment, the extended Berkeley packet filter map type and aggregation operation used include: using hash map BPF_MAP_TYPE_HASH to count the frequency of occurrence of various events; using histogram map BPF_MAP_TYPE_HISTOGRAM to record the logarithmic distribution of key event delays; for delay data, directly calculating its maximum value, minimum value, cumulative value and count value in the kernel for calculating the average value in the user state. The aggregated structured data is read by the user state monitoring program through the extended Berkeley packet filter map, so as to complete the data collection.

[0069] S2: Identify the current system load mode. This step is responsible for converting the collected data into a load mode label for decision making. The specific steps are as follows:

[0070] S2-1: Data preprocessing and feature engineering. The user state monitoring program reads the aggregated data in the extended Berkeley packet filter map at a fixed time interval (1 second in this embodiment) to form a time window. For each time window data, feature engineering calculation is performed to calculate key quantile values (50th percentile, 90th percentile, 95th percentile) and Gini coefficient.

[0071] S2-2: Feature vector construction and classification. All the statistics calculated in the previous step are combined with the aggregated data directly read from the extended Berkeley packet filter map to construct a structured feature vector with a dimension of 150. The feature vector is input into a pre-trained gradient boosting decision tree classification model. The model is implemented using the XGBoost framework in this embodiment and is optimized for hyperparameters through Bayesian optimization. The output of the model is the load mode label of the current time window. The label is stored in a first-in-first-out queue, i.e. the mode label pool, along with the current timestamp.

[0072] To verify the representativeness of the system indicators selected by the invention, the effectiveness of the feature engineering and the classification model, a series of experiments were conducted. First, the model was trained and tested using about 70,000 seconds of effective system indicator data from 4 prototype devices. As shown in Figure 3 , the classification accuracy of the model on the test set reached 99.68%, proving that the feature engineering and model proposed by the invention can fully identify the pattern characteristics of system load indicator data and accurately identify known load types.

[0073] Second, to further verify the generalization ability of the model, i.e. its performance in hardware environments that have not been learned, the model was directly applied to 6 different hardware configurations of unseen devices. As shown in Figure 4As shown, in these brand-new environments, the classification model described in the present application can still achieve a classification accuracy of 88.36%, proving that the feature engineering and model proposed in the present application can avoid dependence on specific device hardware conditions, thereby avoiding the overhead of repeated training.

[0074] S3: Adopting a history window mechanism based on time-weighted voting to predict the next system load pattern. Due to the minimum time interval limit allowing switching in the scheduling strategy switching stage, the impact of the classification model's incorrect prediction will be further amplified. To solve this problem, the present step adopts the flow shown in Figure 2 to smooth the instantaneous identification results through historical data, improving the stability of the prediction. The specific steps of the prediction are as follows:

[0075] S3-1: Maintaining a history record window. The pattern label pool is used as a fixed time length history record window. In the present embodiment, the length N of the window is set to 6 seconds. When a new label enters, if the window is full, the earliest label is removed.

[0076] S3-2: Calculate the time weight. For each identification result (L i ,τ i ) in the history record window, calculate its time weight W(r i ). The calculation formula is:

[0077]

[0078] Wherein, r i is the identification result record, L i is the identified load pattern, τ i is its timestamp, t cur is the current timestamp, and a is a preset time decay factor, which is a value from 0.1 to 0.5, and in the present embodiment, it is set to 0.1.

[0079] To verify the effectiveness of the history window mechanism based on time-weighted voting, an ablation experiment was conducted. The prediction accuracy of the classification model in S2-2 was used as the test benchmark, the length N of the history record window was changed, and the proportion of time that the system correctly applied the target scheduling strategy under the minimum time interval limit of the scheduling strategy switching was observed to observe the improvement effect on correct prediction time. As Figure 5As shown, when the window length is 1 second (equivalent to not using historical weighting, only relying on instantaneous results), the superposition of the error of instantaneous prediction and the cooling period of policy switching causes about 9% accuracy reduction. When the window length increases to more than 3 seconds, the positive effect of the mechanism begins to appear, and when the window length is 6-10 seconds, it reaches about 4% stable performance improvement. The experimental results prove that the time-weighted voting mechanism proposed in the application can alleviate the influence of the minimum time interval limit when the scheduling policy is switched, effectively improve the stability of the decision, and ensure that the application can achieve good performance in a real dynamic environment.

[0080] S3-3: Weighted voting. For all records in the history window, the load pattern L i is grouped. For each group, the weight W(r i of all records in the group is accumulated to obtain the weighted total score of the load pattern. Finally, the load pattern with the highest weighted total score is selected as the final prediction of the running state of the system in the next stage.

[0081] S4: Select the target scheduling policy. Since the device to which the method is applied can have different hardware resources and system environments, the same load pattern can have different matching target scheduling policies on different devices. Therefore, before running the method, a "load pattern-scheduler" mapping table needs to be constructed offline as shown in the flow. Figure 6 At runtime, this step will select the target scheduling policy according to the pre-constructed mapping table.

[0082] S4-1: In the offline state, for each scheduler in the scheduler library, run performance tests under all predefined load patterns, and calculate the performance score of each scheduler under each load pattern according to a preset evaluation standard. First, define a scheduler library, which in this embodiment includes the following schedulers: scx_nest, scx_bpfland, scx_p2dq, scx_lavd, scx_simple, scx_flash, scx_rusty, and the system default earliest eligible virtual deadline first scheduler.

[0083] Then, for each predefined load pattern (a total of 28, such as "game running + mixed IO", "web browsing + network transmission"), use a standardized benchmark test program to run multiple rounds of performance tests under each scheduler.

[0084] According to the performance indicators (such as game frame rate, compilation time) collected by the test, the preset evaluation standard is used to calculate the comprehensive performance score of each scheduler under the load pattern.

[0085] In the embodiment, the evaluation criteria are divided into absolute numerical score and stability score, the obtained scores are averaged to obtain the comprehensive performance score.

[0086] The absolute numerical score is calculated by an improved Sigmoid function, and the formula is:

[0087]

[0088] wherein, S abs (x) is the absolute numerical score, x is the performance index value, S max is the preset full score, which is 10000 in the embodiment, θ is the performance threshold, f is the scaling factor, k v is the value coefficient, γ v is the value index, and ± in the formula is used to adapt to positive and negative performance indicators.

[0089] The stability score is calculated according to the variance of the performance index value, and the formula is:

[0090]

[0091] wherein, S stability is the stability score, S max is the preset full score, which is 10000 in the embodiment, σ is the variance of the performance index value, k s is the stability coefficient, and γ s is the stability index.

[0092] S4-2: Based on the calculated comprehensive performance score, find the highest scoring scheduler for each load mode, and store the corresponding relationship as a "load mode-scheduler" mapping table.

[0093] S4-3: Runtime query. In runtime, the scheduling agent uses the predicted load mode determined by S3 (which completely corresponds to the pre-defined load mode set in S4-1) as a key to query the mapping table constructed in S4-2, so as to obtain the corresponding target scheduling strategy.

[0094] S5: Dynamic switching of scheduling strategy. This step is realized by the extensible scheduler class framework provided by the kernel, and the detailed process is shown in FIG. 5. Figure 7 When the target scheduling strategy determined by S4 is inconsistent with the currently active scheduling strategy of the system, the execution module of the scheduling agent sends an instruction to the kernel to safely unload the currently active scheduler extension Berkeley packet filter program through the interface of the extensible scheduler class, and loads, verifies and attaches a new target scheduler extension Berkeley packet filter program.

[0095] The extensible scheduler class framework guarantees the safety of this process. If the new program fails to load, the system will automatically fall back to the default scheduler, and will not cause the system to crash.

[0096] To prevent the scheduler from frequently switching due to rapid oscillation between different modes, in this embodiment, after each successful policy switch, the system will enter a 10-second cooling period. During the cooling period, even if the decision module outputs a new target policy, the execution module will ignore the switch request.

[0097] Through the cyclic execution of the above five steps, the method of the present application can continuously monitor the system state, intelligently judge the load mode, and adaptively adjust the scheduling policy, so as to achieve near-optimal system performance in various complex application scenarios.

[0098] In order to verify the effectiveness of the present application, a series of experiments were conducted on 10 devices with different hardware configurations, and were compared with the earliest virtual deadline priority scheduler used by Linux by default, and the target scheduler of each scene.

[0099] In the performance comparison with the Linux default scheduler, first, run the test under the predefined load mode, and apply the quantitative scoring method in S4-1 to evaluate the scene performance indicators. When comparing scene scores, performance scores exceeding the benchmark by 5% are defined as "performance improvement", scores lower than the benchmark by 5% are defined as "performance degradation", and scores between the two are considered as "performance equivalent".

[0100] Table 2: Performance comparison of the present application with the Linux default scheduler under the predefined load mode

[0101]

[0102] As shown in Table 2, on the prototype device used for training and the brand new unseen device, the present application can achieve performance improvement or flatline with the baseline in more than 90% of the mixed load scenarios, proving that the present application has universal effectiveness and significant optimization effect in diversified environments.

[0103] To verify the generalization ability of the present application under completely unseen, standardized workloads, multiple test items in OpenBenchmark were introduced for comparison experiments, and the item score improvement ratio was calculated.

[0104] Table 3: Performance improvement ratio of the present application under standardized unseen tasks

[0105] Benchmark Description Performance improvement (%) cyclictest Real-time scheduling latency of the test system +20.36 Stockfish Chess engine computation performance test +26.13 CoreMark CPU performance test +43.37 schbench Scheduler benchmark +7.26 espeak Text-to-speech performance test +4.06 SQLite SQLite database operation performance test +2.71 NPB SP.B Parallel benchmark: scalar five-diagonal solver -1.54

[0106] As shown in Table 3, the application can bring significant performance improvement in most standard tests, proving the generalization ability of the application in the unseen load mode.

[0107] Embodiment 2

[0108] With reference to Figure 8 The embodiment relates to a real-time state-aware adaptive process scheduling system, including a memory and one or more processors, the memory stores executable code, and the one or more processors execute the executable code to implement the real-time state-aware adaptive process scheduling method of embodiment 1.

[0109] Embodiment 3

[0110] The embodiment relates to a computer readable storage medium, which stores a program, and the program is executed by a processor to implement the real-time state-aware adaptive process scheduling method of the application.

[0111] The content described in the embodiments of the present specification is only a list of implementation forms of the inventive concept, and the protection scope of the application should not be regarded as being limited to the specific forms stated in the embodiments, and the protection scope of the application also extends to equivalent technical means that can be thought of by those skilled in the art according to the inventive concept.

Claims

1. A method for adaptive process scheduling based on real-time state awareness, the method comprising: The method comprises the following steps: S1: Real-time acquisition of system state data. Real-time acquisition of information reflecting the operating system runtime load condition by extending the Berkeley packet filter technology. S2: Identify the current system load pattern. Preprocess and feature engineer the system state raw data to generate a structured feature vector, and analyze the feature vector using a pre-trained gradient boosting decision tree classification model to obtain the current load pattern label and store it in the pattern label pool. S3: Predict the system load pattern of the next decision cycle. According to the historical pattern label data in the pattern label pool, use a fixed-size history window and exponential decay weight for weighted voting to predict the next system load pattern. S4: Select the target scheduling strategy. According to the predicted load pattern, query the preset load pattern-target scheduling strategy mapping table to retrieve and determine the corresponding target scheduling strategy. S5: Perform scheduling strategy dynamic switching. Compare the target scheduling strategy with the current active scheduling strategy of the system. If they are inconsistent, unload the current scheduling strategy and load and apply the target scheduling strategy dynamically at runtime through the extensible scheduler class technology, thereby realizing real-time adaptive adjustment of the scheduling strategy.

2. The real-time state-aware based adaptive process scheduling method according to claim 1, wherein, In S1, the kernel-level monitoring technology uses the extended Berkeley packet filter technology to collect multi-dimensional system state raw data. The specific steps of collecting data are as follows: S1-1: Through the user-mode tool chain of the extended Berkeley packet filter technology, the pre-written extended Berkeley packet filter program is dynamically mounted to multiple preset event points in the kernel, the event points are selected from at least one of the following: process scheduling, memory usage, network protocol stack, file system, and interrupt handling related kernel function probes, trace points, or performance events, and the collected system running indicators cover six dimensions of central processor, memory, disk, process, scheduling, and network. S1-2: When the kernel runs the event point, the corresponding extended Berkeley packet filter program is automatically executed. The program directly accesses the event data structure in the kernel state, collects raw performance indicator data, and performs preliminary real-time aggregation and statistics in the kernel, including statistics of event occurrence frequency and statistics of delay distribution histogram, maximum value, minimum value, and average value. Subsequently, the aggregated structured data is transmitted to the user-mode monitoring program through the extended Berkeley packet filter mapping.

3. The real-time state-aware based adaptive process scheduling method according to claim 1, wherein, The method of step S1-1 Central processor indicators include: user state utilization rate; priority adjustment state utilization rate; system state utilization rate; idle utilization rate; IO waiting utilization rate; hard interrupt handling utilization rate; soft interrupt handling utilization rate; virtualization stealing rate; hot core proportion; Memory indicators include: total memory capacity; total swap memory; free memory capacity; buffer memory; cache memory; available swap memory; Disk indicators include: disk read / write queue length; read / write operation count; read / write delay; read / write operation size; Process indicators include: total number of system processes; process central processor utilization; graphics acceleration process central processor utilization; number of graphics acceleration processes; focus process central processor utilization; focus process memory usage; input event count; Scheduling indicators include: task migration times; lock contention times; lock holding time; thread blocking time; lock acquisition failure times; thread wake-up delay; task switch delay; task switch times; run queue length; Network indicators include: connection count; new connection count; closed connection count; reset connection count; data size; data packet quantity; data packet retransmission rate; TCP / UDP proportion; network interruption number; network interruption processing time; data transmission delay; kernel-user space delay.

4. The real-time state-aware based adaptive process scheduling method according to claim 1, wherein, In S2, the collected raw data is preprocessed and feature engineered to obtain a structured feature vector aligned with the input of the gradient boosting decision tree classification model. The specific steps of obtaining the structured feature vector are as follows: S2-1: The user state program receives the aggregated data stream from the extended Berkeley packet filter mapping and performs time window division; and calculates multiple dimensions of statistics for the data in each time window, including key quantile values (50th percentile, 90th percentile, 95th percentile) and Gini coefficient. S2-2: Combine and normalize the multi-dimensional statistics in the time window to construct a 150-dimensional feature vector as the input of the gradient boosting decision tree classification model.

5. The real-time state-aware based adaptive process scheduling method according to claim 1, wherein, In S3, the historical window mechanism based on time-weighted voting includes the following steps: S3-1: Maintain a fixed-length queue as a historical record window to store the preliminary identification results of the load pattern output by the classification model in the recent N seconds and their corresponding time stamps, where N is a value from 3 to 10. S3-2: Calculate the time weight W(r i ) for each recognition result (L i ,τ i ) in the history record window, the calculation formula is: wherein r i is a recognition result record, L i is a recognized load pattern, τ i is a time stamp thereof, t cur is a current time stamp, and a is a preset time decay factor, which is a value from 0.1 to 0.

5. S3-3: Group the recognition results in the history record window by load pattern type, and accumulate the weight values W(r i ) within each group to obtain a weighted score for each load pattern; select the load pattern L i with the highest weighted score as the final prediction result for the future system state.

6. The method of claim 1, wherein, In S4, the specific steps of determining the target scheduling strategy are as follows: S4-1: In an offline state, for each scheduler in the scheduler library, run performance tests under all predefined load patterns, and calculate the performance score of each scheduler under each load pattern according to a preset evaluation standard; S4-2: Based on the performance score, determine the scheduler with the highest performance score for each load pattern, and construct a "load pattern-target scheduler" mapping table; S4-3: At runtime, use the target load pattern determined by S3 as the key to query the mapping table to obtain the corresponding scheduling strategy as the target scheduling strategy.

7. The real-time state-aware based adaptive process scheduling method according to claim 1, wherein, In S5, the preset scheduler library contains at least two or more scheduling strategies, and all schedulers in the preset scheduler library must be calibrated by S4-1 and S4-2 to establish the "load pattern-target scheduler" mapping table.

8. The method of claim 6, wherein, The evaluation standard at least includes the steps of calculating the absolute numerical score and the stability score according to the performance indicator value, wherein: The absolute numerical score is calculated by an improved Sigmoid function, the formula is: wherein S abs (x) is an absolute numerical score, x is a performance indicator value, S max is a pre-set full score, θ is a performance threshold, f is a scaling factor, k v is a value coefficient, γ v is a value exponent, and ± in the formula is used to adapt positive and negative performance indicators. And the stability score is calculated according to the variance of the performance indicator value, the formula is: Wherein, S stability is a stability score, S max is a preset full score value, σ is a variance of the performance index value, k s is a stability coefficient, γ s is a stability index.

9. An adaptive process scheduling system based on real-time state awareness, characterized by, An apparatus comprising a memory having executable code stored therein and one or more processors that, when executing the executable code, implement the real-time state-aware adaptive process scheduling method of any of claims 1-8.

10. A computer-readable storage medium, characterized in that, A program stored thereon, which, when executed by a processor, implements the real-time state-aware adaptive process scheduling method of any of claims 1-8.

Images