DQN-based multi-core real-time system task sorting and partition scheduling optimization method

Abstract

The invention relates to a multi-core real-time system task sorting and partition scheduling optimization method based on a DQN, and the method comprises the following steps: obtaining a real-time task set to be scheduled, and extracting the execution time, period, deadline and other feature information of tasks; modeling a task sorting process as a Markov decision process, and performing adaptive optimization on a task processing sequence by using a DQN model to generate a task sorting sequence; on the premise that the structure of a partition scheduling heuristic algorithm is not changed, the task sorting sequence is input into the partition scheduling algorithm, and tasks are distributed; and generating scheduling feedback information according to a task allocation result, wherein the scheduling feedback information is used for updating the DQN model. According to the method, intelligent optimization is carried out on the task sorting stage, the problem that a traditional fixed sorting strategy is difficult to adapt to a complex task set structure is solved, a partition scheduling algorithm can find a feasible task allocation scheme under the high system load condition, and therefore the schedulability and the scheduling success rate of the isomorphic multi-core real-time system are improved.

Description

Technical Field

[0001] This invention belongs to the field of embedded real-time systems and multi-core processor scheduling technology, specifically involving a method for optimizing task sorting and partition scheduling in multi-core real-time systems based on DQN. Background Technology

[0002] With the continuous development of technologies such as the Internet of Things, autonomous driving, and industrial automation, real-time embedded systems have been widely used in fields with strict timing requirements, such as smart cars, aerospace, medical devices, and industrial control. These systems typically require each real-time task to complete execution before a specified deadline; otherwise, system malfunctions or security risks may occur. Therefore, a reasonable real-time scheduling mechanism is one of the key technologies for ensuring the correctness of system timing and operational reliability.

[0003] Early research on real-time scheduling primarily focused on single-processor systems, and the related scheduling theories and analysis methods are relatively mature. However, as application scenarios have increased their demands for computing performance and parallel processing capabilities, single-processor architectures have gradually become insufficient to meet practical needs. Meanwhile, constrained by the limited upside potential of processor clock speeds, multi-core processors, with their advantage in balancing performance and power consumption, have become the mainstream hardware platform for real-time embedded systems.

[0004] In multi-core real-time systems, scheduling strategies mainly fall into two categories: global scheduling and partition scheduling. Global scheduling allows tasks to dynamically migrate between multiple processors, which is beneficial for load balancing; however, frequent task migrations introduce additional runtime overhead, reducing system determinism and predictability. In contrast, partition scheduling statically allocates tasks to each processor core before scheduling and maintains the binding relationship between tasks and processors during execution, thus significantly reducing scheduling overhead and improving system predictability. Therefore, it has been widely used in engineering practice.

[0005] Task allocation in partitioned scheduling can typically be modeled as a bin packing problem, which is a strongly NP-hard problem. Due to the difficulty in obtaining the optimal solution, existing techniques generally employ heuristic algorithms for approximate solutions, such as first-fit, best-fit, and worst-fit methods. In practical applications, these heuristic algorithms are usually combined with pre-defined task sorting rules, such as sorting tasks in descending or ascending order based on utilization or cycle time.

[0006] However, in existing technologies, task sorting rules are usually fixed rules that remain unchanged throughout the task allocation process, making it difficult to dynamically adjust them according to the structural characteristics of specific task sets. Since the task allocation results of partition scheduling are largely affected by the task input order, when the system load is high or the task set is at the schedulable boundary, the fixed sorting method can easily lead to uneven load distribution on some processors, thus preventing some task sets that should be schedulable from passing the scheduling decision.

[0007] In recent years, reinforcement learning methods have attracted attention in combinatorial optimization and resource scheduling problems, with some studies attempting to apply learning methods to task allocation or scheduling parameter tuning. However, in partition scheduling scenarios, existing techniques typically treat task sorting as a fixed preprocessing step in the allocation algorithm, lacking effective technical solutions for independent modeling and adaptive optimization of the task sorting stage. Since task sorting directly affects the subsequent task allocation process, its lack of flexible adjustment mechanisms limits further improvement in the overall performance of partition scheduling algorithms.

[0008] Therefore, it is necessary to provide a new technical solution that adaptively optimizes the task sorting process without changing the existing partition scheduling and allocation mechanism, so as to improve the schedulability of multi-core real-time systems under complex task sets and high load conditions. Summary of the Invention

[0009] The purpose of this invention is to provide a multi-core real-time system task sorting and partition scheduling optimization method based on DQN, which aims to improve the task scheduling success rate of multi-core real-time systems by intelligently optimizing the task processing order, breaking through the performance bottleneck of traditional heuristic algorithms.

[0010] The technical solution adopted in this invention is: a multi-core real-time system task sorting and partition scheduling optimization method based on DQN, comprising the following steps:

[0011] Step 1: Obtain the set of periodic real-time tasks to be scheduled and define the task set. Each task By triplet express, The worst-case execution time. For the task cycle, For the deadline, this example uses an implicit deadline model, i.e. = .Task Utilization rate is defined as = relaxation Defined as .

[0012] Step 2: For each task in the task set, extract scheduling-related task features, including task utilization. Task cycle Worst execution time Deadline and task slack This is used to describe the resource requirements of a task during the scheduling process.

[0013] Step 3: Model the task ranking process as a Markov decision process. In each decision stage, select a task from the unranked task set as the task with the current ranking position. Construct the decision states. This is used to represent the currently sorted set of tasks and the remaining unsorted set of tasks, and the action set A is defined as the currently available set of unsorted tasks.

[0014] Step 4: Select the next task to be sorted based on the current decision state, gradually generating a complete task sorting sequence. In a preferred embodiment, the reinforcement learning model is a DQN model, which uses a neural network to analyze the state-action value function. Approximation is performed to achieve adaptive sorting decisions for different task set structures.

[0015] Step 5: Combining the sorting results with partition scheduling, the task sorting sequence generated in Step 4 is input into the partitioned task allocation algorithm. Tasks are processed sequentially according to the sorting sequence, and the tasks are allocated to multiple processor cores. This ensures that the task set on each processor core meets the preset scheduling constraints. The partitioned task allocation algorithm includes at least one heuristic algorithm such as First Fit, Best Fit, or Worst Fit, and within each processor core, the earliest deadline first (EDF) scheduling strategy is used to schedule the tasks allocated to that processor core.

[0016] Step 6: Based on the task allocation results and the load on each processor core, determine whether the EDF schedulable conditions are met. Generate scheduling feedback information based on the task allocation results. If the conditions are met, generate a positive scheduling feedback reward; otherwise, generate a negative scheduling feedback reward. Then, use the scheduling feedback information to update the DQN model to guide it in learning a task sequencing strategy that is more conducive to system schedulability.

[0017] Compared with existing technologies, the present invention has the following advantages:

[0018] This invention reduces the limitations of fixed sorting rules on partition scheduling performance and improves the overall schedulability of multi-core real-time systems by adaptively learning and optimizing the task sorting process.

[0019] This invention achieves performance improvement without changing the existing partition scheduling and allocation algorithm structure, and is highly compatible and easy to implement in engineering.

[0020] This invention can dynamically adjust the task sorting strategy according to different task set structures, and is especially suitable for real-time scheduling scenarios with high system load or schedulable boundary conditions. Attached Figure Description

[0021] Figure 1 This is a schematic diagram illustrating an example execution process of a multi-core real-time system task sorting and partition scheduling optimization method based on DQN according to the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of this invention. Figure 1 As shown, this embodiment provides a multi-core real-time system task ordering and partition scheduling method based on DQN. This method improves the schedulability of the multi-core real-time system by adaptively learning and optimizing the task ordering process without changing the underlying partition scheduling heuristic algorithm. The method includes the following steps:

[0023] Step 1: Generate a real-time task set using the given task parameters, and define... Given a set of n tasks, each real-time task... By triplet It means that, among them, Indicates task Worst execution time on the processor, Indicates the task cycle. This indicates the task deadline. Each execution of a task is called a job, which is an instance of a task and the smallest unit of execution in scheduling. Task The j-th assignment is denoted as .

[0024] Step 2: Construct the DQN model decision state based on the current scheduling stage. This decision state represents the set of sorted tasks and the set of unsorted tasks. For each task in the unsorted task set... Extract its task feature information, which includes: worst-case execution time. Task cycle Relative deadline Task utilization It is defined as: = / relaxation It is defined as The aforementioned task features together constitute the input state of the DQN model.

[0025] Step 3: Model the task ranking process as a Markov decision process. At each decision stage, the DQN model selects a task from the unranked task set as the task for the current ranking position based on the current decision state. The DQN model uses a deep neural network to analyze the state-action value function. Perform modeling and determine the task selection results for the current stage based on the model output. Repeat the above process until all tasks have been selected, forming a complete task ranking sequence. .

[0026] Step 4: Sort the task sequence obtained in Step 3. In the input partitioned task allocation algorithm, assume a homogeneous multi-core processor platform as follows: In this system, all processor cores have the same computing power. Tasks are processed sequentially according to the task sorting sequence, and the current task is assigned to a processor core that meets the scheduling constraints based on a preset partitioned task allocation method. This partitioned task allocation method includes at least one of First Fit, Best Fit, or Worst Fit, and within each processor core, the earliest deadline first (EDF) scheduling strategy is used to schedule the tasks assigned to that core. In a homogeneous multi-core platform, the execution time of the same task remains consistent across different processor cores.

[0027] Step 5: Based on the task allocation results, calculate the task load on each processor core and determine whether it meets the EDF schedulable conditions. Generate scheduling feedback information based on the task allocation results. Positive scheduling feedback is generated when the task load on all processor cores meets the scheduling constraints; negative scheduling feedback is generated when the task load on any processor core exceeds the scheduling constraints. The scheduling feedback information is used to update the parameters of the DQN model to guide the model in learning and generating a better task ranking strategy.

[0028] Step 6: In one specific embodiment, the homogeneous multi-core processor platform consists of two processor cores. Among them, processor core and The computing power is the same. The task set is: ,in, The numbers () represent the worst-case execution time, deadline, and period of each task, respectively. The task set is sorted using an increasing utilization sorting strategy, and the sorting result is input into the Best Fit partitioned task allocation algorithm for task allocation. Under this sorting sequence, at least one processor core will have a sum of task utilization exceeding 1 during task allocation, resulting in failure to meet the EDF schedulable condition and causing task allocation failure. Following steps 2 and 3 of this invention, the DQN model generates a task sorting sequence based on the execution time, period, utilization, and slack of each task: Subsequently, the task sorting sequence is input into the same optimal adaptive partitioned task allocation algorithm for task allocation, and the allocation result is: processor cores upper-assigned task set { , The sum of their task utilization rates satisfies the EDF schedulable condition; processor cores upper-assigned task set { The sum of task utilization on each processor core satisfies the EDF (Earliest Deadline First) schedulable condition. Within each processor core, tasks are scheduled using the Earliest Deadline First (EDF) scheduling strategy. It is determined that the task utilization on each processor core does not exceed 1, satisfying the EDF schedulable condition, thus generating positive scheduling feedback information. This positive scheduling feedback information is used to update the parameters of the DQN (Device Quantization) model, thereby guiding the model to generate a task ordering sequence that is more conducive to system schedulability in subsequent scheduling processes. As can be seen from the above embodiments, the method of the present invention can improve the task schedulability of a homogeneous multi-core synchronous system by optimizing the task ordering strategy without changing the partitioned scheduling algorithm structure.

Claims

1. A method for optimizing task sequencing and partition scheduling in a multi-core real-time system based on DQN, characterized in that, Includes the following steps: Step 1: Obtain the set of periodic real-time tasks to be scheduled Each task By triplet express, The worst-case execution time. For the task cycle, The deadline is specified, and the implicit deadline constraint is satisfied. = ; Step 2: Model the task sorting process as a Markov decision process, where at each decision stage, a task is selected from the set of unsorted tasks as the task with the current sorting position; construct a state vector to describe the current scheduling state. and the state vector Input into the DQN model, based on the state-action value function Output the action value of each candidate task in the current state; Step 3: Select the next task to be sorted based on the maximum action value criterion, and gradually generate a complete task sorting sequence; use the complete task sorting sequence as input, process the tasks sequentially using a partitioned task allocation method, and assign each task to a specific processor core. The partitioned task allocation method includes at least one of First Fit, Best Fit, or Worst Fit, and within each processor core, the earliest deadline first (EDF) scheduling strategy is used to schedule the tasks allocated to that processor core. Step 4: After task allocation is completed, based on the task set on each processor core, determine whether it meets the EDF schedulable condition. Generate scheduling feedback information based on the task allocation result. If it meets the condition, generate a positive scheduling feedback reward; otherwise, generate a negative scheduling feedback reward. Then, use the scheduling feedback information to update the DQN model, thereby optimizing the subsequent sorting strategy.

2. The method for optimizing task sorting and partition scheduling in a multi-real-time system based on DQN according to claim 1, characterized in that, The real-time tasks are periodic real-time tasks, each task... By triplet It means that among them The worst-case execution time of the task. For the task cycle, This is a relative deadline.

3. The method for optimizing task sorting and partition scheduling in a multi-real-time system based on DQN according to claim 2, characterized in that, Task utilization rate With task slack They are defined as , .

4. The method for optimizing task sorting and partition scheduling in a multi-real-time system based on DQN according to claim 1, characterized in that, The decision state This includes the utilization set, slack set, and cycle set of unsorted tasks.

5. The method for optimizing task sorting and partition scheduling in a multi-real-time system based on DQN according to claim 1, characterized in that, The reinforcement learning model uses a state-action value function. For in state Select Task It is represented as the long-term scheduling benefit of the current sorting position.

6. The method for optimizing task sorting and partition scheduling in a multi-real-time system based on DQN according to claim 5, characterized in that, The state-action value function is adjusted according to the following update relationship: ,in, For learning rate, denoted as the discount factor, and r as the feedback reward generated based on the scheduling result.

7. The method for optimizing task sorting and partition scheduling in a multi-real-time system based on DQN according to claim 1, characterized in that, The feedback reward r is determined based on the sum of task utilization on each processor core in the partition scheduling result, provided that the sum of task utilization on any processor core satisfies: .

Images