Multi-constraint college course arrangement method based on Q learning hyper-heuristic algorithm

Abstract

The invention discloses a multi-constraint college course arrangement method based on a Q learning hyper-heuristic algorithm. The method comprises the following steps: step 1, establishing a mathematical model of a college course arrangement problem with complex constraints; step 2, coding and decoding; 3, initializing an optimization algorithm and Q learning parameters; 4, selecting a current state and a current action; step 5, generating a new course arrangement method by using the selected action; step 6, updating the Q table; step 7, calculating a Pt value, and constructing a state space based on the Pt value; and 8, judging whether the number t of iterations reaches Itr or not until the Gantt chart corresponding to the optimal solution X is output. According to the method, the efficiency and quality of course arrangement of colleges and universities are effectively improved, and relatively high robustness is achieved; the method is of great significance in improving the teaching management level and service quality of colleges, effectively improving the resource utilization rate, reducing the learning cost and the like.

Description

Technical Field

[0001] The present invention relates to a multi-constraint college course scheduling method, and in particular to a multi-constraint college course scheduling method based on a Q-learning hyper-heuristic algorithm. Background Art

[0002] With the continuous expansion of university enrollment, the space and teaching resources of colleges and universities are becoming increasingly tight. From the perspective of colleges and universities, how to effectively use limited teaching resources to better meet the needs of education and teaching, so as to maximize resource utilization and optimize service quality, has become a core issue of college reform and development. Scheduling is an important part of the teaching management process of colleges and universities. It is necessary to obtain a feasible scheduling plan under complex and multi-constrained conditions such as resources, time sequence, and personalized needs. An efficient and reasonable scheduling plan can not only improve the level of teaching management and service quality, but also effectively improve resource utilization and reduce school costs.

[0003] The class scheduling problem in colleges and universities is an NP-hard problem. In order to meet the personalized needs of teachers and students and ensure the actual teaching effect, it is usually necessary to consider the influence of various complex factors. Due to the complexity of the class scheduling problem in colleges and universities, the current research on the class scheduling problem is still in its infancy. In the prior art, meta-heuristic algorithms are usually used to solve the class scheduling problem, such as genetic algorithms, particle swarm algorithms, simulated annealing, etc. The application number is 2023110632064, and the name of the invention patent is "A method and system for automatic and rapid class scheduling in colleges and universities based on global optimization". It discloses a class scheduling method based on an improved polar bear optimization algorithm for realizing automatic and rapid class scheduling in colleges and universities. However, like other meta-heuristic algorithms, this invention can achieve better results when dealing with situations with fewer constraints, but it still has deficiencies in efficiency and effectiveness when facing situations with complex multiple constraints such as resources and timing.

[0004] In addition, as a cross-domain problem-solving model, the hyper-heuristic algorithm dynamically generates the optimal heuristic method to solve different problems through a high-level heuristic (HLH) strategy to manage and manipulate a series of low-level heuristics (LLH) methods, which provides a new way to solve complex and diverse problems. However, the hyper-heuristic algorithm has the problem of high computational complexity, one of the main reasons being that the HLH strategy itself takes a lot of time to find the optimal heuristic method. Reducing the algorithm complexity of the HLH strategy also has an important impact on improving the overall performance of the algorithm. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a multi-constraint university course scheduling method based on a Q-learning hyper-heuristic algorithm, which is simple and practical, has low optimization method complexity, and can effectively improve the efficiency and quality of university course scheduling.

[0006] The present invention firstly establishes an optimization model for university course scheduling with complex constraints. The optimization model aims to minimize dissatisfaction. On this basis, combined with the hyper-heuristic algorithm framework, the reinforcement learning mechanism is introduced, and Q-learning is used as the HLH strategy of the hyper-heuristic algorithm. According to the characteristics of the university course scheduling problem, low-level heuristic rules oriented to the problem characteristics are designed to construct a set of LLH methods, thereby realizing the optimization solution to the multi-constrained university course scheduling problem.

[0007] The present invention is achieved through the following technical solutions:

[0008] 1. A multi-constraint college course scheduling method based on Q-learning hyper-heuristic algorithm, the method comprises the following steps:

[0009] Step 1: Establish a mathematical model for the college course scheduling problem with complex constraints;

[0010] The multi-constraint college scheduling problem is considered to be a complex multi-constraint combinatorial optimization problem in terms of courses, classes, classrooms, teachers, time, etc., where the constraints are divided into hard constraints and soft constraints. Hard constraints include: (1) a teacher cannot have more than one course in the same time period; (2) a classroom cannot have more than one course in the same time period; (3) a class can only have one course in the same time period; (4) the classroom capacity must not be less than the total number of students in the class; (5) no courses are scheduled in a specific time period. Only when the above hard constraints are met can multi-constraint college scheduling be achieved. Soft constraints include: (1) the distance between two classrooms where students of each class take consecutive classes is minimized; (2) the number of classes a teacher teaches in a day is less than or equal to the specified number; (3) the theory class and the experimental class of the same course are not scheduled on the same day.

[0011] Set up a college with teachers Course Collection Classroom Collection Class collection in Respectively represent the number of teachers, courses, classrooms and classes; the set Indicates the number of days of classes per week, where The number of days per week for classes; Indicates the number of classes per day, is the number of classes per day; U n represents the number of students in class n (n∈N); S r represents the capacity of classroom r (r∈R); Q t,d represents the number of classes that teacher t (t∈T) teaches on the dth (d∈D) day; D n,d,r,r′It represents the distance between the classrooms r and r′ of two adjacent courses of class n on day d. In addition, the decision variables in the multi-constrained college scheduling problem are defined as follows:

[0012]

[0013] Where c∈C, p∈P;

[0014] On the basis of the above, a fitness function is constructed with the goal of minimizing dissatisfaction, that is, minimizing soft constraints, as follows:

[0015]

[0016]

[0017] Among them, formula (6) is the objective function, fit i represents the i-th (i={1,2,3}) soft constraint fitness value, w i is the weight corresponding to each fitness value; Formulas (7)-(9) are soft constraint fitness functions, where Formula (7) represents the distance between two classrooms where students take classes consecutively, and the closer the distance, the better; Formula (8) represents that the number of classes a teacher teaches in a day does not exceed the upper limit λ, and if it exceeds the upper limit λ, a penalty coefficient δ will be imposed on the number of classes that exceed the upper limit λ; Formula (9) represents that the theory and experiment classes of the same course are scheduled on different days, and if they are scheduled on the same day, they will be multiplied by the penalty coefficient γ. It is worth noting that although class scheduling can still be performed when the soft constraint conditions are not met, the soft constraint fitness value is the standard for measuring the quality of the class scheduling plan;

[0018] The hard constraints in the multi-constrained college course scheduling problem are expressed as follows:

[0019]

[0020] U n ≤S r (13)

[0021] F d,p =1 (14)

[0022] Formula (10) indicates that a teacher cannot have more than one course in the same time period; Formula (11) indicates that a classroom cannot have more than one course in the same time period; Formula (12) indicates that a class can only have one course in the same time period; Formula (13) indicates that the allocated classroom capacity must not be less than the total number of students; Formula (14) indicates that the specific time period specified by the university for the convenience of carrying out school-wide activities is not allowed to schedule classes. The mathematical model of the university scheduling problem with complex constraints established in this step takes into account the complex multi-constraint situations in the dimensions of courses, classes, classrooms, teachers, time, etc., and is more in line with the actual situation of university scheduling;

[0023] Step 2: Encoding and decoding;

[0024] Aiming at the characteristics of multi-constraint college course scheduling problem, the present invention adopts a solution individual coding method based on a single column vector, and the course vector V = {c1, c2, ..., c n} represents a solution individual, and the course scheduling method corresponding to the course vector V generated by the dissatisfaction value minimization strategy is as follows:

[0025] For the course vector V, we first i First, traverse all available classroom resources to determine whether the current available classroom resources meet the hard constraints shown in equations (10)-(14). If there is a feasible classroom resource set R i If the constraints are met, then course c i Arrange to the classroom r with the closest capacity, that is If all available classroom resources cannot meet all hard constraints, the solution individual V is considered an infeasible solution, and the fitness value corresponding to the solution individual is increased by the penalty value ξ based on formula (6). Repeat the above operation until all courses in the course vector V are assigned;

[0026] Step 3: Initialize the optimization algorithm and Q learning parameters;

[0027] 3.1. Initialize algorithm parameters: maximum number of iterations Itr, discount factor γ, cycle execution number EP and Q table;

[0028] 3.2. Initialize the reinforcement learning action set A, including flipping, forward insertion, backward insertion, 2-opt domain search, random sorting and other actions;

[0029] 3.3. Generate initial solution: randomly generate solution X and calculate its fitness value, and randomly select initial state s t ;

[0030] Step 4. Select the current state and action;

[0031] 4.1. Generate a random number rnd in the range [0,1];

[0032] 4.2. Calculate ε according to formula (15): t value;

[0033]

[0034] 4.3. If the random number rnd is greater than ε t , then select the current state s t The action with the largest Q value is selected, otherwise an action is randomly selected from the action set A;

[0035] Step 5: Apply the selected action to generate a new class scheduling method as follows:

[0036] 5.1. Execute current action a t A total of EP times, and generate a new solution X′;

[0037] 5.2. Generate a class schedule based on the proposed decoding strategy. Use formula (6) to calculate the fitness value of the new solution X′ and compare it with the old solution X. If X′ is better, replace X.

[0038] Step 6: Update Q table

[0039] 6.1. Calculate r according to formula (16) t value;

[0040]

[0041] In formula (16), u is a random number in [0,1];

[0042] 6.2. Update the state s in the Q table based on equations (17) and (18) t Next action a t The corresponding Q value;

[0043]

[0044] Step 7: Calculate P according to formula (19) t value, and based on P t The state space constructed by the value

[0045] {(0,0.1),[0.1,0.4),[0.4,0.7),[0.7,1),[1,∞)} Identify the next state s t+1 ;

[0046]

[0047] In formula (19), and are the fitness values ​​of the solution individuals at the beginning and end of period t respectively;

[0048] Step 8: Determine whether the number of iterations t reaches Itr. If the number of iterations t does not reach Itr, go to step 4 to continue execution. Otherwise, output the optimal solution X corresponding to the class scheduling plan and the corresponding Gantt chart.

[0049] Furthermore, a step is added between step 3.1 and step 3.2, which is to initialize the reinforcement learning action set A, including flipping operation, forward insertion, backward insertion, 2-opt field search, and random sorting action.

[0050] Furthermore, constraint (6) in step 1 takes into account the three soft constraints: the distance between two classrooms where students take classes consecutively, the number of classes a teacher teaches on the same day, and whether the theory and experimental classes of the same course are scheduled on the same day. The corresponding fitness function is set according to these soft constraints.

[0051] The beneficial effects of the present invention are as follows: according to the actual situation of college course scheduling, taking the minimization of dissatisfaction of teachers and students as the optimization goal, a college course scheduling optimization model with complex constraints is established. On this basis, a multi-constraint college course scheduling method based on Q learning hyper-heuristic algorithm is proposed. Under the framework of the hyper-heuristic algorithm, a set of LLH methods with global and local search capabilities facing the problem characteristics is designed; under the reinforcement learning mechanism, the LLH method set is used as an action set, and Q learning is used to dynamically select the corresponding LLH method for single solution iterative optimization. The method adopts a single column coding and single solution iterative scheme, which is simple and practical, with low algorithm complexity, can effectively improve the efficiency and quality of college course scheduling, and has high robustness. It is of great significance for colleges and universities to improve teaching management level and service quality, effectively improve resource utilization, and reduce school running costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] For ease of explanation, the present invention is described in detail with reference to the following specific embodiments and accompanying drawings.

[0053] Figure 1 It is a flow chart of the multi-constraint college course scheduling method based on the Q-learning hyper-heuristic algorithm of the present invention.

[0054] Figure 2 It's a Gantt chart. DETAILED DESCRIPTION

[0055] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more specific definition of the protection scope of the present invention;

[0056] The multi-constraint college course scheduling problem is to maximize the soft constraints under the premise of satisfying all hard constraints, so as to find the optimal course scheduling plan. Based on the above considerations, the embodiment is explained by minimizing the soft constraints, that is, minimizing the dissatisfaction.

[0057] Step 1: Set up a collection of teachers in colleges and universities Course Collection Classroom Collection Class collection in Respectively represent the number of teachers, courses, classrooms and classes; the set Indicates the number of days of classes per week, where The number of days per week for classes; Indicates the number of classes per day, is the number of classes per day; U n represents the number of students in class n (n∈N); S r represents the capacity of classroom r (r∈R); Q t,d represents the number of classes that teacher t (t∈T) teaches on the dth (d∈D) day; D n,d,r,r′ It represents the distance between the classrooms r and r′ of two adjacent courses of class n on day d. In addition, the decision variables in the multi-constrained college scheduling problem are defined as follows:

[0058]

[0059] Where c∈C, p∈P.

[0060] On the basis of the above, a fitness function is constructed with the goal of minimizing dissatisfaction, that is, minimizing soft constraints, as follows:

[0061]

[0062]

[0063] Among them, formula (6) is the objective function, fit i represents the i-th (i={1,2,3}) soft constraint fitness value, w i is the weight corresponding to each fitness value; Formulas (7)-(9) are soft constraint fitness functions, where Formula (7) represents the distance between two classrooms where students take classes consecutively, and the closer the distance, the better; Formula (8) represents that the number of classes a teacher teaches in a day does not exceed the upper limit λ, and if it exceeds the upper limit λ, a penalty coefficient δ will be imposed on the number of classes that exceed the upper limit λ; Formula (9) represents that the theory and experiment classes of the same course are arranged on different days, and if they are arranged on the same day, they will be multiplied by the penalty coefficient γ. It is worth noting that although class scheduling can still be performed when the soft constraint conditions are not met, the soft constraint fitness value is the standard for measuring the quality of the class scheduling plan.

[0064] The hard constraints in the multi-constrained college course scheduling problem are expressed as follows:

[0065]

[0066] U n ≤S r (13)

[0067] F d,p =1 (14)

[0068] Formula (10) indicates that a teacher cannot have more than one course in the same time period; Formula (11) indicates that a classroom cannot have more than one course in the same time period; Formula (12) indicates that a class can only have one course in the same time period; Formula (13) indicates that the allocated classroom capacity must not be less than the total number of students in the class; Formula (14) indicates that classes are not allowed to be scheduled during specific time periods set by colleges and universities to facilitate school-wide activities.

[0069] The specific application examples of solving the multi-constrained college course scheduling method based on the Q-learning hyper-heuristic algorithm are as follows:

[0070] Given a college course scheduling problem example as shown in Tables 1 to 4, the example includes 70 courses (as shown in Table 1), 47 teachers (as shown in Table 2), 8 classes (as shown in Table 3) and 17 classrooms (as shown in Table 4). In addition, there are also distances between classrooms, as shown in Table 5.

[0071] Table 1 Examples of university course scheduling (courses)

[0072]

[0073]

[0074] Table 2 Examples of university class scheduling (teachers)

[0075]

[0076]

[0077] Table 3 Example of university class scheduling (class)

[0078] class Number of students 20 E-commerce Class 1 35 20E-commerce Class 2 36 21 E-commerce Class 1 34 21 E-commerce Class 2 35 20 Artificial Intelligence 1 49 20 Artificial Intelligence 2 45 21 Artificial Intelligence1 46 21 Artificial Intelligence 2 48

[0079] Table 4 Example of class scheduling in colleges and universities (classrooms)

[0080]

[0081]

[0082] Table 5 Distance between classrooms

[0083] A101 A102 B101 B102 C101 C102 D101 D102 9305 9201 9103 9303 9102 Basketball Court tennis court Badminton Court Swimming pool A101 0 285 122 444 438 353 496 416 262 82 493 214 390 233 162 222 86 A102 285 0 86 292 290 130 336 102 108 378 296 409 152 406 208 418 96 B101 122 86 0 428 318 444 114 443 326 297 348 314 237 187 423 133 192 B102 444 292 428 0 340 86 137 403 307 485 62 223 310 389 297 396 158 C101 438 290 318 340 0 148 308 51 497 366 374 196 152 148 433 475 70 C102 353 130 444 86 148 0 208 331 157 229 467 291 301 285 164 86 288 D101 496 336 114 137 308 208 0 437 447 113 100 377 191 99 294 487 141 D102 416 102 443 403 51 331 437 0 439 259 104 447 90 218 300 365 376 9305 262 108 326 307 497 157 447 439 0 126 301 198 458 395 126 271 220 9201 82 378 297 485 366 229 113 259 126 0 496 159 436 51 282 316 241 9103 493 296 348 62 374 467 100 104 301 496 0 425 114 217 202 125 240 9303 214 409 314 223 196 291 377 447 198 159 425 0 304 302 84 405 365 9102 390 152 237 310 152 301 191 90 458 436 114 304 0 416 341 485 280 Basketball Court 233 406 187 389 148 285 99 218 395 51 217 302 416 0 113 336 97 tennis court 162 208 423 297 433 164 294 300 126 282 202 84 341 113 0 55 149 Badminton Court 222 418 133 396 475 86 487 365 271 316 125 405 485 336 55 0 127 Swimming pool 86 96 192 158 70 288 141 376 220 241 240 365 280 97 149 127 0

[0084] therefore, The specific steps of solving this example using the multi-constrained college course scheduling method based on the Q-learning hyper-heuristic algorithm are as follows:

[0085] Step 2: Encoding and decoding.

[0086] Aiming at the characteristics of multi-constraint college course scheduling problem, the present invention adopts a solution individual coding method based on a single column vector, and the course vector V = {c1, c2, ..., c n} represents a solution individual, and the dissatisfaction value minimization strategy is used to generate the course scheduling plan corresponding to the course vector V. The specific description is as follows:

[0087] For the course vector V, we first i First, traverse all available classroom resources to determine whether the current available classroom resources meet the hard constraints shown in equations (10)-(14). If there is a feasible classroom resource set R i If the constraints are met, then course c i Arrange to the classroom r with the closest capacity, that is If all available classroom resources cannot meet all hard constraints, the solution individual V is considered an infeasible solution, and the fitness value corresponding to the solution individual is increased by the penalty value ξ based on formula (6). Repeat the above operation until all courses in the course vector V are assigned;

[0088] Step 3: Initialize the optimization algorithm and Q learning parameters;

[0089] 3.1. Initialize algorithm parameters: maximum number of iterations Itr = 100, discount factor γ = 0.7, number of execution cycles EP = 10, and all data in the Q table are initialized to 0;

[0090] 3.2. Initialize the reinforcement learning action set A, including flipping, forward insertion, backward insertion, 2-opt domain search, random sorting and other actions;

[0091] 3.3. Generate initial solution: randomly generate solution X and calculate its fitness value, and randomly select initial state s t ;

[0092] Step 4. Select the current state and action;

[0093] 4.1. Generate a random number rnd in the range [0,1];

[0094] 4.2. Calculate ε according to formula (15): t value;

[0095]

[0096] 4.3. If the random number rnd is greater than ε t , then select the current state s t The action with the largest Q value is selected, otherwise an action is randomly selected from the action set A;

[0097] Step 5: Apply the selected action to generate a new class schedule;

[0098] 5.1. Execute current action a t A total of EP times, and generate a new solution X′;

[0099] 5.2. Generate a class schedule based on the proposed decoding strategy. Use formula (6) to calculate the fitness value of the new solution X′ and compare it with the old solution X. If X′ is better, replace X.

[0100] Step 6: Update Q table

[0101] 6.1. Calculate r according to formula (16) t value;

[0102]

[0103] In formula (16), u is a random number in [0,1];

[0104] 6.2. Update the state s in the Q table based on equations (17) and (18) t Next action a t The corresponding Q value;

[0105] Q t+1 (s t ,a t )=Q t (s t ,a t )+σ (17)

[0106]

[0107] Step 7: Calculate P according to formula (19) t value, and based on P t The state space constructed by the value

[0108] {(0,0.1),[0.1,0.4),[0.4,0.7),[0.7,1),[1,∞)} Identify the next state s t+1 ;

[0109]

[0110] In formula (19), and are the fitness values ​​of the solution individuals at the beginning and end of period t respectively;

[0111] Step 8: Determine whether the number of iterations t reaches Itr. If not, go to step 4 to continue execution. Otherwise, output the optimal solution X corresponding to the class scheduling plan and its corresponding Gantt chart. The Gantt chart obtained in this embodiment is as follows: Figure 2 shown.

[0112] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that are not conceived through creative work should be included in the protection scope of the present invention; therefore, the protection scope of the present invention should be based on the protection scope defined in the claims.

Claims

1. A multi-constraint college course scheduling method based on Q-learning hyper-heuristic algorithm, characterized in that: The method comprises the following steps: Step 1: Establish a mathematical model; Set up a college with teachers Course Collection Classroom Collection Class collection in Represents the number of teachers, courses, classrooms and classes respectively; the set Indicates the number of days of classes per week, where The number of days per week for classes; Indicates the number of classes per day, is the number of classes per day; U n represents the number of students in class n (n∈N); S r represents the capacity of classroom r (r∈R); Q t,d represents the number of classes that teacher t (t∈T) teaches on the dth (d∈D) day; D n,d,r,r′ represents the distance between classrooms r and r′ where two adjacent courses of class n are located on day d; the decision variables are set as follows: Where c∈C, p∈P; The fitness function with the goal of minimizing the soft constraint is constructed as follows: Among them, formula (6) is the objective function, fit i represents the i-th (i={1,2,3}) soft constraint fitness value, w i is the weight corresponding to each fitness value; Formulas (7)-(9) are soft constraint fitness functions. Formula (7) represents the distance between two classrooms where students take classes consecutively. The closer the distance, the better. Formula (8) represents that the number of classes a teacher teaches in one day shall not exceed the upper limit λ. If the upper limit λ is exceeded, a penalty coefficient δ will be imposed on the number of classes that exceed the upper limit λ. Formula (9) represents that the theory and experiment classes of the same course are arranged on different days. If they are arranged on the same day, they will be multiplied by the penalty coefficient γ. The hard constraints are expressed as follows: IN n ≤S r (13) F d,p ＝1 (14) Formula (10) indicates that a teacher cannot have more than one course in the same time period; Formula (11) indicates that a classroom cannot have more than one course in the same time period; Formula (12) indicates that a class can only have one course in the same time period; Formula (13) indicates that the allocated classroom capacity must not be less than the total number of students in the class; Formula (14) indicates that classes are not allowed to be scheduled in a specific time period; Step 2: Encoding and decoding; The solution based on a single column vector is an individual coding method: the course vector V = {c1, c2, ..., c n } represents a solution individual, and the course scheduling method corresponding to the course vector V is as follows: For the course vector V, we first i Perform resource allocation; first traverse all available classroom resources to determine whether the current available classroom resources meet the hard constraints shown in equations (10)-(14). If there is a feasible classroom resource set R i If the constraints are met, then course c i Arrange to the classroom r with the closest capacity, that is If all available classroom resources cannot meet all hard constraints, the solution individual V is deemed to be an infeasible solution, and the fitness value corresponding to the solution individual is increased by a penalty value ξ based on formula (6); repeat the above operation until all courses in the course vector V are assigned; Step 3: Initialize the optimization algorithm and Q learning parameters; 3.

1. Initialize algorithm parameters: maximum number of iterations Itr, discount factor γ, cycle execution number EP and Q table; 3.

2. Generate initial solution: randomly generate solution X and calculate the fitness value of solution X, and randomly select the initial state s t ; Step 4. Select the current state and action; 4.

1. Generate a random number rnd in the range [0,1]; 4.

2. Calculate ε according to formula (15): t value; 4.

3. If the random number rnd is greater than ε t , then select the current state s t The action with the largest Q value is selected, otherwise an action is randomly selected from the action set A; Step 5: Apply the selected action to generate a new class scheduling method as follows: 5.

1. Execute current action a t A total of EP times, and generate a new solution X′; 5.

2. Use formula (6) to calculate the fitness value of the new solution X′ and compare it with the old solution X. If X′ is better, replace X. Step 6: Update the Q table; 6.

1. Calculate r according to formula (16) t value; In formula (16), u is a random number in [0,1]; 6.

2. Update the state s in the Q table based on equations (17) and (18) t Next action a t The corresponding Q value; Q t+1 (s t ,a t )=Q t (s t ,a t )+σ (17) Step 7: Calculate P according to formula (19) t value, and based on P t The state space constructed by the value {(0,0.1), [0.1,0.4), [0.4,0.7), [0.7,1), [1,∞)} identifies the next state s t+1 ; In formula (19), and are the fitness values ​​of the solution individuals at the beginning and end of period t respectively; Step 8: Determine whether the number of iterations t reaches Itr. If the number of iterations t does not reach Itr, go to step 4 and continue executing until the Gantt chart corresponding to the optimal solution X is output.

2. The multi-constraint college course scheduling method based on Q-learning hyper-heuristic algorithm according to claim 1 is characterized by: Add a step between step 3.1 and step 3.2, which is to initialize the reinforcement learning action set A, including flipping operation, forward insertion, backward insertion, 2-opt field search, and random sorting action.

3. The multi-constraint college course scheduling method based on Q-learning hyper-heuristic algorithm according to claim 1 is characterized by: Constraint (6) in step 1 takes into account three soft constraints: the distance between two classrooms where students have consecutive classes, the number of classes a teacher has on the same day, and whether the theory and experimental classes of the same course are scheduled on the same day. The fitness function is set based on these soft constraints.

Images