Multi-objective optimization scheduling method based on improved genetic algorithm and reinforcement learning and application thereof
By improving the fusion of genetic algorithms and reinforcement learning, the task and resource allocation are dynamically optimized to generate Pareto optimal solution sets. This solves the problems of slow convergence speed and insufficient adaptability in traditional scheduling methods, and achieves efficient and flexible multi-objective optimization scheduling.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANGHAI OCEAN UNIV
- Filing Date
- 2026-04-02
- Publication Date
- 2026-07-24
AI Technical Summary
Traditional genetic algorithms have slow convergence speed and are prone to getting trapped in local optima in multi-objective optimization scheduling problems, and they are difficult to handle complex constraints. Traditional reinforcement learning requires a lot of training time in multi-objective optimization problems and the quality of solutions needs to be improved.
By combining improved genetic algorithms and reinforcement learning, a global task priority sequence is generated by dynamically calculating task priority scores. A reinforcement learning mechanism is introduced to adjust the population, optimize task and resource allocation, and generate a Pareto optimal solution set through state feature extraction and environment selection mechanisms.
It improves the rationality and efficiency of scheduling schemes, ensures the global optimality and adaptability of scheduling results, and can respond to environmental changes in real time, providing flexible and efficient solutions for complex scheduling problems.
Smart Images
Figure CN122452983A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of multi-objective optimization scheduling technology, specifically to a multi-objective optimization scheduling method and its application based on the fusion of improved genetic algorithms and reinforcement learning. Background Technology
[0002] In modern complex systems, multi-objective optimization scheduling problems are widespread across various fields, such as production line scheduling, cloud computing resource management, logistics vehicle routing planning, and power system economic and environmental scheduling. These fields typically involve multiple conflicting objectives, such as minimizing completion time, reducing costs, improving resource utilization, reducing energy consumption, and enhancing customer satisfaction. In recent years, genetic algorithms, as an effective global optimization algorithm, have been widely applied to solve multi-objective optimization scheduling problems. Reinforcement learning, as an intelligent decision-making method, learns optimal strategies through the interaction between agents and the environment, and has achieved significant results in many complex systems.
[0003] Traditional scheduling methods often struggle to optimize multiple objectives simultaneously, especially when faced with complex constraints and dynamically changing environments. Genetic algorithms, by simulating natural selection and genetic mutation processes, can effectively explore the solution space and find approximate optimal solutions. However, traditional genetic algorithms have some limitations when dealing with multi-objective optimization problems, such as slow convergence speed, susceptibility to getting trapped in local optima, and difficulty in effectively handling complex constraints.
[0004] Reinforcement learning can dynamically adjust policies to adapt to changes in the environment, exhibiting good adaptability and flexibility. However, traditional reinforcement learning typically requires a large amount of training time and data support when solving multi-objective optimization problems, and its convergence speed and solution quality still need improvement in complex scheduling problems. Summary of the Invention
[0005] The purpose of this invention is to provide a multi-objective optimization scheduling method and application based on the fusion of improved genetic algorithm and reinforcement learning, which can improve the rationality and efficiency of scheduling schemes, better cover the solution space, ensure the global optimality and adaptability of scheduling results, and solve the problems mentioned in the background art.
[0006] To achieve the above objectives, the present invention provides the following technical solution:
[0007] A multi-objective optimization scheduling method based on the fusion of improved genetic algorithms and reinforcement learning includes:
[0008] The parameters for initializing the scheduling problem include the task set, the resource set, and the characteristic information of each task and the attribute information of the resources;
[0009] Based on the parameters of the target scheduling problem, an optimization model is defined that includes at least two optimization objectives; the parameters include a task set, a resource set, and their respective characteristics and attributes; the optimization model includes decision variables, constraints, and at least two objective functions.
[0010] Based on the optimization model, an improved genetic algorithm is used to initialize a population with multiple individuals, and the individuals in the population are genetically encoded; the genetic encoding is used to represent the allocation relationship between tasks and resources.
[0011] Genetic operations, including selection, crossover, and mutation, are performed on the initial population to generate a new population;
[0012] By introducing a reinforcement learning mechanism, during the population evolution process of the genetic algorithm, the reinforcement learning agent selects corresponding actions based on the current state of the population to adjust the individuals in the population, thereby accelerating the convergence speed of the population and improving the quality of the solution.
[0013] The fitness of the newly generated population is evaluated, and the fitness value of each individual is calculated according to the objective function of the multi-objective optimization scheduling model.
[0014] Determine if the termination condition is met. If it is, output the optimized scheduling result; otherwise, continue with population evolution.
[0015] Preferably, an improved genetic algorithm is used to initialize a population with multiple individuals, including:
[0016] A task priority-based coding strategy is introduced. For each task in the task set, a comprehensive priority score is dynamically calculated in a weighted manner based on its urgency factor, resource requirement factor, and task dependency factor.
[0017] The tasks are sorted according to their overall priority scores to generate a global task priority sequence; this sequence is used as the basic structure for individual coding or to guide code generation.
[0018] Task priority scores are converted into priority allocation rules, and each encoded individual consists of a sequence of priority allocation rules for online scheduling decisions.
[0019] The encoding strategy based on priority allocation rules encodes the genes of individuals in the population to generate the initial population; and executes a population evolution loop with a fusion reinforcement learning mechanism until the termination condition is met, outputting the optimal scheduling scheme set.
[0020] Preferably, the urgency factor is quantified based on the reciprocal of the task deadline and / or the delay penalty coefficient; the resource demand factor is quantified based on the type, quantity, total cost, or resource scarcity index of the required resources; and the impact of the task dependency factor is quantified based on the depth of the task in the dependency graph, the number of subsequent tasks, or their impact on the total project duration.
[0021] Preferably, an agent utilizing reinforcement learning selects appropriate actions based on the current state of the population to adjust individuals within the population, including:
[0022] At the beginning of each generation of evolution, the state features of the current population are extracted to form a state vector; this state vector includes: convergence features, diversity features, frontier features and problem-specific features;
[0023] Perform genetic operations on the current population to generate an intermediate offspring population; invoke a pre-trained reinforcement learning agent to output an enhancement action based on the input state; the enhancement action is a heuristic rule selection, in which the current generation prioritizes the rule of assigning the longest time-consuming task to the earliest idle machine, and decodes and corrects 50% of the individuals in the intermediate offspring population.
[0024] Based on the actions output by the agent, individuals in the intermediate offspring population are adjusted and optimized in a targeted manner to generate an enhanced offspring population.
[0025] The current population is merged with the enhanced offspring population, and a new generation of population is generated through environmental selection mechanisms.
[0026] The new population state is used as the input for the next round. It is then determined whether the termination condition is met. If it is met, the Pareto optimal solution set in the new generation of the population is output as the final scheduling scheme. Otherwise, the population evolution continues.
[0027] Preferably, the convergence features include the current generation's optimal fitness value, average fitness value, and improvement rate compared to the previous generation; the diversity features include the mean Hamming distance of the population genotypes and the distribution entropy of the phenotypes in the target space; the frontier features include the number of non-dominated solutions of the Pareto front, the distribution uniformity of the frontier, and the expansion range of the frontier; and the problem-specific features include the utilization rate of critical resources and the average completion time of high-priority tasks.
[0028] Preferably, the termination condition is reaching the maximum number of generations or the Hypervolume index showing no significant improvement for N consecutive generations.
[0029] Preferably, the target scheduling problem is production workshop operation scheduling, cloud computing resource scheduling, logistics vehicle route planning, or power system economic environment scheduling; the optimization objective is at least two of the following: completion time, total cost, energy consumption, resource utilization rate, and customer satisfaction.
[0030] According to another aspect of the present invention, an application of the multi-objective optimization scheduling method based on the fusion of improved genetic algorithm and reinforcement learning as described above is provided, wherein the multi-objective optimization scheduling method based on the fusion of improved genetic algorithm and reinforcement learning is applied to multi-objective optimization scheduling decision-making.
[0031] Compared with the prior art, the beneficial effects of the present invention are:
[0032] 1. This invention generates a global task priority sequence by dynamically calculating the comprehensive priority score of tasks, optimizes the allocation relationship between tasks and resources, and ensures that high-priority tasks are processed first during scheduling, thereby improving the rationality and efficiency of the scheduling scheme; the task priority score is transformed into priority allocation rules, and gene encoding is performed based on these rules, which not only enhances the targeting and guidance of the encoding, but also makes the generated initial population more representative and better covers the solution space.
[0033] 2. This invention extracts detailed state features of the population at the beginning of each generation of evolution, forming a state vector that includes convergence, diversity, frontier features, and problem-specific features, enabling a comprehensive assessment of the current population's evolutionary state. Furthermore, it generates a new generation of population through an environment selection mechanism and outputs a Pareto optimal solution set as the final scheduling scheme when the termination condition is met, ensuring the global optimality and adaptability of the scheduling results. This ensures real-time response to environmental changes, providing a flexible, efficient, and high-quality solution for complex scheduling problems. Attached Figure Description
[0034] Figure 1 The flowchart shows the multi-objective optimization scheduling method based on the fusion of improved genetic algorithm and reinforcement learning of the present invention. Detailed Implementation
[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0036] To address the shortcomings of traditional genetic algorithms in handling multi-objective optimization problems, such as slow convergence, susceptibility to local optima, and difficulty in effectively handling complex constraints; and the limitations of traditional reinforcement learning in solving multi-objective optimization problems, such as the need for significant training time and data support, and the remaining need for improvement in convergence speed and solution quality in complex scheduling problems, please refer to [link to relevant documentation]. Figure 1 This embodiment provides the following technical solution:
[0037] A multi-objective optimization scheduling method based on the fusion of improved genetic algorithms and reinforcement learning includes:
[0038] The parameters for initializing the scheduling problem include the task set, the resource set, and the characteristic information of each task and the attribute information of the resources;
[0039] Based on the parameters of the target scheduling problem, an optimization model is defined that includes at least two optimization objectives; the parameters include a task set, a resource set, and their respective characteristics and attributes; the optimization model includes decision variables, constraints, and at least two objective functions.
[0040] It should be noted that the target scheduling problem refers to production workshop operation scheduling, cloud computing resource scheduling, logistics vehicle route planning, or power system economic environment scheduling.
[0041] It should be noted that the optimization objectives are at least two of the following: completion time, total cost, energy consumption, resource utilization rate, and customer satisfaction.
[0042] Based on the optimization model, an improved genetic algorithm is used to initialize a population with multiple individuals, and the individuals in the population are genetically encoded; the genetic encoding is used to represent the allocation relationship between tasks and resources.
[0043] It should be noted that an improved genetic algorithm is used to initialize a population with multiple individuals, including:
[0044] A task priority-based coding strategy is introduced. For each task in the task set, a comprehensive priority score is dynamically calculated in a weighted manner based on its urgency factor, resource requirement factor, and task dependency factor.
[0045] The tasks are sorted according to their overall priority scores to generate a global task priority sequence; this sequence is used as the basic structure for individual coding or to guide code generation.
[0046] Task priority scores are converted into priority allocation rules, and each encoded individual consists of a sequence of priority allocation rules for online scheduling decisions.
[0047] The encoding strategy based on priority allocation rules encodes the genes of individuals in the population to generate the initial population; and executes a population evolution loop with a fusion reinforcement learning mechanism until the termination condition is met, outputting the optimal scheduling scheme set.
[0048] It should be noted that the urgency factor is quantified based on the reciprocal of the task deadline and / or the delay penalty coefficient.
[0049] It should be noted that the quantification of the resource demand factor is based on the type, quantity, total cost, or resource scarcity index of the required resources.
[0050] It should be noted that the impact metric of the task dependency factor is based on the depth of the task in the dependency graph, the number of subsequent tasks, or its impact on the total project duration.
[0051] Genetic operations, including selection, crossover, and mutation, are performed on the initial population to generate a new population;
[0052] By introducing a reinforcement learning mechanism, during the population evolution process of the genetic algorithm, the reinforcement learning agent selects corresponding actions based on the current state of the population to adjust the individuals in the population, thereby accelerating the convergence speed of the population and improving the quality of the solution.
[0053] The fitness of the newly generated population is evaluated, and the fitness value of each individual is calculated according to the objective function of the multi-objective optimization scheduling model.
[0054] Determine if the termination condition is met. If it is, output the optimized scheduling result; otherwise, continue with population evolution.
[0055] Specifically, by dynamically calculating the comprehensive priority score of tasks, a global task priority sequence is generated, optimizing the allocation relationship between tasks and resources, ensuring that high-priority tasks are processed first during scheduling, thereby improving the rationality and efficiency of the scheduling scheme; the task priority score is transformed into priority allocation rules, and gene encoding is performed based on these rules, which not only enhances the targeting and guidance of the encoding, but also makes the generated initial population more representative and better covers the solution space.
[0056] In this embodiment, a reinforcement learning agent selects appropriate actions based on the current state of the population to adjust individuals within the population, including:
[0057] At the beginning of each generation of evolution, state features are extracted from the current population to form a state vector. This state vector includes: convergence features (e.g., the current generation's optimal fitness value, average fitness value, and improvement rate compared to the previous generation), diversity features (e.g., the mean Hamming distance of the population's genotypes (encoded), and the distribution entropy of phenotypes (solutions) in the target space), frontier features (e.g., the number of non-dominated solutions on the Pareto front, the uniformity of the front's distribution (e.g., the spacing index), and the extent of the front's expansion), and problem-specific features (e.g., the utilization rate of critical resources, the average completion time of high-priority tasks, etc.).
[0058] Perform genetic operations on the current population to generate an intermediate offspring population; invoke a pre-trained reinforcement learning agent to output an enhancement action based on the input state; this enhancement action can be a heuristic rule selection, for example: this generation prioritizes the rule of assigning the longest-running task to the earliest idle machine, and performs decoding correction on 50% of the individuals in the intermediate offspring population.
[0059] Based on the actions output by the agent, individuals in the intermediate offspring population are adjusted and optimized in a targeted manner to generate an enhanced offspring population.
[0060] The current population is merged with the enhanced offspring population, and a new generation of population is generated through environmental selection mechanisms.
[0061] The new population state is used as the input for the next round. It is then determined whether the termination condition is met (e.g., reaching the maximum number of generations, or the Hypervolume index showing no significant improvement for N consecutive generations). If the condition is met, the Pareto optimal solution set in the new generation of the population is output as the final scheduling scheme. Otherwise, the population evolution continues.
[0062] Specifically, by dynamically calculating the comprehensive priority score of tasks and generating a task priority sequence, this sequence is transformed into priority allocation rules for gene encoding, thereby generating an initial population. The population then evolves using a reinforcement learning mechanism to output the optimal scheduling scheme set. The reinforcement learning agent dynamically selects optimization actions based on the multi-dimensional state characteristics of the population, making targeted adjustments and optimizations to the intermediate offspring population. Combined with an environmental selection mechanism, a new generation of population is generated until the termination condition is met, outputting the Pareto optimal solution set. This achieves an efficient and dynamically adaptive population evolution scheduling strategy.
[0063] It should be noted that by extracting detailed state features of the population at the beginning of each generation of evolution, a state vector containing convergence, diversity, frontier features, and problem-specific features is formed, which can comprehensively evaluate the current evolutionary state of the population; and by generating a new generation of population through an environment selection mechanism, and outputting a Pareto optimal solution set as the final scheduling scheme when the termination condition is met, the global optimality and adaptability of the scheduling results are ensured; and the ability to respond to environmental changes in real time is ensured, providing a flexible, efficient, and high-quality solution for complex scheduling problems.
[0064] To better demonstrate the application of the multi-objective optimization scheduling method based on the fusion of improved genetic algorithm and reinforcement learning, this embodiment provides an application of the multi-objective optimization scheduling method based on the fusion of improved genetic algorithm and reinforcement learning as described above. Specifically, the multi-objective optimization scheduling method based on the fusion of improved genetic algorithm and reinforcement learning is applied to multi-objective optimization scheduling decision-making.
[0065] In summary, by dynamically calculating the comprehensive priority score of tasks, a global task priority sequence is generated, optimizing the allocation relationship between tasks and resources, and ensuring that high-priority tasks are processed first during scheduling, thereby improving the rationality and efficiency of the scheduling scheme. The task priority score is transformed into priority allocation rules, and gene encoding is performed based on these rules. This not only enhances the targeting and guidance of the encoding but also makes the generated initial population more representative and better covers the solution space. At the beginning of each generation of evolution, detailed state features of the population are extracted, forming a state vector containing convergence, diversity, frontier features, and problem-specific features, enabling a comprehensive assessment of the current population's evolutionary state. Furthermore, a new generation of population is generated through an environment selection mechanism, and when the termination condition is met, a Pareto optimal solution set is output as the final scheduling scheme, ensuring the global optimality and adaptability of the scheduling results. Finally, the system ensures real-time response to environmental changes, providing a flexible, efficient, and high-quality solution for complex scheduling problems.
[0066] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0067] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A multi-objective optimization scheduling method based on the fusion of improved genetic algorithm and reinforcement learning, characterized in that, include: The parameters for initializing the scheduling problem include the task set, the resource set, and the characteristic information of each task and the attribute information of the resources; Based on the parameters of the target scheduling problem, an optimization model is defined that includes at least two optimization objectives; the parameters include a task set, a resource set, and their respective characteristics and attributes; the optimization model includes decision variables, constraints, and at least two objective functions. Based on the optimization model, an improved genetic algorithm is used to initialize a population with multiple individuals, and the individuals in the population are genetically encoded; the genetic encoding is used to represent the allocation relationship between tasks and resources. Genetic operations, including selection, crossover, and mutation, are performed on the initial population to generate a new population; By introducing a reinforcement learning mechanism, during the population evolution process of the genetic algorithm, the reinforcement learning agent selects corresponding actions based on the current state of the population to adjust the individuals in the population, thereby accelerating the convergence speed of the population and improving the quality of the solution. The fitness of the newly generated population is evaluated, and the fitness value of each individual is calculated according to the objective function of the multi-objective optimization scheduling model. Determine if the termination condition is met. If it is, output the optimized scheduling result; otherwise, continue with population evolution.
2. The multi-objective optimization scheduling method based on the fusion of improved genetic algorithm and reinforcement learning according to claim 1, characterized in that, An improved genetic algorithm is used to initialize a population with multiple individuals, including: A task priority-based coding strategy is introduced. For each task in the task set, a comprehensive priority score is dynamically calculated in a weighted manner based on its urgency factor, resource requirement factor, and task dependency factor. The tasks are sorted according to their overall priority scores to generate a global task priority sequence; this sequence is used as the basic structure for individual coding or to guide code generation. Task priority scores are converted into priority allocation rules, and each encoded individual consists of a sequence of priority allocation rules for online scheduling decisions. The encoding strategy based on priority allocation rules encodes the genes of individuals in the population to generate the initial population; and executes a population evolution loop with a fusion reinforcement learning mechanism until the termination condition is met, outputting the optimal scheduling scheme set.
3. The multi-objective optimization scheduling method based on the fusion of improved genetic algorithm and reinforcement learning according to claim 2, characterized in that, The urgency factor is quantified based on the reciprocal of the task deadline and / or the delay penalty coefficient; the resource demand factor is quantified based on the type, quantity, total cost, or resource scarcity index of the required resources; the impact of the task dependency factor is quantified based on the depth of the task in the dependency graph, the number of subsequent tasks, or its impact on the total project duration.
4. The multi-objective optimization scheduling method based on the fusion of improved genetic algorithm and reinforcement learning according to claim 3, characterized in that, Using a reinforcement learning agent, appropriate actions are selected based on the current state of the population to adjust the individuals within the population, including: At the beginning of each generation of evolution, the state features of the current population are extracted to form a state vector; this state vector includes: convergence features, diversity features, frontier features and problem-specific features; Perform genetic operations on the current population to generate an intermediate offspring population; invoke a pre-trained reinforcement learning agent to output an enhancement action based on the input state; the enhancement action is a heuristic rule selection, in which the current generation prioritizes the rule of assigning the longest time-consuming task to the earliest idle machine, and decodes and corrects 50% of the individuals in the intermediate offspring population. Based on the actions output by the agent, individuals in the intermediate offspring population are adjusted and optimized in a targeted manner to generate an enhanced offspring population. The current population is merged with the enhanced offspring population, and a new generation of population is generated through environmental selection mechanisms. The new population state is used as the input for the next round. It is then determined whether the termination condition is met. If it is met, the Pareto optimal solution set in the new generation of the population is output as the final scheduling scheme. Otherwise, the population evolution continues.
5. The multi-objective optimization scheduling method based on the fusion of improved genetic algorithm and reinforcement learning according to claim 4, characterized in that, The convergence characteristics include the current generation's optimal fitness value, average fitness value, and improvement rate compared to the previous generation; the diversity characteristics include the mean Hamming distance of the population genotypes and the distribution entropy of phenotypes in the target space. Frontier characteristics include the number of nondominated solutions to the Pareto front, the uniformity of the front's distribution, and the extent of the front's expansion; problem-specific characteristics include the utilization rate of critical resources and the average completion time of high-priority tasks.
6. The multi-objective optimization scheduling method based on the fusion of improved genetic algorithm and reinforcement learning according to claim 5, characterized in that, The termination condition is reaching the maximum number of generations or the Hypervolume index showing no significant improvement for N consecutive generations.
7. The multi-objective optimization scheduling method based on the fusion of improved genetic algorithm and reinforcement learning according to claim 6, characterized in that, The target scheduling problem is production workshop operation scheduling, cloud computing resource scheduling, logistics vehicle route planning, or power system economic environment scheduling; the optimization objective is at least two of the following: completion time, total cost, energy consumption, resource utilization rate, and customer satisfaction.
8. An application of the multi-objective optimization scheduling method based on the fusion of improved genetic algorithm and reinforcement learning as described in any one of claims 1-7, characterized in that, The proposed multi-objective optimization scheduling method based on the fusion of improved genetic algorithm and reinforcement learning is applied to multi-objective optimization scheduling decision-making.