A competitive evolutionary multi-task optimization method, system and device

Abstract

The application discloses a kind of competitive evolution multitask optimization method, system and equipment, to solve the problem that existing method is difficult to carry out collaborative optimization to fuel cost and gas emission under meeting power balance and unit capacity constraint condition.The method comprises the following steps: constructing main, auxiliary double population parallel search;Real-time calculation state index representing population convergence, diversity and feasibility;Based on the current state, through deep reinforcement learning agent self-adapting selection cooperation mode and evolution operator combination action;According to the improvement of population after action execution in cost, emission and constraint satisfaction, calculate reward;State, action and reward are used to train agent, iteratively optimize decision strategy, until the pareto optimal scheduling scheme that meets the constraint is output.The application realizes the adaptive intelligent guidance of evolution process, significantly improves the optimization quality, convergence speed and algorithm robustness of power system economic emission scheduling scheme.

Description

Technical Field

[0001] This invention relates to the field of power system operation scheduling and intelligent optimization control technology, and in particular to a competitive evolutionary multi-task optimization method, system and device. This method combines evolutionary computation and deep reinforcement learning techniques to generate scheduling schemes under engineering constraints. Background Technology

[0002] In practical applications such as power system operation, energy dispatch, and engineering system design, it is often necessary to optimize multiple conflicting performance indicators while meeting various engineering constraints. For example, in the process of joint economic emission dispatch of power systems, it is necessary to reduce system operating costs and pollutant emission levels while meeting power balance constraints and generator capacity constraints.

[0003] Such problems can usually be abstracted as Constrained Multi-objective Optimization Problems (CMOPs). Their feasible solution space is limited by physical constraints and engineering conditions, and often exhibits characteristics such as a small feasible region, discontinuous distribution, and complex form, which makes the search and generation process of scheduling schemes face great engineering implementation difficulties.

[0004] While existing evolutionary multi-objective optimization methods can generate candidate solution sets that satisfy constraints to a certain extent, they still have significant shortcomings in practical engineering scheduling applications. On the one hand, traditional methods usually rely on a single evolutionary operator or a fixed population cooperation mechanism, making it difficult to achieve a dynamic balance between convergence speed, uniformity of solution set distribution, and constraint satisfaction based on changes in the system's operating state. This can easily lead to unstable quality of the generated scheduling schemes, and even a large number of infeasible schemes, affecting the operational safety of the engineering system.

[0005] To improve the efficiency of solving complex scheduling problems, the Evolutionary Multi-tasking (EMT) framework provides a feasible approach to accelerate the search process by transferring knowledge between multiple related optimization tasks. However, existing multi-tasking evolutionary methods still face two technical bottlenecks in engineering applications: First, the knowledge transfer process lacks effective perception of the system's operating state and intelligent decision-making mechanisms, which can easily introduce invalid or even negative transfer results, thus wasting computational resources and reducing scheduling efficiency. Second, the selection of evolutionary operators and cooperation strategies usually relies on human experience or preset rules, making it difficult to adapt to the dynamic needs of complex operating scenarios such as changes in power system load and constraints.

[0006] Although the competitive evolutionary multitasking framework has improved the diversity and search capability of scheduling schemes to some extent by introducing an auxiliary population and a cooperation mechanism of multiple evolutionary operators, in the absence of a unified intelligent decision-making mechanism, there are still problems such as unreasonable selection of cooperation methods and low efficiency of operator use. This leads to uneven allocation of computing resources and unstable performance in meeting engineering constraints during the scheduling process, making it difficult to meet the reliability and stability requirements of actual engineering scenarios such as joint economic emission scheduling of power systems.

[0007] Therefore, how to dynamically perceive the scheduling process status, intelligently select cooperation methods and evolution strategies under complex engineering constraints, and generate a scheduling scheme that meets engineering constraints, has excellent performance indicators and is stable, has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0008] To address the challenge of achieving stable and efficient joint optimization across multiple performance metrics in existing engineering scheduling scenarios while satisfying various engineering constraints, this invention aims to provide a competitive evolutionary multi-task optimization method, system, and device for constrained multi-objective scheduling problems. This method introduces a deep reinforcement learning mechanism to adaptively regulate the cooperation methods and reproduction strategies during the evolutionary process, thereby generating scheduling schemes that meet engineering operation requirements under complex constraints and improving the stability, feasibility, and overall operational performance of the scheduling process. The technical solution provided by this invention is as follows:

[0009] A competitive evolutionary multi-task optimization method is provided to solve the joint economic emission dispatch problem of a power system. This dispatch problem is a constrained multi-objective optimization problem that jointly optimizes fuel cost and gas emissions while satisfying power balance constraints and generator capacity constraints. The method includes the following steps:

[0010] S1. Construct a main population and an auxiliary population for parallel solving of constrained multi-objective optimization problems, where each population individual represents a set of candidate solutions for generator output allocation that satisfy the engineering constraints;

[0011] S2. Based on the objective function values ​​and constraint violation degrees of the candidate solutions in the main population and auxiliary population, calculate and generate indices characterizing the optimization states of the two populations, including convergence indices, diversity indices, and feasibility indices; wherein, the objective function values ​​include fuel cost values ​​and gas emission values ​​calculated based on the candidate solutions;

[0012] S3. Based on the current optimization state, a deep reinforcement learning agent selects and executes an action from a set of preset combinations of cooperation methods and reproduction operators to update the candidate solutions of the main population and the auxiliary population.

[0013] S4. Based on the change in the overall performance of the population after the action is performed, calculate the reward that reflects the degree of improvement in scheduling performance. The overall performance of the population is calculated by comprehensively considering the convergence index, diversity index, and feasibility index.

[0014] S5. Using the optimized state, the action performed, and the reward, train the deep reinforcement learning agent to update the decision-making strategy used to guide the selection of subsequent actions;

[0015] Iteratively execute steps S2 to S5 until the preset termination condition is met, and output the Pareto optimal solution set of the scheduling problem as the generator output allocation scheme.

[0016] Preferably, the cooperation method includes: tightly coupled cooperation and loosely coupled cooperation;

[0017] The tightly coupled cooperation exchanges solutions in each iteration, while the loosely coupled cooperation exchanges solutions only during the environment selection phase.

[0018] Preferably, the operators in the set of reproductive operators include genetic algorithm operators, a first differential evolution operator, and a second differential evolution operator.

[0019] Preferably, the convergence index is calculated based on the changes between the ideal point and the worst point between adjacent iterations;

[0020] The diversity index is calculated based on the reciprocal of the average Euclidean distance between a solution and its neighboring solutions;

[0021] The feasibility index is calculated based on the average of the constraint violation values ​​of all solutions in the population.

[0022] Preferably, the step of calculating the reward reflecting the degree of improvement in scheduling performance based on the change in the overall performance of the population after performing the action specifically involves:

[0023] If the overall performance of the population improves in the current iteration compared to the previous iteration, a first positive reward value is assigned.

[0024] If the overall performance of the population remains unchanged in the current iteration compared to the previous iteration, then zero reward or the second smallest reward value is assigned.

[0025] If the overall performance of the population declines in the current iteration compared to the previous iteration, a negative reward value is assigned.

[0026] Preferably, training the deep reinforcement learning agent using the optimized state, the executed action, and the reward specifically involves:

[0027] An experience replay mechanism is used to train a deep Q-network by minimizing the loss function, the optimized state, the action, and the reward, wherein the deep reinforcement agent is specifically the deep Q-network;

[0028] The loss function formula is as follows:

[0029] ;

[0030] in, This represents the numerical value of the loss function. This represents the sampled training data. and They represent the first The state and actions in the next iteration. Indicates will and The output after inputting the action value function. Indicates the state Take action below Q value;

[0031] Get The formula is:

[0032] ;

[0033] in, Indicates the first In the next iteration, in the state Take action below The rewards received Indicates the discount factor. Indicates the state Next steps The maximum Q value that can be obtained.

[0034] Preferably, it further includes:

[0035] During each iteration, when the deep reinforcement agent has not completed training, the action is randomly selected from the preset set of combinations of cooperation methods and reproduction operators, and the selection result is stored in the experience replay pool.

[0036] Once the deep reinforcement agent has been trained, the action is selected based on the current optimization state of the trained deep reinforcement agent.

[0037] Preferably, the formula for obtaining the fuel cost is:

[0038] ;

[0039] in, For total fuel cost, The total number of generators. For generator index, For the first The output power of the generator , , The first Cost coefficient of a generator;

[0040] The formula for obtaining the gas emission value is:

[0041] ;

[0042] in, Total gas emissions, , , The first The emission coefficient of the generator;

[0043] The formula for the optimization objective of the scheduling problem is:

[0044] ;

[0045] in, and To optimize the objective, For total power load demand, This represents the actual total power generation. For the first The generator is allowed to generate the minimum power. For the first The maximum allowable power output of the generator.

[0046] A competitive evolutionary multi-task optimization system for performing any of the methods described above, for solving a power system joint economic emission dispatch problem, wherein the dispatch problem is a constrained multi-objective optimization problem that jointly optimizes fuel cost and gas emissions under the conditions of satisfying power balance constraints and generator capacity constraints, comprising:

[0047] The population construction module is used to construct the main population and auxiliary population for parallel solving of constrained multi-objective optimization problems. Each population individual represents a set of candidate solutions for generator output allocation that satisfy the engineering constraints.

[0048] The state generation module is used to calculate and generate indices characterizing the optimization states of the two populations based on the objective function values ​​and constraint violation degrees of the candidate solutions in the main population and auxiliary population. The indices include convergence indices, diversity indices, and feasibility indices. The objective function values ​​include fuel cost values ​​and gas emission values ​​calculated based on the candidate solutions.

[0049] The action selection module is used to select and execute an action from a set of preset combinations of cooperation methods and reproduction operators to update the candidate solutions of the main population and the auxiliary population based on the current optimization state through a deep reinforcement learning agent.

[0050] The reward calculation module is used to calculate a reward reflecting the degree of improvement in scheduling performance based on the change in the overall performance of the population after the action is performed. The overall performance of the population is calculated by comprehensively considering the convergence index, diversity index, and feasibility index.

[0051] The model training module is used to train the deep reinforcement learning agent using the optimized state, the action performed, and the reward, so as to update the decision-making strategy used to guide the selection of subsequent actions.

[0052] The population construction module, the state generation module, the action selection module, the reward calculation module, and the model training module are iteratively run until a preset termination condition is met, and the Pareto optimal solution set of the scheduling problem is output as the generator output allocation scheme.

[0053] A competitive evolutionary multitasking optimization device, comprising:

[0054] One or more processors;

[0055] A memory that stores computer programs;

[0056] When the computer program is executed by the processor, it implements any of the above-described competitive evolutionary multitasking optimization methods.

[0057] This invention, through a closed loop of "state perception → intelligent decision-making → effect evaluation → model evolution", can respond according to the real-time status (convergence, diversity, feasibility) of the optimization process. For different problems and different optimization stages of the same problem, the system can automatically adjust to the most suitable collaboration method and evolution operator. The DRL model learns through long-term cumulative rewards, and its decision is based on maximizing long-term benefits, avoiding local optima that may be caused by greedy algorithms.

[0058] The present invention also discloses a competitive evolutionary multi-task optimization system and device. Since it belongs to the same technical concept as the method and solves the same technical problem, it should have the same beneficial effects, and will not be described in detail here. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a flowchart of a competitive evolutionary multi-task optimization method provided in an embodiment of the present invention;

[0061] Figure 2 This is a flowchart of the iterative training strategy provided in the embodiments of the present invention;

[0062] Figure 3 This is a schematic diagram of the structure of a competitive evolutionary multi-task optimization system provided in an embodiment of the present invention. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] The embodiments of this invention are written in a progressive manner.

[0065] This invention provides a competitive evolutionary multi-task optimization method for constrained multi-objective scheduling problems. This method introduces a deep reinforcement learning mechanism to adaptively regulate the cooperation mode and reproduction strategy in the evolution process, thereby generating scheduling parameter schemes under the premise of meeting engineering constraints. This solves the problems of unreasonable resource allocation and lack of adaptive strategy selection in existing scheduling methods.

[0066] like Figure 1 As shown, a competitive evolutionary multi-task optimization method includes the following steps:

[0067] S1. Construct a main population and an auxiliary population for parallel solving of constrained multi-objective optimization problems, where each population individual represents a set of candidate solutions for generator output allocation that satisfy the engineering constraints;

[0068] S2. Based on the objective function values ​​and constraint violation degrees of candidate solutions in the main population and auxiliary population, calculate and generate indices characterizing the optimization state of the two populations. The indices include convergence indices, diversity indices, and feasibility indices. The objective function values ​​include fuel cost values ​​and gas emission values ​​calculated based on candidate solutions.

[0069] S3. Based on the current optimization state, a deep reinforcement learning agent selects and executes an action from a set of preset combinations of cooperation methods and reproduction operators to update the candidate solutions of the main population and the auxiliary population.

[0070] In this embodiment, the “cooperation mode and reproduction operator combination set” consists of a variety of predefined combination actions. Each combination action is used to describe the cooperation strategy between the main population and the auxiliary population and the generation method of candidate scheduling parameter schemes during an evolutionary iteration.

[0071] Specifically, each combined action consists of a cooperation mode and a reproduction operator. The cooperation mode is used to define the information interaction rules between the main population and the auxiliary population in the current iteration cycle, and the reproduction operator is used to define the generation and updating method of the candidate scheduling parameter scheme under the cooperation mode.

[0072] During an iteration, once a certain combination of actions is selected, candidate scheduling parameter schemes are first exchanged or information is shared between the main population and the auxiliary population according to the selected cooperation method. Then, based on the cooperation results, the corresponding reproduction operator is used to generate and update the candidate scheduling parameter schemes, thereby forming a new set of candidate scheduling parameter schemes for subsequent scheduling performance evaluation and state update.

[0073] By uniformly selecting and executing the cooperation method and the reproduction operator in the form of combined actions, the intensity of information interaction between populations and the generation strategy of scheduling schemes can be flexibly adjusted under different optimization states, so as to achieve a dynamic balance between global search capability and local development capability in the scheduling process, thereby better adapting to the scheduling needs under different operating stages and engineering constraints.

[0074] S4. Based on the change in the overall performance of the population after the action is performed, calculate the reward that reflects the degree of improvement in scheduling performance. The overall performance of the population is calculated by combining convergence index, diversity index and feasibility index.

[0075] S5. Train the deep reinforcement learning agent using the optimized state, the actions performed, and the rewards to update the decision-making strategy used to guide the selection of subsequent actions;

[0076] Iteratively execute steps S2 to S5 until the preset termination condition is met, and output the Pareto optimal solution set of the scheduling problem as the generator output allocation scheme.

[0077] The constrained multi-objective optimization problem in step S1 refers to a mathematical problem that simultaneously optimizes two or more conflicting objective functions while satisfying one or more constraints. The main population is the set of candidate solutions in this optimization method that is responsible for directly solving the original and complete constrained multi-objective optimization problem. The auxiliary population is the set of candidate solutions in this optimization method that is constructed to assist the main population in its search and is used to solve a simplified auxiliary problem related to the main task.

[0078] Convergence in step S2 refers to the ability and degree to which the solution set generated by the optimization algorithm approximates the true Pareto optimal front of the problem. It measures how close the solution set is to the ideal optimal solution in the target space. Diversity refers to the uniformity and breadth of the distribution of the non-dominated solution set found by the optimization algorithm in the target space. It measures whether the solution set can cover all parts of the Pareto front, reflecting the richness of solutions and the scope of exploration. Feasibility refers to the degree to which the candidate solutions generated by the optimization algorithm satisfy all given constraints. It is the core concept that distinguishes constrained optimization problems from unconstrained problems. It measures whether a solution is located within the feasible region. The optimization state is a vector or information set composed of one or more quantitative indicators used to comprehensively characterize the current search status of the population during the iteration process of the optimization algorithm.

[0079] In step S3, the cooperation method refers to the rules and frequency followed by the knowledge transfer and information exchange between the main population and the auxiliary population in the competitive evolutionary multi-task framework; the set of reproductive operator combinations refers to the set of different evolutionary operation strategies that can be selected in the optimization algorithm to generate new candidate solutions; the action refers to a specific choice made by the deep reinforcement agent (hereinafter referred to as DQN, or "DQN" or "deep Q network") based on the current optimization state from the preset cooperation method and the set of reproductive operator combinations. It represents the evolutionary strategy adopted by the algorithm in a specific optimization state and directly controls and influences the subsequent search behavior of the main population and the auxiliary population.

[0080] The overall performance of the population in step S4 is a comprehensive evaluation metric used to quantify the overall quality of a population (especially the main population) at a specific iteration time; the reward is a scalar feedback signal used to evaluate the quality of an action in a deep reinforcement learning environment.

[0081] In this embodiment, the overall population performance is used to quantitatively evaluate the population's overall operational quality at the current iteration. It is a comprehensive performance evaluation metric constructed based on convergence, diversity, and feasibility indicators. Specifically, during one iteration, for the main population, the corresponding convergence, diversity, and feasibility indicators are calculated, and these indicators are normalized to eliminate the influence of different dimensions on the comprehensive evaluation results.

[0082] Based on this, the overall performance of the population It can be represented as:

[0083] ;

[0084] in, This represents the convergence index after normalization. This represents the normalized diversity index. This represents the normalized feasibility index; These are the weighting coefficients for the corresponding indicators, used to reflect the relative importance of different indicators in the comprehensive performance evaluation. In one implementation, the weighting coefficients are preset constants and satisfy the following conditions: In another implementation, the weighting coefficients can be adaptively adjusted according to the scheduling phase or constraints.

[0085] The overall performance value of the population obtained by the above method can simultaneously reflect the comprehensive performance of the population in terms of target approximation, solution set distribution, and engineering constraint satisfaction, and is suitable as a performance evaluation basis in reinforcement learning environments.

[0086] The deep reinforcement agent in step S5 is the core of intelligent decision-making in this optimization method. It is a computational model that combines deep neural networks with reinforcement learning. It is responsible for learning and executing the mapping strategy from the optimization state to the action in order to maximize the long-term cumulative reward.

[0087] A Pareto optimal solution set is a special set of solutions in a multi-objective optimization problem, where each solution has the property that it cannot improve any other objective without compromising any other objective. This means that any solution in this set is the best compromise solution after weighing all conflicting objectives.

[0088] The above steps detail the implementation of the competitive evolutionary multi-task optimization method. Step S1, with its dual-population mechanism, provides a general framework for handling constrained problems. Steps S3 to S5, guided by the DRL (Digital Reinforcement Learning) framework, act as its "intelligent brain," enabling it to flexibly adapt to different problem scenarios and adaptively handle various types of constrained multi-objective optimization problems (such as different constraint types and Pareto front shapes), exhibiting stronger robustness. The "optimization state" defined in step S2 encompasses these three key indicators. The learning objective of the DRL model is to find strategies that simultaneously improve these three indicators, naturally guiding the population to achieve a balance among them. This resolves the core contradiction in constrained multi-objective optimization, preventing the algorithm from excessively pursuing one indicator at the expense of others, thus enabling the search for a wider, higher-quality, and fully feasible set of non-dominated solutions. Steps S3, S4, and S5 constitute the reinforcement learning loop. Based on the reward in step S4, the system can identify which actions and under what conditions can bring performance improvements, thereby allocating resources more accurately in subsequent iterations and avoiding wasting computational resources (such as the number of function evaluations) on invalid or inefficient search directions. Through the iterative cycle from steps S2 to S5, intelligent collaboration promotes the transfer of beneficial knowledge among the population and accelerates convergence. The selection of intelligent operators ensures efficient searching at each stage and jointly improves the quality of the final solution.

[0089] The aforementioned scheme successfully addresses two core technical problems in existing multi-task optimization methods—"blind resource allocation" and "rigid strategy selection"—by deeply integrating deep reinforcement learning with a competitive evolutionary multi-task framework. Its technical effect is to transform the optimization process from a static, automated process requiring prior knowledge into a dynamic, self-learning, intelligent process. This not only significantly improves the algorithm's search efficiency and the quality of the final solution but also greatly enhances its adaptability and robustness in various complex optimization scenarios. These effects have been fully experimentally verified in practical application cases such as DOC, the LIR-CMOP test set, and Joint Economic Emissions Scheduling (CEED) mentioned in the background section.

[0090] Preferably, the cooperation methods include: tightly coupled cooperation and loosely coupled cooperation;

[0091] Tightly coupled collaborations exchange solutions in each iteration, while loosely coupled collaborations exchange solutions only during the environment selection phase.

[0092] Here, iteration refers to each loop execution from S3 to S5, that is, each time an action is selected and executed, the population is updated, the reward is calculated, and the model is trained.

[0093] Exchange solutions are the specific behaviors of the cooperative approach during execution, and are a way to "update the population".

[0094] The environmental selection phase is a sub-step in evolutionary algorithms, usually performed after population renewal (reproduction), and is used to select high-quality solutions for the next generation;

[0095] Tight coupling: The solutions are swapped in each iteration (i.e., each execution of S3);

[0096] Loose coupling: The solution is exchanged only during the environment selection phase (i.e., a sub-phase after S3 execution).

[0097] Tightly coupled cooperation is a high-frequency, close knowledge transfer strategy between the main population and the auxiliary population. Its core feature is that the two populations exchange solutions in each iteration of evolution. Loosely coupled cooperation is a low-frequency, loose knowledge transfer strategy between the main population and the auxiliary population. Its core feature is that solutions are exchanged only in the environmental selection phase, while the reproduction phase in the iteration process is independent of each other.

[0098] The above scheme details the specific implementation of the collaboration method in step S3. By employing tight-coupled collaboration in a timely manner, the algorithm can quickly traverse vast infeasible or low-quality regions, rapidly transmitting potential regions discovered in the auxiliary population to the main population, thereby significantly shortening the time required to converge to the vicinity of the Pareto front. By employing loose-coupled collaboration in a timely manner, the algorithm can effectively discover and explore small-scale, discrete, or narrow feasible regions. The loose-coupled approach avoids the "destruction" or "overwhelming" of such fine structures by high-frequency information exchange, allowing the auxiliary population to deeply explore specific regions before injecting high-quality solutions into the main population. Because it possesses two tools with different characteristics, the algorithm no longer relies on a fixed collaboration frequency to address all types of problems or all optimization stages. This enables the method to adaptively handle constrained multi-objective optimization problems with different feasible region characteristics, exhibiting stronger robustness.

[0099] The “tightly coupled and loosely coupled collaboration” defined in the above scheme is not simply two parallel implementation methods, but rather provides two strategic options with complementary functions, which greatly enhances and enriches the decision space of the DRL model, enabling its intelligent decision-making capabilities to be truly implemented, and ultimately jointly achieves the technical effects of accelerating convergence, improving the quality of the solution set, and enhancing the robustness of the algorithm.

[0100] Preferably, the operators in the set of reproductive operators include genetic algorithm operators, a first differential evolution operator, and a second differential evolution operator.

[0101] Among them, genetic algorithm operators are the core operations used to generate new candidate solutions in evolutionary algorithms that mimic the mechanisms of natural selection and genetics in the biological world. They mainly include selection, crossover, and mutation. The first differential evolution operator and the second differential evolution operator are two different individual mutation strategies in differential evolution algorithms. They differ in the selection method of parent individuals and the number of differential vectors during the generation of test vectors, which leads to different focuses of their exploration and development capabilities.

[0102] The above scheme details the specific implementation of the operators in the combination of reproductive operators in step S3. The algorithm does not need to use a fixed operator or rely on preset switching rules. The DRL model can intelligently select the most suitable operator based on the current "optimization state" (e.g., low diversity index, poor convergence index). When it is necessary to maintain the diversity of solutions, GA is selected. When it is necessary to develop potential high-quality solution regions, the first differential evolution operator DE / rand / 1 is selected. When it is necessary to balance global exploration and local development, the second differential evolution operator DE / rand / 2 is selected. This dynamic balancing ability is far superior to the strategy of using a fixed single operator or polling operator, enabling the algorithm to adapt to different stages of the optimization process; the three operators constitute a system with variable search "resolution". GA performs "coarse search" to locate promising regions, while the two DE operators can perform "fine search" to find high-quality solutions within those regions. The DRL model intelligently controls this "resolution" switching;

[0103] The multi-operator combination defined in the above scheme is not a simple listing of operators, but rather a collection of complementary strategies covering the entire search process, which greatly enhances and visualizes the decision-making and execution capabilities of the DRL model. It directly contributes to improving the algorithm's convergence speed, solution set quality, robustness, and adaptability.

[0104] Preferably, the convergence index is calculated based on the changes between the ideal point and the worst point between adjacent iterations;

[0105] The diversity index is calculated based on the reciprocal of the average Euclidean distance between a solution and its neighboring solutions;

[0106] The feasibility index is calculated based on the average of the constraint violation values ​​of all solutions in the population.

[0107] Among these metrics, the change in the ideal point and the worst point between adjacent iterations is a quantitative measure of convergence. It assesses the speed and stability of the population's approach to the Pareto optimal front by calculating the magnitude of the movement of the population towards the ideal point and the worst point reached in two adjacent iterations. The reciprocal of the average Euclidean distance between a solution and its nearest neighbors is a quantitative measure of diversity. It assesses the distribution density and uniformity of the solution set in the target space by calculating the average distance between each solution and its nearest neighbors and taking its reciprocal. A smaller value indicates a larger distance between solutions, a sparser distribution, and potentially better diversity; a larger value indicates a smaller distance between solutions, a denser distribution, and potentially worse diversity. In practical applications, its reciprocal or related transformations are usually used to make its value positively correlated with diversity. The average of the constraint violation values ​​of all solutions in the population is a quantitative measure of feasibility. It assesses the degree to which the entire population, as a whole, deviates from the feasible region by calculating the constraint violation value of each solution in the population and then taking their arithmetic mean.

[0108] The above scheme details the implementation of convergence, diversity, and feasibility indicators in step S2. These three indicators monitor the optimization process in real time from different dimensions:

[0109] Convergence metrics (changes between ideal and worst points): These tell the DRL model whether the population is moving forward at the right speed and in the right direction, and whether it is accelerating. If the change slows down, it indicates that convergence has stalled, and a more aggressive strategy may be needed.

[0110] The diversity index (inverse of local crowding) tells the DRL model whether the solution set is evenly distributed over a wide area or crowded in a few small areas. If the value is too large (crowded), measures need to be taken to enhance the exploration.

[0111] The feasibility metric (average constraint violation value) tells the DRL model "how far the current population is from satisfying all constraints." If the value is too high, reducing constraint violations must be prioritized.

[0112] This multi-dimensional diagnostic approach avoids the one-sidedness of making decisions based on a single indicator (such as focusing only on the objective function value). Since these three indicators are general and essential characteristics for multi-objective optimization and constrained optimization, rather than empirical parameters for a specific problem, they are applicable to a wide range of CMOPs.

[0113] The three indicators defined in the above scheme are not simply a list of formulas, but rather represent the construction of a comprehensive, quantitative, and universal optimization state perception system. This transforms intelligent decision-making from theory into reality, directly contributing to improvements in the algorithm's adaptive capabilities, robustness, and final performance.

[0114] In this embodiment, the state is defined. It consists of three categories of indicators: convergence, diversity, and feasibility; action Defined as a set of cooperative methods and operator combinations;

[0115] The specific implementation process is as follows:

[0116] ;

[0117] In the formula, Indicates the first One action, Indicates the last action;

[0118] Therefore, the set of action combinations used in the proposed deep reinforcement learning guided competition mechanism is as follows:

[0119] ;

[0120] In the formula, In Indicates the selected collaboration method. Indicates the selected operator;

[0121] The mathematical expression for state s is: ;

[0122] In the formula, metric1, metric2 and metric3 represent three different quantitative formulas for convergence, diversity and feasibility, respectively;

[0123] Regarding convergence indices metric The mathematical expression for 1 is:

[0124] ;

[0125] In the formula, and Representing the first The variables of the ideal point and the worst point in the next iteration Indicates the generation threshold. It represents a small positive value;

[0126] The diversity index is based on the solution With It is calculated using the reciprocal of the average Euclidean distance between the nearest neighbors:

[0127] ;

[0128] In the formula, Indicates population size, Indicates the relationship between the population and the solution Recent One of the solutions, Used to calculate the Euclidean distance between two solutions;

[0129] Regarding feasibility indicators metric The mathematical expression for 3 is:

[0130] ;

[0131] In the formula, Representing the solution The constraint violation value.

[0132] Preferably, the reward is calculated based on the change in the overall performance of the population after the action is performed, specifically as follows:

[0133] If the overall performance of the population improves in the current iteration compared to the previous iteration after the action is performed, the reward is defined as 10.

[0134] If the overall population performance remains unchanged in the current iteration compared to the previous iteration, the reward is defined as 5;

[0135] If the overall performance of the population declines in the current iteration compared to the previous iteration, the reward is defined as 0.

[0136] The above scheme details the implementation of step S4. By explicitly informing the model that improving performance is the highest objective (reward 10), maintaining performance is acceptable (reward 5), and performance degradation is unacceptable (reward 0), this design forces the model to discard actions that lead to performance degradation and actively explore and repeat actions that bring improvement. Compared to continuous and complex reward functions, this design greatly simplifies the learning problem and accelerates the model's convergence speed. The model does not need to understand "how much improvement has been made," but only needs to focus on "whether there has been an improvement," resulting in higher learning efficiency. This reward mechanism directly transforms the comprehensive and outcome-based evaluation metric of "overall population performance" into the driving force for model updates. This means that any collaborative approach or operator combination that brings performance improvement, regardless of its specific nature, will be encouraged. This ensures that the optimization objective of the DRL model is completely consistent with the ultimate goal of the entire algorithm (improving population performance); this reward design does not depend on any specific knowledge of the problem or any hyperparameters that need to be adjusted. It is universal for any constrained multi-objective optimization problem, as long as "overall population performance" can be calculated. This universality further enhances the robustness and applicability of the entire method.

[0137] The reward design described above is efficient, stable, and goal-oriented. By simplifying complex performance evaluation into clear learning signals, it greatly promotes the rapid and stable convergence of the DRL model and ensures that the learned strategies always aim to improve the overall performance of the population. This design balances strong guidance capabilities with versatility and stability.

[0138] In this embodiment, a reward is defined. The reward is assigned based on the improvement of the overall performance of the population in adjacent iterations. If the performance decreases, the reward is 0; if it remains unchanged, the reward is 5; and if it improves, the reward is 10.

[0139] This deep reinforcement learning-guided competition mechanism maps the improvement in the population state after each iteration to a reward, thereby controlling the selection of DQN in different tasks. The reward mapping mechanism is defined as follows:

[0140] ;

[0141] In the formula, and These represent the overall performance of the current iteration and the previous iteration, respectively.

[0142] Preferably, the deep reinforcement agent is trained using optimized states, actions, and rewards, specifically as follows:

[0143] An experience replay mechanism is adopted to train a deep Q-network by minimizing the loss function and optimizing the state, action and reward. The deep reinforcement agent is specifically a deep Q-network.

[0144] The loss function formula is:

[0145] ;

[0146] in, This represents the numerical value of the loss function. This represents the sampled training data. and They represent the first The state and actions in the next iteration. Indicates will and The output after inputting the action value function. Indicates the state Take action below Q value;

[0147] Get The formula is:

[0148] ;

[0149] in, Indicates the first In the next iteration, in the state Take action below The rewards received Indicates the discount factor. Indicates the state Next steps The maximum Q value that can be obtained.

[0150] Among them, the experience replay mechanism is a key training technique in deep reinforcement learning. It uses a fixed-size repository (replay pool) to store the experience data generated by the agent's interaction with the environment, and randomly samples a batch of data from the pool for learning during training.

[0151] The above scheme details the implementation of step S5. By combining two cornerstone techniques of deep reinforcement learning—experience replay and a loss function with a target Q-network—it addresses the inherent instability and convergence challenges of reinforcement learning under nonlinear function approximation (i.e., using deep neural networks). Specifically, the experience replay mechanism breaks the strong correlation between consecutive experience data and reuses valuable experience data, effectively preventing "catastrophic forgetting" and training oscillations caused by the neural network learning a series of highly correlated samples. The target Q-value formula is calculated by introducing a slower-updating target network. This solves the "moving target" problem. In the early stages of training, Q-value estimation is inherently inaccurate. If an unstable network is used for both decision-making and evaluation, it's like scoring itself, leading to drastic changes in the evaluation criteria and preventing convergence. The target network provides a short-lived, fixed target, giving training a stable "anchor point." The experience replay mechanism allows the experience of each interaction to be stored and used multiple times for training. In optimization problems, the computational cost of each function evaluation (i.e., executing actions and evaluating performance) can be very high. Experience replay allows expensive evaluation results to be fully utilized, greatly enhancing the value of the data; discount factor... The introduction of this factor allows Q-values ​​to consider not only immediate rewards. It also takes into account the long-term cumulative rewards that may be obtained in the future.

[0152] The specific training method defined in the above scheme is not a simple application of existing DQN technology, but rather a targeted and necessary application to solve the specific and complex problem of "competitive evolutionary multi-task optimization". By introducing a series of stabilization and efficiency techniques, it ensures that the intelligent decision-making model can be reliably trained, thereby transforming the core concept of the entire invention—deep reinforcement learning-guided optimization—from a theoretical idea into an implementable, convergent, and efficient technical solution.

[0153] In this embodiment, the specific implementation process is as follows:

[0154] In the competition mechanism guided by deep reinforcement learning, a backpropagation neural network is used to train the DQN. To ensure that the data used to train the Q-network can fully reflect the optimization process of the constrained multi-objective optimization problem, an experience replay mechanism is used to store training records, and its structure is defined as follows:

[0155] ;

[0156] In the formula, Indicating the first in experience replay 1 record This indicates the capacity of the experience replay. The records in the experience replay are structured as follows:

[0157] ;

[0158] In the formula, Indicates state, Indicates an action, It signifies a reward.

[0159] The training dataset is denoted as A deep Q-network is represented as When a deep Q-network is trained for the first time, all its parameters are initialized randomly. Subsequently, the deep Q-network uses selected... The data was used for training, and the parameters were adjusted to better approximate the optimal value. In subsequent updates, the parameters of the deep Q-network are updated using the latest data from empirical replay. Fine-tune the model to ensure it remains consistent with the current environmental conditions.

[0160] like Figure 2 As shown, preferably, it also includes:

[0161] A1. During each iteration, when the deep reinforcement agent has not been trained, the action is randomly selected from the preset set of cooperation methods and reproduction operators, and the selection result is stored in the experience replay pool.

[0162] A2. Once the deep reinforcement agent has been trained, the action is selected based on the current optimization state of the trained deep reinforcement agent.

[0163] In this embodiment, the training completion determination condition is at least one of the following conditions: (1) the number of training iterations of the deep reinforcement agent reaches the preset maximum number of training rounds; (2) in several consecutive training iterations, the change in the average reward value calculated based on the overall performance of the population is less than a preset threshold, indicating that the learning process of the model has converged; (3) in several consecutive training iterations, the action selection result output by the deep reinforcement agent under the same or similar optimization state remains stable, indicating that the learned decision strategy has been stably formed. When any of the above training completion determination conditions are met, it is considered that the deep reinforcement agent has been trained and enters the action selection stage based on model decision.

[0164] Steps A1 and A2 detail the implementation of the iterative training strategy. In the initial stage, by randomly selecting actions, the algorithm can broadly explore all possible cooperative methods and operator combinations in an unbiased manner. This avoids getting stuck in localized regions due to poor early decisions when the model is unaware, resulting in fragmented and singular collected empirical data. As data accumulates in the experience replay pool and the model continues to train, the DRL model gradually becomes "aware" from "unknowing." After the model training is complete, the system automatically switches to a stage where the model intelligently selects actions based on the current state. At this point, the exploration does not stop but transforms into a guided and directed exploration; this two-stage design ensures that the algorithm always produces an effective behavior. Even in the worst case (such as DRL model training failure), the first stage guarantees that the algorithm runs as a stochastic multi-task evolutionary algorithm and outputs a solution set, ensuring the algorithm's basic functionality and robustness.

[0165] The two-stage strategy defined in the above scheme is a key mechanism to ensure that the intelligent system can be robustly "booted up" and "operate autonomously." Through clear stage division, it resolves the core contradictions of DRL in the early stages of application, guarantees the quality of training data, achieves a smooth transition in the learning process, and ultimately ensures the reliability, effectiveness, and practicality of the entire invention method.

[0166] In this embodiment, the iterative optimization process is divided into two stages: the random evolution multi-task stage: when the deep Q network has not yet been trained, the cooperative mode and operator combination are randomly selected for reproduction and de-exchange, and the results are stored in the experience replay pool; the deep reinforcement learning guidance stage: when the experience replay pool has accumulated enough training data, the deep Q network is used to select the optimal action, dynamically guide the cooperative mode of different populations and operators, and update the deep Q network periodically.

[0167] Throughout the evolutionary process, the experience replay pool is continuously updated, and the deep Q-network is continuously optimized based on the latest data until a preset termination condition is reached. Finally, the non-dominated solution set in the main population is output as the optimization result.

[0168] To further verify the technical effectiveness of the competitive evolutionary multi-task optimization algorithm based on deep Q-networks proposed in this invention, the following verification method was designed:

[0169] Eight advanced constrained multi-objective evolutionary algorithms were selected for comparison and evaluation: APSEA, BiCo, CMOET, CMOES, EMCMO, MOEA / D-CMT, MTCMO, and CMOQLMT. Comprehensive comparative experiments with these algorithms rigorously validated the effectiveness of the proposed CEMT-DQN.

[0170] Two benchmark sets, DOC and LIR-CMOP, were used to evaluate the algorithm's performance. Details of each benchmark set are as follows:

[0171] (1) DOC Test Set: This set includes both decision constraints and objective constraints, covering various constraint types from practical applications, including inequalities, equality constraints, linear constraints, and nonlinear constraints. The Pareto Feasibility Front (CPF) of the DOC problem exhibits various complex characteristics, such as nonlinearity, extrema, continuity, discreteness, and mixed structures. In the DOC test set, the number of objective constraints... For DOC8 and DOC9, the value is set to 3, and for the remaining problems, it is set to 2. Each problem corresponds to a predefined and fixed number of decision variables. .

[0172] (2) LIR-CMOP test set: Its characteristic is that the feasible region is small, while most regions are infeasible, often hindering the search for CPF. This makes CMOEAs prone to getting trapped in local feasible regions during the optimization process, making it difficult to effectively approximate the true CPF. In this test set, the target number of LIR-CMOP13 and LIR-CMOP14 is... Set the number of decision variables to 3, and set the remaining questions to 2. It is always fixed at 10.

[0173] In both the DOC and LIR-CMOP test problems, the maximum number of function evaluations was set to 300,000. Each algorithm was run independently 30 times on each test instance. The mean and standard deviation of the IGD and HV indices were recorded, and statistical analysis of all index results was performed using a Wilcoxon rank-sum test at a significance level of 0.05.

[0174] Evaluation Metric - HV:

[0175] ;

[0176] In the formula, Represents the Lebesgue measure, used to measure volume. Represents the number of non-dominated solution sets. Indicates the reference point and the first solution in the solution set. The hypervolume formed by the solutions.

[0177] Evaluation Metric - IGD:

[0178] ;

[0179] In the formula, It is the solution set obtained by the algorithm. It is a set of uniformly distributed reference points sampled from the Pareto front. Representation of reference set Points in To the solution set Points in The Euclidean distance between them.

[0180] Comparison results:

[0181] To evaluate the performance of the proposed CEMT-DQN, the Friedman test was applied to the IGD and HV indices obtained from the test set. The average ranking obtained through this statistical analysis is shown in Table 1. The results show that CEMT-DQN achieved the highest overall ranking among the nine comparative algorithms, demonstrating its superior performance. Among the EMT-based CMOEAs, MTCMO, EMCMO, and CMOPT outperformed APSEA and BiCo, reflecting the advantages of multi-task mechanisms in CMOPs. Although CMOQLMT also incorporates reinforcement learning to guide the EMT process, it is less efficient in the design and selection of evolutionary tasks, and its ranking is comparable to the baseline algorithms. Although MOEA / D-CMT performs competitively on the LIR-CMOP problem, it performs poorly on the DOC test set, thus resulting in a low average IGD ranking. As shown in Table 1, the average ranking results of the Friedman test show that the CEMT-DQN algorithm of this invention achieved the best ranking (2.3478) in both IGD and HV indicators, which is significantly better than the comparison algorithm APSEA (7.3043, 7.2174), demonstrating its superior overall performance.

[0182] Table 1. Average ranking of IGD and HV values ​​obtained by nine comparison algorithms in the Friedman test

[0183]

[0184] Preferably, the formula for obtaining the fuel cost value is:

[0185] ;

[0186] in, For total fuel cost, The total number of generators. For generator index, For the first The output power of the generator , , The first Cost coefficient of a generator;

[0187] The formula for obtaining gas emission values ​​is:

[0188] ;

[0189] in, Total gas emissions, , , The first The emission coefficient of the generator;

[0190] The formula for the optimization objective of the scheduling problem is:

[0191] ;

[0192] in, and To optimize the objective, For total power load demand, This represents the actual total power generation. For the first The generator is allowed to generate the minimum power. For the first The maximum allowable power output of the generator.

[0193] The above-described scheme combines the aforementioned highly abstract and general optimization method with a classic application scenario of significant practical industrial importance. Specifically, through its dual-population mechanism (handling constrained and unconstrained problems separately) and DRL intelligent guidance, it can more effectively explore the solution space and dynamically adjust the search strategy to find a better scheduling scheme that balances fuel costs and gas emissions. Since the output of this invention is a Pareto optimal solution set (i.e., a set of scheduling schemes) rather than a single solution, it provides decision-makers with a rich selection space. Decision-makers can flexibly select the final scheme from the solution set based on current policy preferences (e.g., whether to prioritize economic efficiency or environmental protection).

[0194] By combining this invention with the classic and important industrial problem of joint economic emissions dispatch, significant technical effects have been achieved. It not only verifies the efficiency of general methods in practice but also highlights its enormous industrial value and economic and environmental benefits in solving complex real-world problems. This limitation elevates the invention from an abstract algorithm to a dedicated solution that can directly serve the intelligent and green operation of modern power systems, greatly enhancing the invention's technical persuasiveness and market application prospects.

[0195] In this embodiment, the specific implementation process is as follows:

[0196] The performance of CEMT-DQN was further validated on three types of CEED problems: 6-unit CEED problems, 11-unit CEED problems, and 40-unit CEED problems. It should be noted that different numbers of CEED problems were chosen for testing to gradually increase the optimization difficulty, thereby comprehensively evaluating the performance of the proposed algorithm under different test scenarios.

[0197] In subsequent tests, losses during the power generation process were considered negligible, and the load demand (i.e., the load demand of the power generation system) for different units with CEED problems was considered. The settings are as follows:

[0198] CEED issue for Unit 6: Load demand set at 400, 500, 600, 700, 800, 900, 1000;

[0199] CEED issue for Unit 11: Load demand set at 1500, 1750, 2000, 2250, 2500;

[0200] CEED issue for Unit 40: Load demand set to 10500.

[0201] The maximum number of function evaluations for all CEED problem instances was set to 100,000, and each algorithm was run independently 30 times on each test instance. Since the true constraint Pareto front (CPF) of the actual problem is unknown, the mean and standard deviation of the hypervolume (HV) values ​​were recorded. At the same time, the Wilcoxon rank-sum test with a significance level of 0.05 was used to perform statistical analysis on all index results.

[0202] Table 2 reports the HV values ​​of nine algorithms on CEED problems with 6, 11, and 40 units. The results show that the MOEA / D-CMT algorithm performs the worst on all CEED problem instances. Conversely, the proposed competitive evolutionary multi-task optimization algorithm based on deep Q-networks (CEMT-DQN) achieves the best HV value on 7 out of 13 instances, demonstrating its robust performance and excellent scalability. Overall, these results fully demonstrate the effectiveness of CEMT-DQN in solving practical CEED problems, further validating the advantages of its evolutionary task design and the efficiency of its deep reinforcement learning (DRL)-based competitive mechanism.

[0203] Table 2. HV values ​​of APSEA, BiCo, CMOET, CMOES, EMCMO, MOEA / D-CMT, MTCMO, CMOQLMT and the proposed CEMT-DQN algorithm on the CEED problem.

[0204]

[0205] like Figure 3 As shown, a competitive evolutionary multi-task optimization system is used to solve the joint economic emission dispatch problem of a power system. The dispatch problem is a constrained multi-objective optimization problem that jointly optimizes fuel cost and gas emissions under the conditions of satisfying power balance constraints and generator capacity constraints, including:

[0206] The population construction module is used to construct the main population and auxiliary population for parallel solving of constrained multi-objective optimization problems. Each population individual represents a set of candidate solutions for generator output allocation that satisfy the engineering constraints.

[0207] The state generation module is used to calculate and generate indices characterizing the optimization states of the two populations based on the objective function values ​​and constraint violation degrees of candidate solutions in the main population and auxiliary population. The indices include convergence indices, diversity indices, and feasibility indices. The objective function values ​​include fuel cost values ​​and gas emission values ​​calculated based on candidate solutions.

[0208] The action selection module is used to select and execute an action from a set of preset combinations of cooperation methods and reproduction operators to update the candidate solutions of the main population and the auxiliary population based on the current optimization state through a deep reinforcement learning agent.

[0209] The reward calculation module is used to calculate the reward that reflects the degree of improvement in scheduling performance based on the change in the overall performance of the population after the action is performed. The overall performance of the population is calculated by combining convergence index, diversity index and feasibility index.

[0210] The model training module is used to train the deep reinforcement learning agent using the optimized state, the actions performed, and the rewards, so as to update the decision-making strategy used to guide the selection of subsequent actions.

[0211] The population construction module, state generation module, action selection module, reward calculation module, and model training module are run iteratively until the preset termination condition is met, and the Pareto optimal solution set of the scheduling problem is output as the generator output allocation scheme.

[0212] This invention visualizes each logical step in the competitive evolutionary multi-task optimization method as an independent system module. Through modular design, the system assigns population construction, state generation, action selection, reward calculation, and model training functions to different units, enabling the competitive evolutionary multi-task optimization method to be executed effectively and reliably.

[0213] A competitive evolutionary multitasking optimization device, comprising:

[0214] One or more processors;

[0215] A memory that stores computer programs;

[0216] When a computer program is executed by a processor, it implements any of the competitive evolutionary multitasking optimization methods described above.

[0217] This invention also discloses a multi-competitive evolutionary multi-task optimization device, which transforms the aforementioned competitive evolutionary multi-task optimization method into a concrete and implementable physical system or product. Its core technological advantage lies in "materializing" innovative algorithmic ideas into a stable, reliable, and deployable industrial-grade solution, achieving a leap from theory to practice.

[0218] One or more embodiments in this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments in this application should be included within the protection scope of this application.

[0219] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by program instructions and related hardware. The aforementioned program instructions can be stored in a computer-readable storage medium. When the program instructions are executed, they perform the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0220] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0221] If a flowchart is used in this application, it is used to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps can be processed in reverse order or simultaneously. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0222] The foregoing has provided a detailed description of a competitive evolutionary multi-task optimization method, system, and device provided in this application. The above description of the disclosed embodiments enables those skilled in the art to implement or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A competitive evolutionary multi-task optimization method for solving the joint economic emission dispatch problem of a power system, wherein the dispatch problem is a constrained multi-objective optimization problem that jointly optimizes fuel cost and gas emissions under the conditions of satisfying power balance constraints and generator capacity constraints, characterized in that... Includes the following steps: S1. Construct a main population and an auxiliary population for parallel solving of constrained multi-objective optimization problems, where each population individual represents a set of candidate solutions for generator output allocation that satisfy the engineering constraints; S2. Based on the objective function values ​​and constraint violation degrees of the candidate solutions in the main population and auxiliary population, calculate and generate indices characterizing the optimization states of the two populations, including convergence indices, diversity indices, and feasibility indices; wherein, the objective function values ​​include fuel cost values ​​and gas emission values ​​calculated based on the candidate solutions; S3. Based on the current optimization state, a deep reinforcement learning agent selects and executes an action from a preset set of combinations of cooperation methods and reproduction operators to update the candidate solutions of the main population and the auxiliary population; wherein, the set of combinations of cooperation methods and reproduction operators consists of a variety of predefined combinations of actions, each combination of action consists of a cooperation method and a reproduction operator, the cooperation method is used to define the information interaction rules between the main population and the auxiliary population in the current iteration cycle, and the reproduction operator is used to define the generation and updating method of the candidate scheduling parameter scheme under the cooperation method; S4. Based on the change in the overall performance of the population after the action is performed, calculate the reward that reflects the degree of improvement in scheduling performance. The overall performance of the population is calculated by comprehensively considering the convergence index, diversity index, and feasibility index. S5. Using the optimized state, the action performed, and the reward, train the deep reinforcement learning agent to update the decision-making strategy used to guide the selection of subsequent actions; Iteratively execute steps S2 to S5 until the preset termination condition is met, and output the Pareto optimal solution set of the scheduling problem as the generator output allocation scheme. The cooperation methods include: tightly coupled cooperation and loosely coupled cooperation; The tightly coupled cooperation exchanges solutions in each iteration, while the loosely coupled cooperation exchanges solutions only during the environment selection phase. The set of combined reproductive operators includes genetic algorithm operators, first differential evolution operators, and second differential evolution operators.

2. The competitive evolutionary multi-task optimization method as described in claim 1, characterized in that, The convergence index is calculated based on the changes between the ideal point and the worst point between adjacent iterations; The diversity index is calculated based on the reciprocal of the average Euclidean distance between a solution and its neighboring solutions; The feasibility index is calculated based on the average of the constraint violation values ​​of all solutions in the population.

3. The competitive evolutionary multi-task optimization method as described in claim 1, characterized in that, The reward, which reflects the improvement in scheduling performance, is calculated based on the change in the overall performance of the population after the action is performed. Specifically: If the overall performance of the population improves in the current iteration compared to the previous iteration, a first positive reward value is assigned. If the overall performance of the population remains unchanged in the current iteration compared to the previous iteration, then zero reward or the second smallest reward value is assigned. If the overall performance of the population declines in the current iteration compared to the previous iteration, a negative reward value is assigned.

4. The competitive evolutionary multi-task optimization method as described in claim 1, characterized in that, The process of training the deep reinforcement learning agent using the optimized state, the executed action, and the reward specifically involves: An experience replay mechanism is used to train a deep Q-network by minimizing a loss function, wherein the deep reinforcement learning agent is specifically the deep Q-network; The loss function formula is as follows: ； in, This represents the numerical value of the loss function. This represents the sampled training data. and They represent the first The state and actions in the next iteration. Indicates will and The output after inputting the action value function. Indicates the state Take action below Q value; Get The formula is: ； in, Indicates the first In the next iteration, in the state Take action below The rewards received Indicates the discount factor. Indicates the state Next steps The maximum Q value that can be obtained.

5. The competitive evolutionary multi-task optimization method as described in claim 1, characterized in that, Also includes: During each iteration, when the deep reinforcement learning agent has not completed training, the action is randomly selected from the preset set of combinations of cooperation methods and reproduction operators, and the selection result is stored in the experience replay pool. Once the deep reinforcement learning agent has been trained, the action is selected based on the current optimization state of the trained deep reinforcement learning agent.

6. The competitive evolutionary multi-task optimization method as described in claim 1, characterized in that, The formula for obtaining the fuel cost value is: ； in, For total fuel cost, The total number of generators. For generator index, For the first The output power of the generator , , The first Cost coefficient of a generator; The formula for obtaining the gas emission value is: ； in, Total gas emissions, , , The first The emission coefficient of the generator; For the first The output power of the generator; The formula for the optimization objective of the scheduling problem is: ； in, and To optimize the objective, For total power load demand, This represents the actual total power generation. For the first The generator is allowed to generate the minimum power. For the first The maximum allowable power output of the generator.

7. A competitive evolutionary multi-task optimization system for performing the method as described in any one of claims 1 to 6, for solving a power system joint economic emission dispatch problem, wherein the dispatch problem is a constrained multi-objective optimization problem that jointly optimizes fuel cost and gas emissions under the conditions of satisfying power balance constraints and generator capacity constraints, characterized in that, include: The population construction module is used to construct the main population and auxiliary population for parallel solving of constrained multi-objective optimization problems. Each population individual represents a set of candidate solutions for generator output allocation that satisfy the engineering constraints. The state generation module is used to calculate and generate indices characterizing the optimization states of the two populations based on the objective function values ​​and constraint violation degrees of the candidate solutions in the main population and auxiliary population. The indices include convergence indices, diversity indices, and feasibility indices. The objective function values ​​include fuel cost values ​​and gas emission values ​​calculated based on the candidate solutions. The action selection module is used to select and execute an action from a set of preset combinations of cooperation methods and reproduction operators to update the candidate solutions of the main population and the auxiliary population based on the current optimization state through a deep reinforcement learning agent. The reward calculation module is used to calculate a reward that reflects the degree of improvement in scheduling performance based on the change in the overall performance of the population after the action is performed. The overall performance of the population is calculated by comprehensively considering the convergence index, diversity index, and feasibility index. The model training module is used to train the deep reinforcement learning agent using the optimized state, the actions performed, and the reward, so as to update the decision-making strategy used to guide the selection of subsequent actions. The population construction module, the state generation module, the action selection module, the reward calculation module, and the model training module are iteratively run until a preset termination condition is met, and the Pareto optimal solution set of the scheduling problem is output as the generator output allocation scheme.

8. A competitive evolutionary multitasking optimization device, characterized in that, include: One or more processors; A memory that stores computer programs; When the computer program is executed by the processor, it implements the competitive evolutionary multitasking optimization method as described in any one of claims 1 to 6.

Images