Dynamic constraint multi-objective evolution method based on large language model interaction
By using large language model interaction technology in dynamic constraint multi-objective optimization, it is divided into target-type and constrained populations, and combined with deep learning framework, the problem of lack of response strategies during dynamic environment changes is solved, and more efficient strategy generation and Environmental adaptability.
Patent Information
- Application Number
- CN202410997650.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-11-26
AI Technical Summary
The existing technology is difficult to effectively respond to environmental changes in a dynamic environment, resulting in a lack of targeted response strategies when environmental changes are changed. Especially in the dynamic constraint multi-objective optimization problem, traditional methods often can only produce random solutions or retain the previous optimal solutions. Unable to effectively respond to environmental changes.
Adopt a dynamically constrained multi-objective evolution method based on interaction of large language models. By initializing populations and dividing them into target and constraints, using large language models to save historical information, detect environmental changes, generate response individuals according to the change type, and combine depth The learning framework performs individual assessments and population updates to ensure adaptive strategies can be generated in the event of environmental changes.
The generation of targeted response strategies in dynamic environments is realized, the performance and adaptability of the algorithm is improved, the dependence on human experience is reduced, and better solutions can be generated in complex environments.
Smart Images

Figure CN119026632A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computational intelligence technology, and in particular relates to a dynamic constrained multi-objective evolutionary method based on large language model interaction. Background Art
[0002] In real life, we often encounter multi-objective optimization problems with multiple conflicts and time-varying objectives. Such problems are usually called dynamic multi-objective problems. However, many real-world dynamic multi-objective optimization problems involve multiple constraints that may change over time. Such problems are called dynamic constrained multi-objective optimization problems. Since such problems involve complex environmental changes, it is a challenge to cope with environmental changes and produce better quality solutions. In addition, dynamic constrained multi-objective optimization problems are not simple objective functions or constraints that change with environmental changes, but it is possible that both are changing. Therefore, under such complex environmental changes, most of the environmental response strategies proposed in the past only generate random solutions or retain the optimal solution in the previous environment to cope with the new environment, and are not very targeted to the problem of how the environment changes.
[0003] In addition, when generating offspring to update the population, it is often necessary to use an operator. However, the design of the operator requires the designer to have rich experience and knowledge to manually test and adjust the parameters. Therefore, the evolutionary technology that uses the large language model (LLM) to help the evolutionary algorithm came into being. The large language model is a black box decision-making process. Users can give task prompts and output requirements to let the LLM generate ideal answers and return them. In this process, the LLM has the function of saving historical conversation records. Therefore, during the evolution process, it can learn the hidden information in the historical records, automatically learn the changing rules of individuals in dynamic environments to generate corresponding solutions, and further improve the performance of the algorithm to cope with more complex actual environments.
[0004] Deep reinforcement learning is now used to solve many real-world problems from different fields, and is very popular in the field of gaming to help design intelligent AI for games. However, the use of evolutionary algorithms to solve problems in the field of reinforcement learning has attracted the attention of many people. In games, intelligent AI faces a dynamically changing environment and needs to take complex actions. This scenario is similar to the dynamic constraint multi-objective evolution method, which converts the rewards obtained from the dynamic changes in the game environment into target evaluation values under the evolutionary algorithm, thereby determining the corresponding optimal strategy as a trade-off between feature objectives. Summary of the invention
[0005] The purpose of the present invention is to provide a dynamic constrained multi-objective evolution method based on large language model interaction, which solves the problem of no targeted response strategy when the environment changes in the later stage.
[0006] The technical solution adopted by the present invention is a dynamic constrained multi-objective evolution method based on large language model interaction, which is specifically implemented according to the following steps:
[0007] Step 1, initialization parameters;
[0008] Step 2: Divide the population into target type and constraint type;
[0009] Step 3: Pass the non-dominated individuals to the large language model to save historical information;
[0010] Step 4: Detect whether the environment has changed, and if so, classify the type of environment change;
[0011] Step 5: If the environment changes, the interaction strategy with the large language model is initiated to generate a response individual according to the type of environmental change;
[0012] Step 6: If the environment has not changed, the LLM interaction is used to generate offspring based on the historical information saved in step 3;
[0013] Step 7: Connect the deep learning framework to return the reward value as the objective function for individual evaluation, select non-dominated individuals, and update the population and archive;
[0014] Step 8: Determine whether the number of fitness evaluations is less than the maximum number of function evaluations; if so, output the optimal solution and optimal value of the population; otherwise, return to step 2 and enter the next generation of population evolution.
[0015] The present invention is also characterized in that:
[0016] In step 1, the initialization parameters, including the population size N, the target archive F and the constraint archive C, are initialized to N / 2, and the non-dominated archive A is N; initialize N individuals as the initial population
[0017] Step 2 is implemented as follows:
[0018] Step 2.1: The specific classification steps of decision variables are as follows: Gaussian perturbation of a certain dimension of individual decision variables:
[0019]
[0020] In the formula, is the individual after disturbance, σ is a very small number, u j and l j are the upper and lower bounds of the population, respectively, and d is the population dimension;
[0021] Step 2.2: Compare the individual constraint difference Δ after the disturbance with the individual constraint difference Δ before the disturbance cj , if Δ cj If it is greater than 0.5, it is classified as a constraint-sensitive decision variable, otherwise it is a target-type decision variable; if for a d-dimensional individual, the number of constraint-type decision variables in the decision variables is greater than the number of target-type decision variables, then the individual is considered to be constraint-sensitive and classified as a constraint-type archive C, otherwise it is classified as a target-type archive F:
[0022]
[0023] In the formula, is the individual after a certain dimension of disturbance The constrained difference from x, Δ ci is the normalized constraint difference, is the minimum constraint value of the g-th constraint, is the maximum constraint value of the gth constraint.
[0024] Step 3 is implemented as follows:
[0025] Step 3.1, sort the individuals in the current environment by non-dominance, obtain the number of layers of the Pareto frontier where each individual is located, select the best individual in the first layer and put it into the non-dominated archive A, which can store non-dominated individuals in at most two environments;
[0026] Step 3.2: Connect to the large language model API interface and pass some non-dominated individuals into the LLM.
[0027] Step 4 is implemented according to the following steps:
[0028] Step 4.1: Select some individuals in the non-dominated archive in the t-1 environment and the t environment for environmental detection, and compare the degree of change of the objective function of individuals in the t-1 and t environments Δ F If the value is greater than 0.5, compare the degree of constraint change Δ C If the constraint change degree is less than 0.5, it means that there is no change and the objective function is changing. Otherwise, it means that both the objective function and the constraint condition are changing. If the constraint change degree between individual t-1 and environment t is greater than 0.5, the objective function change degree Δ F If it is less than 0.5, it is classified as a changing constraint;
[0029]
[0030] In the formula, Δ F Indicates the degree of change of the objective function between t-1 and t, Nc hoice represents the number of individuals used to detect whether the environment has changed. represents the objective function value of the kth individual in the t-1 environment, represents the objective function value of the kth individual in the t environment, represents the maximum objective function value of the individual in the t-1 environment, It represents the minimum objective function value of an individual under environment t;
[0031]
[0032] In the formula, Δ C Indicates the degree of change of constraints between t-1 and t, N choice represents the number of individuals used to detect whether the environment has changed. represents the constraint value of the kth individual in the t-1 environment, represents the constraint value of the kth individual in environment t, represents the maximum constraint value of the individual in the t-1 environment, It represents the minimum constraint value of an individual under environment t;
[0033] Step 4.2: Determine the type of environmental change. The three types are as follows:
[0034] Category 1: The target environment is changing, but the constraints remain unchanged:
[0035]
[0036] The second category: the target environment remains unchanged, but the constraints change:
[0037]
[0038] The third category: The target environment is changing, and the constraints are also changing:
[0039]
[0040] Step 5 is implemented according to the following steps:
[0041] Step 5.1: Determine the type of environmental change, and first select feasible solutions from the t-1 environment to add to P′, where P′ is the population in the new environment;
[0042] Step 5.2, if only the objective function is changing, select individuals from the target archive F as parents and pass them into LLM and prompt it to generate offspring solutions, and add the offspring solutions to P′. This step generates offspring that are 2 / 3 of the population, and the remaining individuals are random solutions to increase population diversity;
[0043] Step 5.3, if only the constraints are changing, select individuals from the constraint archive as parents and pass them into the LLM and prompt it to generate offspring solutions, and add the offspring solutions to P′. This step generates offspring that are 2 / 3 of the population, and the remaining individuals are random solutions to increase population diversity;
[0044] Step 5.4: If both of them change, select a part of individuals from the constraint archive and the target archive as parents to generate offspring, and add the offspring solutions to P′. The rest of the operations are the same as the other two categories.
[0045] Step 6 is as follows: If the target change degree Δ F The degree of constraint change Δ C If both are less than 0.5, it is determined that the environment has not changed. In this case, the saved historical information is needed to call LLM for interaction so that it can generate offspring by learning the hidden information in the historical information. At the same time, the LLM prompt strategy needs to be given.
[0046] Step 7 is implemented according to the following steps:
[0047] Step 7.1: Return the generated offspring and connect them to the deep reinforcement learning framework, evaluate their reward values and return them. The d-dimensional decision variables of each individual correspond to the network parameters of each layer of reinforcement learning, and finally obtain the strategy network with the maximum reward value through evolutionary selection;
[0048]
[0049] In the formula, q (s,a) is the state-action value function, which is the maximum reward sum obtained under strategy π, R(s, a) is the reward value obtained by executing action a in the current state s, γ is used as a discount factor to control the proportion of learning future rewards, maxq (s′,a′) Represents the maximum reward obtained by executing action a′ in the future state s′;
[0050] R(s,a)=ω0·b(s,a)+ω1·r attack +ω1·r avoid (10)
[0051] In the formula, ω0, ω1, ω2 represent the proportion of each part, b(s, a) represents the remaining health of executing action a in state s, r attack Represents the damage taken by attacking the enemy, r avoid Representing the harm done to oneself;
[0052] Step 7.2: Update the population and each archive using the reward value returned by the offspring solution as the target evaluation value, and mainly adopt the dominance principle during the update.
[0053] The beneficial effects of the present invention are:
[0054] The present invention is based on a dynamic constraint multi-objective evolutionary method of large language model interaction, conducts in-depth research and analysis on the environmental detection part under a dynamic constraint environment, and considers the sensitivity of decision variables to targets and constraints. First, the population is divided into two types: target-sensitive and constraint-sensitive, which solves the problem of no targeted response strategy when the environment changes in the later stage; in addition, the environmental change is divided into three types: only the target is changing, only the constraint is changing, and both are changing. Then, according to the type of environmental change, the large language model is called to generate offspring through interaction; the large language model is a black box decision process, and the problem of excessive reliance on human experience in traditional operator design is solved by using black box operation; at the same time, each iterative update of the population needs to pass part of the non-dominated optimal solution into the LLM, and the characteristic of its ability to save historical information is used to help the later response strategy part to generate better solutions. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] Figure 1 It is a flow chart of the dynamic constrained multi-objective evolutionary method based on large language model interaction of the present invention;
[0056] Figure 2 It is a schematic diagram of judging whether an individual is constraint-sensitive or target-sensitive in the dynamic constraint multi-objective evolutionary method based on large language model interaction of the present invention;
[0057] Figure 3 It is a schematic diagram of the interaction between the dynamic constraint multi-objective evolution method based on large language model interaction and deep reinforcement learning in the present invention;
[0058] Figure 4 It is a schematic diagram of using a large language model to generate a response strategy after the environment changes in the dynamic constrained multi-objective evolution method based on large language model interaction of the present invention. DETAILED DESCRIPTION
[0059] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0060] Example 1
[0061] The present invention is based on a dynamic constraint multi-objective evolutionary method based on large language model interaction, such as Figure 1 As shown, the specific implementation steps are as follows:
[0062] Step 1: Initialize the parameters, including the population size N, the target archive F and the constraint archive C, which are all initialized to N / 2, and the non-dominated archive A size is N; initialize N individuals as the initial population;
[0063] Set the number of iterations t = 1, the population size N = 50 / 100, and the population dimension D is determined according to the corresponding game network framework; generate N individuals as the initial population P, the size of the target archive F and the constraint archive C are initialized to N / 2, and the size of the non-dominated archive A is N; set the maximum function evaluation number FESMAX = 100000 / 200000, and the initial evaluation number FES = 0;
[0064] Step 2: Examine the sensitivity of individuals to goals and constraints from a dimensional perspective, and divide the population into target type and constraint type, so that corresponding response measures can be made when the environment changes. The classification diagram is as follows: Figure 2 As shown;
[0065] Step 3: Pass the non-dominated individuals to the large language model to save historical information, so as to learn the change rules and generate later response plans;
[0066] Step 4: Check whether the environment has changed. If so, classify the environment change types into three categories: only the objective function is changing, only the constraints are changing, and both the objective function and the constraints are changing;
[0067] Step 5: If the environment changes, the interaction strategy with the large language model is initiated to generate a response individual according to the type of environmental change;
[0068] Step 6: If the environment has not changed, the LLM interaction is used to generate offspring based on the historical information saved in step 3;
[0069] Step 7: Connect the deep learning framework to return the reward value as the objective function for individual evaluation, such as Figure 3 As shown, non-dominated individuals are selected and the population and archive are updated;
[0070] Step 8: Determine whether the number of fitness evaluations is less than the maximum number of function evaluations; if so, output the optimal solution and optimal value of the population; otherwise, return to step 2 and enter the next generation of population evolution.
[0071] Example 2
[0072] The present invention is based on a dynamic constrained multi-objective evolutionary method for large language model interaction, wherein step 2 is specifically implemented according to the following steps:
[0073] Step 2.1: The specific classification steps of decision variables are as follows: Gaussian perturbation of a certain dimension of individual decision variables:
[0074]
[0075] In the formula, is the individual after disturbance, σ is a very small number, u j and l jare the upper and lower bounds of the population, respectively, and d is the population dimension;
[0076] Step 2.2: Compare the individual constraint difference Δ after the disturbance with the individual constraint difference Δ before the disturbance cj , if Δ cj If it is greater than 0.5, it is classified as a constraint-sensitive decision variable, otherwise it is a target decision variable; if the number of constraint decision variables in an individual decision variable is greater than the target decision variable, the individual is classified as a constraint archive C, otherwise it is classified as a target archive F:
[0077]
[0078] In the formula, is the individual after a certain dimension of disturbance The constrained difference from x, Δ ci is the normalized constraint difference, is the minimum constraint value of the g-th constraint, is the maximum constraint value of the g-th constraint;
[0079] Step 3 is implemented as follows:
[0080] Step 3.1: sort the individuals in the current environment by non-dominance, obtain the number of layers of the Pareto frontier where each individual is located, select the best individual in the first layer and put it into the non-dominated archive A, which can store non-dominated individuals in at most two environments.
[0081] Step 3.2: Connect to the large language model API interface and pass some non-dominated individuals into the LLM;
[0082] Step 4 is implemented according to the following steps:
[0083] Step 4.1: Select some individuals in the non-dominated archive in the t-1 environment and the t environment for environmental detection, and compare the degree of change of the objective function of individuals in the t-1 and t environments Δ F If the value is greater than 0.5, compare the degree of constraint change Δ C If the constraint change degree is less than 0.5, it is classified as the first category if no change occurs, otherwise it is classified as the third category; if the constraint change degree between individual t-1 and environment t is greater than 0.5, the objective function change degree Δ F If it is less than 0.5, it is classified as the second category.
[0084]
[0085] In the above formula, Δ F Indicates the degree of change of the objective function between t-1 and t, N choice represents the number of individuals used to detect whether the environment has changed. represents the objective function value of the kth individual in the t-1 environment, represents the objective function value of the kth individual in the t environment, represents the maximum objective function value of the individual in the t-1 environment, It represents the minimum objective function value of an individual under environment t;
[0086]
[0087] In the formula, Δ C Indicates the degree of change of constraints between t-1 and t, N choice represents the number of individuals used to detect whether the environment has changed. represents the constraint value of the kth individual in the t-1 environment, represents the constraint value of the kth individual in environment t, represents the maximum constraint value of an individual in a t-1 environment, It represents the minimum constraint value of an individual under environment t;
[0088] Step 4.2: Determine the type of environmental change. The three types are as follows:
[0089] Category 1: The target environment is changing, but the constraints remain unchanged:
[0090]
[0091] The second category: the target environment remains unchanged, but the constraints change:
[0092]
[0093] The third category: The target environment is changing, and the constraints are also changing:
[0094]
[0095] Example 3
[0096] The present invention is based on a dynamic constrained multi-objective evolutionary method for large language model interaction, wherein step 5 is specifically implemented according to the following steps:
[0097] Step 5.1: Determine the type of environmental change, then first select a feasible solution from the t-1 environment and add it to P′, where P′ is the population in the new environment; the LLM interaction diagram is as follows: Figure 4 shown.
[0098] Step 5.2: If only the objective function is changing, select individuals from the target archive F as parents and pass them into LLM to prompt them to generate offspring solutions, and add the offspring solutions to P′. This step generates offspring that are 2 / 3 of the population, and the remaining individuals are random solutions to increase population diversity. The following is the prompt strategy for LLM interaction when the environmental change belongs to the first category:
[0099]
[0100] Step 5.3: If only the constraints are changing, select individuals from the constraint archive as parents and pass them into the LLM and prompt it to generate offspring solutions, and add the offspring solutions to P ′ , this step produces offspring that are 2 / 3 of the population, and the remaining individuals are random solutions to increase population diversity; the following is the prompt strategy of LLM interaction when the environmental change belongs to the second category:
[0101]
[0102] Step 5.4: If both of them change, select a part of individuals from the constraint archive and the target archive as parents to generate offspring, and add the offspring solution to P ′ , the remaining operations are the same as the other two categories; the following is the prompt strategy of LLM interaction when the environmental change belongs to the third category:
[0103]
[0104] Step 6 is as follows:
[0105] If the target changes to a certain extent, Δ F The degree of constraint change Δ C If both are less than 0.5, it is determined that the environment has not changed. In this case, the saved historical information needs to be used to call LLM for interaction so that it can generate offspring by learning the hidden information in the historical information. At the same time, a prompt strategy needs to be given:
[0106]
[0107] Step 7 is implemented according to the following steps:
[0108] Step 7.1: Return the generated offspring and connect them to the deep reinforcement learning framework, evaluate their reward values and return them. The d-dimensional decision variables of each individual correspond to the network parameters of each layer of reinforcement learning. Through evolutionary selection, the strategy network with the maximum reward value is finally obtained.
[0109]
[0110] Where q(s, a) is the state-action value function, which is the maximum sum of rewards obtained under strategy π, R(s, a) is the reward value obtained by executing action a in the current state s, γ is used as a discount factor to control the proportion of learning future rewards, and maxq(s′, a′) represents the maximum reward obtained by executing action a′ in the future state s′.
[0111] R(s,a)=ω0·b(s,a)+ω1·r attack +ω1·r avoid (10)
[0112] In the formula, ω0, ω1, ω2 represent the proportion of each part, b(s, a) represents the remaining health of executing action a in state s, r attack Represents the damage taken by attacking the enemy, r avoid Representing the harm done to oneself;
[0113] Step 7.2: Update the population and each archive using the reward value returned by the offspring solution as the target evaluation value, and mainly adopt the dominance principle during the update.
[0114] Through the above content, the present invention adopts a dynamic constraint multi-objective evolution method based on the interaction of a large language model, conducts in-depth research and analysis on the environmental detection part under a dynamic constraint environment, and considers from the perspective of the sensitivity of decision variables to targets and constraints. First, the population is divided into two types: sensitive to targets and sensitive to constraints, which solves the problem of no targeted response strategy when the environment changes in the later stage; In addition, the environmental changes are divided into three types: only the target is changing, only the constraint is changing, and both are changing. Then, according to the type of environmental change, the large language model is called to generate offspring through interaction; the large language model is a black box decision process, and the problem of over-reliance on human experience in traditional operator design is solved by black box operation; At the same time, each iteration update of the population needs to pass part of the non-dominated optimal solution into the LLM, and use its ability to save historical information to help the later response strategy part to produce a better solution. Based on this, the present invention is combined with a deep reinforcement learning framework and applied to the game industry to help create game artificial intelligence with diverse behavioral characteristics.
Claims
1. A dynamic constrained multi-objective evolutionary method based on large language model interaction, characterized in that: Follow the steps below to implement it: Step 1, initialization parameters; Step 2: Divide the population into target type and constraint type; Step 3: Pass the non-dominated individuals to the large language model to save historical information; Step 4: Detect whether the environment has changed, and if so, classify the type of environment change; Step 5: If the environment changes, the interaction strategy with the large language model is initiated to generate a response individual according to the type of environmental change; Step 6: If the environment has not changed, the LLM interaction is used to generate offspring based on the historical information saved in step 3; Step 7: Connect the deep learning framework to return the reward value as the objective function for individual evaluation, select non-dominated individuals, and update the population and archive; Step 8: Determine whether the number of fitness evaluations is less than the maximum number of function evaluations; If satisfied, output the optimal solution and optimal value of the population; otherwise return to step 2 and enter the next generation of population evolution.
2. The dynamic constraint multi-objective evolution method based on large language model interaction according to claim 1 is characterized in that: In the step 1, the initialization parameters, including the population size N, the target archive F and the constraint archive C, are all initialized to N / 2, and the size of the non-dominated archive A is N; N individuals are initialized as the initial population.
3. The dynamic constraint multi-objective evolution method based on large language model interaction according to claim 1 is characterized in that: The step 2 is specifically implemented according to the following steps: Step 2.1: The specific classification steps of decision variables are as follows: Gaussian perturbation of a certain dimension of individual decision variables: In the formula, is the individual after disturbance, σ is a very small number, u j and l j are the upper and lower bounds of the population, respectively, and d is the population dimension; Step 2.2: Compare the individual constraint difference Δ after the disturbance with the individual constraint difference Δ before the disturbance cj , if Δ cj If it is greater than 0.5, it is classified as a constraint-sensitive decision variable, otherwise it is a target-type decision variable; if for a d-dimensional individual, the number of constraint-type decision variables in the decision variables is greater than the number of target-type decision variables, then the individual is considered to be constraint-sensitive and classified as a constraint-type archive C, otherwise it is classified as a target-type archive F: In the formula, is the individual after a certain dimension of disturbance The constrained difference from x, Δ ci is the normalized constraint difference, is the minimum constraint value of the g-th constraint, is the maximum constraint value of the gth constraint.
4. The dynamic constraint multi-objective evolution method based on large language model interaction according to claim 1 is characterized in that: The step 3 is specifically implemented according to the following steps: Step 3.1, sort the individuals in the current environment by non-dominance, obtain the number of layers of the Pareto frontier where each individual is located, select the best individual in the first layer and put it into the non-dominated archive A, which can store non-dominated individuals in at most two environments; Step 3.2: Connect to the large language model API interface and pass some non-dominated individuals into the LLM.
5. The dynamic constraint multi-objective evolution method based on large language model interaction according to claim 1 is characterized in that: The step 4 is specifically implemented according to the following steps: Step 4.1: Select some individuals in the non-dominated archive in the t-1 environment and the t environment for environmental detection, and compare the degree of change of the objective function of individuals in the t-1 and t environments Δ F If the value is greater than 0.5, compare the degree of constraint change Δ C If the constraint change degree is less than 0.5, it means that there is no change and the objective function is changing. Otherwise, it means that both the objective function and the constraint condition are changing. If the constraint change degree between individual t-1 and environment t is greater than 0.5, the objective function change degree Δ F If it is less than 0.5, it is classified as a changing constraint; In the formula, Δ F Indicates the degree of change of the objective function between t-1 and t, N choice represents the number of individuals used to detect whether the environment has changed. represents the objective function value of the kth individual in the t-1 environment, represents the objective function value of the kth individual in the t environment, represents the maximum objective function value of the individual in the t-1 environment, It represents the minimum objective function value of an individual under environment t; In the formula, Δ C Indicates the degree of change of constraints between t-1 and t, N chioce represents the number of individuals used to detect whether the environment has changed. represents the constraint value of the kth individual in the t-1 environment, represents the constraint value of the kth individual in environment t, represents the maximum constraint value of the individual in the t-1 environment, Represents the minimum constraint value of an individual under environment t; Step 4.2: Determine the type of environmental change. The three types are as follows: Category 1: The target environment is changing, but the constraints remain unchanged: The second category: the target environment remains unchanged, but the constraints change: The third category: The target environment is changing, and the constraints are also changing:
6. The dynamic constraint multi-objective evolution method based on large language model interaction according to claim 1 is characterized in that: The step 5 is specifically implemented according to the following steps: Step 5.1: Determine the type of environmental change, and first select feasible solutions from the t-1 environment to add to P′, where P′ is the population in the new environment; Step 5.2, if only the objective function is changing, select individuals from the target archive F as parents and pass them into LLM and prompt it to generate offspring solutions, and add the offspring solutions to P′. This step generates offspring that are 2 / 3 of the population, and the remaining individuals are random solutions to increase population diversity; Step 5.3, if only the constraints are changing, select individuals from the constraint archive as parents and pass them into the LLM and prompt it to generate offspring solutions, and add the offspring solutions to P′. This step generates offspring that are 2 / 3 of the population, and the remaining individuals are random solutions to increase population diversity; Step 5.4: If both of them change, select a part of individuals from the constraint archive and the target archive as parents to generate offspring, and add the offspring solutions to P′. The rest of the operations are the same as the other two categories.
7. The dynamic constraint multi-objective evolution method based on large language model interaction according to claim 1 is characterized in that: The step 6 is specifically as follows: if the target change degree Δ F The degree of constraint change Δ C If both are less than 0.5, it is determined that the environment has not changed. In this case, the saved historical information is needed to call LLM for interaction so that it can generate offspring by learning the hidden information in the historical information. At the same time, the LLM prompt strategy needs to be given.
8. The dynamic constraint multi-objective evolution method based on large language model interaction according to claim 1 is characterized in that: The step 7 is specifically implemented according to the following steps: Step 7.1: Return the generated offspring and connect them to the deep reinforcement learning framework, evaluate their reward values and return them. The d-dimensional decision variables of each individual correspond to the network parameters of each layer of reinforcement learning, and finally obtain the strategy network with the maximum reward value through evolutionary selection; In the formula, q (,a) is the state-action value function, which is the maximum reward sum obtained under strategy π, R(s,a) is the reward value obtained by executing action a in the current state s, γ is used as a discount factor to control the proportion of learning future rewards, maxq (′,′) Represents the maximum reward obtained by executing action a′ in the future state s′; R(s,a)0·b(s,a)+ω1· attack +1·r avoid (10) In the formula, ω0, ω1, ω2 represent the proportion of each part, b(s,) represents the remaining health of executing action a in state s, r attack Represents the damage taken by attacking the enemy, r avoid Representing the harm done to oneself; Step 7.2: Update the population and each archive using the reward value returned by the offspring solution as the target evaluation value, and mainly adopt the dominance principle during the update.
Citation Information
Cited By
Interactive speech synthesis method and system based on emotion regulation and control, medium and product
CN120260537A
Concurrent debugging method and system based on sandbox-llm session bidirectional binding
CN120723618B
Dynamic multi-objective optimization method and device for group coevolution, equipment and medium
CN121328325A
Dynamic multi-objective optimization method, device and equipment for population co-evolution and medium
CN121328325B
Large and small model collaborative optimization method and system based on rational number fusion and storage medium
CN122088553A
