Intelligent agent decision optimization method and system based on reinforcement learning

Abstract

The invention relates to an intelligent agent decision optimization method and system based on reinforcement learning, and the method comprises the steps: obtaining current state data from a plurality of agents through environment interaction, and constructing an agent group initial decision model; simulating dynamic environment change according to the agent group initial decision model, updating individual strategy parameters by adopting reinforcement learning, and determining group collaboration parameters; based on the group collaboration parameters, strategy evolution is carried out, and decision features after evolution are obtained; based on the evolved decision-making features, optimizing an initial decision-making model of the agent group; and deploying and testing the optimized agent group initial decision model in a dynamic environment, obtaining feedback data, and determining a final adaptive decision strategy. Through adaptive feature extraction and collaborative optimization, the decision-making efficiency and robustness of the multi-intelligent agent in a complex dynamic environment are remarkably improved, and an efficient solution is provided for group intelligent decision-making.

Description

Technical Field

[0001] This invention relates to the field of intelligent agent technology, and in particular to a method and system for optimizing intelligent agent decision-making based on reinforcement learning. Background Technology

[0002] Intelligent agent decision optimization is a crucial research area in artificial intelligence, its core being the construction of systems capable of autonomously learning and optimizing their decisions. These systems are critical in complex scenarios such as autonomous driving, financial trading, and industrial control, as they require rapid, high-quality decision-making in dynamic and uncertain environments. Compared to traditional manually designed rules, intelligent agents learn through interaction with the environment, adapting to complex changes and significantly improving decision-making efficiency and robustness. This makes research in this field not only theoretically valuable but also essential for efficiency improvement and risk control in practical applications.

[0003] However, existing methods face significant limitations in optimizing intelligent agent decision-making. Many solutions rely too heavily on a single optimization strategy, leading to poor agent performance in diverse or highly dynamic environments. For example, traditional reinforcement learning methods typically focus on training individual agents, lacking group collaboration mechanisms. This approach makes it difficult for agents to develop diverse decision-making strategies in complex environments, thus limiting their adaptability to unknown scenarios. Furthermore, existing methods often neglect the differences between individual agents during the optimization process, resulting in limited improvement in overall decision-making ability, especially in scenarios requiring long-term evolution and rapid adaptation.

[0004] Against this backdrop, the core technical challenges of optimizing intelligent agent decision-making are becoming increasingly apparent. The primary issue is the insufficient co-evolutionary capability of agent groups in dynamic environments. Co-evolution requires multiple agents to develop diverse decision-making strategies through competition and cooperation in a shared environment, but existing technologies struggle to effectively balance the differences between individual agents with the improvement of overall group performance. For example, one agent may excel at rapid response, while another may be better at long-term planning. Enabling these agents to learn from each other and optimize overall performance within the group becomes a key technical bottleneck.

[0005] This problem further leads to another core technical challenge: how to preserve and propagate superior decision-making features during population evolution. Given the diverse network structures and parameter configurations of individual agents, how to propagate superior decision-making features throughout the population through competitive learning and effective feature fusion mechanisms, while avoiding getting trapped in local optima, presents a complex technical challenge. Summary of the Invention

[0006] The purpose of this invention is to propose a method and system for optimizing intelligent agent decisions based on reinforcement learning, so as to solve the problems existing in the prior art.

[0007] On the one hand, to achieve the above objectives, the present invention provides the following solution: Reinforcement learning-based intelligent agent decision optimization methods include: By interacting with the environment, current state data is obtained from multiple agents to construct an initial decision-making model for the agent group. Based on the initial decision-making model of the agent group, dynamic environmental changes are simulated, and reinforcement learning is used to update individual policy parameters and determine the group collaboration parameters. Based on the aforementioned group collaboration parameters, strategy evolution is performed to obtain the evolved decision characteristics; Based on the evolved decision-making characteristics, the initial decision-making model of the agent group is optimized; The optimized initial decision-making model of the agent group is deployed and tested in a dynamic environment to obtain feedback data and determine the final adaptive decision-making strategy.

[0008] Optionally, based on the initial decision-making model of the agent group to simulate dynamic environmental changes, reinforcement learning is used to update individual policy parameters, and the group collaboration parameters are determined, including: Based on the initial decision-making model of the agent group, simulate dynamic environmental changes, obtain the state update data of each agent in the environmental interaction, and obtain the set of state changes; Based on the set of state changes, a reinforcement learning algorithm is used to update the individual policy parameters of each agent, resulting in an adjusted set of individual policies. By analyzing the collaborative interaction behavior among agents through the adjusted set of individual strategies, collaborative behavior data is obtained. If the collaborative behavior data is inconsistent with the preset group collaboration parameters, the collaborative behavior data is grouped through cluster analysis to obtain the behavior grouping results; Based on the behavioral grouping results, a Markov decision process model is constructed to optimize the group collaboration parameters and obtain an optimized set of collaboration parameters.

[0009] Optionally, based on the group coordination parameters, strategy evolution is performed to obtain the evolved decision characteristics, including: If the group collaboration parameter is lower than the preset threshold, then individual differences are integrated through a competition and cooperation mechanism to obtain a diverse set of strategies; Genetic algorithm evolution is performed on a diverse set of strategies to retain excellent decision characteristics and obtain the evolved decision characteristics.

[0010] Optionally, if the group coordination parameter is lower than a preset threshold, then individual differences are integrated through a competition-cooperation mechanism to obtain a diverse set of strategies, including: If the group coordination parameter is lower than the preset threshold, the group behavior data is used to obtain the group coordination parameter value. The group behavior data is processed using a difference analysis algorithm to determine the set of individual differences. A competition-cooperation mechanism is used to merge the set of individual differences to generate an initial strategy set; Based on the initial policy set, the policies are classified using a clustering algorithm to determine the classification policy set; The evaluation module performs collaborative optimization on the set of classification strategies to determine whether the group collaboration parameters have reached the preset threshold. If the group collaboration parameters are still below the preset threshold, the competition and cooperation mechanism parameters are adjusted through an iterative update mechanism to generate an updated set of diverse strategies.

[0011] Optionally, a genetic algorithm is used to evolve the diverse set of strategies, preserving the best decision-making features, resulting in the following evolved decision-making features: An initial policy population is obtained from a diverse policy set using a genetic algorithm, and the performance of each policy is evaluated using a fitness function to obtain a subset of superior policies. If the fitness value of the subset of superior strategies is higher than the preset threshold, the corresponding decision features are retained and stored in a temporary feature set. If the value is below the threshold, the corresponding strategy is eliminated. Based on the temporary feature set, crossover and mutation operations are used to generate a new policy population, resulting in an updated policy set. The updated policy set is evolved using an iterative genetic algorithm to obtain the optimal decision features of each generation and store them in the evolution feature set. If the number of features in the evolutionary feature set reaches a preset threshold, the iteration stops, and the final evolutionary policy features are obtained.

[0012] Optionally, optimizing the initial decision-making model of the agent group based on the evolved decision-making characteristics includes: Highly adaptive features are extracted from the post-evolutionary decision feature library and injected into low-performance agents through a propagation mechanism to determine the feature integration effect; If the feature integration effect is higher than the preset threshold, the overall performance is optimized based on long-term evolution simulation to obtain the optimized group decision model.

[0013] Optionally, if the feature integration effect is higher than a preset threshold, the overall performance is optimized based on long-term evolution simulation to obtain an optimized group decision-making model, including: If the feature integration effect is higher than the preset threshold, the injected data is processed by the feature extraction method to obtain the feature integration result; Based on the feature integration results, long-term evolutionary simulation parameters were adjusted to obtain the group behavior patterns. By analyzing the patterns of group behavior, the feature weight allocation is optimized to obtain the adjusted weight set; If the adjusted weight set meets the performance requirements of the decision model, the population decision model is updated using a genetic algorithm to obtain a temporary optimized model. Based on the temporary optimization model, the number of simulation iterations is used to control the long-term evolution simulation to obtain the evolutionary stable state; By evolving to a stable state, the model optimization strategy is updated to obtain the optimized model; If the performance of the optimized model is higher than the preset threshold, the optimized model is output; otherwise, the feature weight allocation is repeatedly adjusted to obtain the final optimized model.

[0014] On the other hand, to achieve the above objectives, the present invention provides the following solution: A reinforcement learning-based intelligent agent decision optimization system, comprising: The building module is used to obtain current state data from multiple agents through environmental interaction and build an initial decision model for the agent group; The reinforcement learning module is used to simulate dynamic environmental changes based on the initial decision-making model of the agent group, update individual policy parameters using reinforcement learning, and determine the group collaboration parameters. The evolution module is used to perform strategy evolution based on the group coordination parameters and obtain the evolved decision characteristics; The optimization module optimizes the initial decision-making model of the agent group based on the evolved decision-making characteristics. The strategy output module deploys and tests the optimized initial decision-making model of the agent group in a dynamic environment, obtains feedback data, and determines the final adaptive decision-making strategy.

[0015] Optionally, the evolution module includes: The fusion unit is used to determine that the group coordination parameter is low. If the group coordination parameter is lower than a preset threshold, the individual differences are fused through a competition and cooperation mechanism to obtain a diverse set of strategies. Genetic evolutionary units are used to perform genetic algorithm evolution on diverse policy sets, retaining excellent decision characteristics and obtaining evolved decision characteristics.

[0016] Optionally, the optimization module includes: The extraction unit is used to extract highly adaptive features from the post-evolutionary decision feature library, inject them into low-performance agents through a propagation mechanism, and judge the feature integration effect. The simulation optimization unit is used to optimize the overall performance based on long-term evolution when the feature integration effect is higher than a preset threshold, and obtain the optimized group decision model.

[0017] The beneficial effects of this invention are as follows: This invention discloses a method and system for intelligent agent decision optimization based on reinforcement learning. Addressing the problems of imperfect decision models, insufficient collaboration, and poor adaptability to dynamic environments in multi-agent systems, the method acquires state data from multi-agent interactions to construct an initial decision model. Reinforcement learning is then used to dynamically update individual policy parameters and determine group collaboration parameters. When the collaboration parameters fall below a threshold, a competition-cooperation mechanism is introduced to integrate individual differences, forming a diverse set of strategies. A genetic algorithm is then used to evolve and retain highly adaptive features, constructing a decision feature library. Finally, a propagation mechanism injects these highly adaptive features into low-performance agents, and the group decision model is optimized based on long-term evolutionary simulation. Testing is conducted in a dynamic environment to obtain feedback and determine the final adaptive decision strategy. This invention significantly improves the decision efficiency and robustness of multi-agent systems in complex dynamic environments through adaptive feature extraction and collaborative optimization, providing an efficient solution for swarm intelligent decision-making. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the intelligent agent decision optimization method based on reinforcement learning according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of an intelligent agent decision optimization system based on reinforcement learning, according to an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] like Figure 1 As shown, this embodiment proposes an intelligent agent decision optimization method based on reinforcement learning, including: By interacting with the environment, current state data is obtained from multiple agents to construct an initial decision-making model for the agent group. Based on the initial decision-making model of the agent group, dynamic environmental changes are simulated, and reinforcement learning is used to update individual policy parameters and determine the group collaboration parameters. Based on the aforementioned group collaboration parameters, strategy evolution is performed to obtain the evolved decision characteristics; Based on the evolved decision-making characteristics, the initial decision-making model of the agent group is optimized; The optimized initial decision-making model of the agent group is deployed and tested in a dynamic environment to obtain feedback data and determine the final adaptive decision-making strategy.

[0023] Specifically, in this embodiment, constructing the initial decision-making model for the agent swarm includes: First, designing a simulation environment where multiple agents run and generate state data, such as location coordinates, energy values, and interaction frequencies. Next, using a distributed stream processing framework (such as Flink), this state data is collected in real time to ensure real-time data processing and efficient handling. Then, key features are extracted from the collected data, such as average energy, location distribution, and variance of interaction frequencies; these features describe the static distribution characteristics of the agent swarm. Finally, based on these features, a simple initial decision-making model is constructed, which describes the swarm characteristics and initially simulates the agent's decision-making behavior. This model will serve as the basis for subsequent optimization and dynamic environment adaptation.

[0024] Furthermore, based on the initial decision-making model of the agent group and simulating dynamic environmental changes, reinforcement learning is used to update individual policy parameters, and the group collaboration parameters are determined, including: Based on the initial decision-making model of the agent group, simulate dynamic environmental changes, obtain the state update data of each agent in the environmental interaction, and obtain the set of state changes; Based on the set of state changes, a reinforcement learning algorithm is used to update the individual policy parameters of each agent, resulting in an adjusted set of individual policies. By analyzing the collaborative interaction behavior among agents through the adjusted set of individual strategies, collaborative behavior data is obtained. If the collaborative behavior data is inconsistent with the preset group collaboration parameters, the collaborative behavior data is grouped through cluster analysis to obtain the behavior grouping results; Based on the behavioral grouping results, a Markov decision process model is constructed to optimize the group collaboration parameters and obtain an optimized set of collaboration parameters.

[0025] Specifically, in this embodiment, exemplarily, agent group behavior is simulated in a dynamic environment. It is assumed that there are 200 agents in the environment, each generating state data every 2 seconds, including location coordinates (x, y), energy value e, and interaction frequency f. For example, agent B's data at t=2 seconds is (x=15.7, y=22.4, e=3.5, f=2.1). Data is collected using the distributed stream processing framework Flink, with the topic "AgentDynamic," 8 partitions, and stored in JSON format, such as {"agent_id":"B", "timestamp":2, "x":15.7, "y":22.4, "e":3.5, "f":2.1}. Historical interaction records are stored in a Cassandra database. For example, if agent B selects "reduce frequency" at t=1 seconds, the record is {"agent_id":"B", "time":1, "action":"reduce_frequency", "reward":0.6}. Data is processed every 10 seconds using Flink to calculate population characteristics, such as average energy μ_e = ∑e_i / 200 = 3.2 and interaction frequency variance σ_f = 0.9. The DBSCAN algorithm (ε = 2.5, MinPts = 5) is used for clustering, generating three behavioral pattern clusters centered at D1 (μ_x = 14.2, μ_y = 20.1), D2 (μ_x = 10.5, μ_y = 28.3), and D3 (μ_x = 18.9, μ_y = 12.7). Based on this, the reinforcement learning SARSA algorithm is used to update individual policies. The state space is (cluster label, energy range), and the action space is {increase frequency, decrease frequency, maintain}. The update formula is Q(s,a)←Q(s,a)+α[r+γQ(s',a')-Q(s,a)], where α=0.15, γ=0.85, and the reward r is based on cooperative efficiency (e.g., an increase of 0.5 in interaction frequency earns 0.3 points). After the Q-table convergence, the group cooperative parameter is the Q value of 0.82 for "increase frequency" in state (cluster D2,e∈[3,4]), indicating that the agent tends to increase the interaction frequency in cluster D2 to optimize cooperation. Logically, the model adapts to environmental changes by dynamically adjusting the frequency, maintaining the consistency of group cooperation.

[0026] Furthermore, based on the aforementioned group coordination parameters, strategy evolution is performed to obtain the evolved decision characteristics, including: If the group collaboration parameter is lower than the preset threshold, then individual differences are integrated through a competition and cooperation mechanism to obtain a diverse set of strategies; Genetic algorithm evolution is performed on a diverse set of strategies to retain excellent decision characteristics and obtain the evolved decision characteristics.

[0027] Furthermore, if the group coordination parameter is lower than a preset threshold, then a competitive cooperation mechanism is used to integrate individual differences to obtain a diverse set of strategies, including: If the group coordination parameter is lower than the preset threshold, the group behavior data is used to obtain the group coordination parameter value. The group behavior data is processed using a difference analysis algorithm to determine the set of individual differences. A competition-cooperation mechanism is used to merge the set of individual differences to generate an initial strategy set; Based on the initial policy set, the policies are classified using a clustering algorithm to determine the classification policy set; The evaluation module performs collaborative optimization on the set of classification strategies to determine whether the group collaboration parameters have reached the preset threshold. If the group collaboration parameters are still below the preset threshold, the competition and cooperation mechanism parameters are adjusted through an iterative update mechanism to generate an updated set of diverse strategies.

[0028] Specifically, in this embodiment, exemplarily, in the evaluation of group collaboration parameters, the collaboration parameter value is first calculated by quantifying group collaboration behavior. Assuming the group contains 100 individuals, each individual's contribution value is a random number between 0 and 1. The collaboration parameter is defined as the average of the individual contribution values, with a preset threshold of 0.6. The average value is calculated using a Python algorithm: mean_contribution = sum(contributions) / len(contributions). If the result is 0.55, which is lower than 0.6, a fusion mechanism is triggered. Next, individual differences are fused through a competition-cooperation mechanism, employing a game theory-based hybrid strategy model, specifically Nash equilibrium calculation. Each individual is assigned an initial weight based on their contribution value (e.g., an individual with a contribution value of 0.8 has a weight of 0.8). The weight is adjusted iteratively using the formula new_weight = old_weight * (1 + fitness_score / total_fitness), where fitness_score is the individual strategy payoff and total_fitness is the total group payoff. Assuming an individual's initial policy payoff is 10 and the total payoff is 500, the weights are updated to `0.8 * (1 + 10 / 500) = 0.816`. After 10 iterations, the weights stabilize, generating a diverse policy set. This diverse policy set is then used to classify individual policies using a K-means clustering algorithm (K=5), with contribution values ​​and weights as input features, outputting 5 policy clusters. Analysis shows that a low collaboration parameter reflects insufficient group consistency; after fusion, the intra-cluster variance decreased from 0.15 to 0.08, and policy diversity increased by 20%, validating the fusion effect. All computations are executed automatically through a distributed system, ensuring efficient processing.

[0029] Furthermore, a genetic algorithm is used to evolve the diverse set of strategies, preserving the best decision-making characteristics. The evolved decision-making characteristics include: An initial policy population is obtained from a diverse policy set using a genetic algorithm, and the performance of each policy is evaluated using a fitness function to obtain a subset of superior policies. If the fitness value of the subset of superior strategies is higher than the preset threshold, the corresponding decision features are retained and stored in a temporary feature set. If the value is below the threshold, the corresponding strategy is eliminated. Based on the temporary feature set, crossover and mutation operations are used to generate a new policy population, resulting in an updated policy set. The updated policy set is evolved using an iterative genetic algorithm to obtain the optimal decision features of each generation and store them in the evolution feature set. If the number of features in the evolutionary feature set reaches a preset threshold, the iteration stops, and the final evolutionary policy features are obtained.

[0030] Specifically, in this embodiment, an initial policy population is obtained from a diverse policy set using a genetic algorithm. The performance of each policy is evaluated using a fitness function to obtain a subset of superior policies. If the fitness value of the superior policy subset is higher than a preset threshold, the corresponding decision feature is retained and stored in a temporary feature set; if it is lower than the threshold, the corresponding policy is removed. Based on the temporary feature set, crossover and mutation operations are used to generate a new policy population, resulting in an updated policy set. The updated policy set is evolved using an iterative genetic algorithm to obtain the optimal decision feature of each generation, which is stored in an evolutionary feature set. If the number of features in the evolutionary feature set reaches a preset threshold, iteration stops, resulting in the final decision feature library; otherwise, iteration continues. Based on the final decision feature library, a clustering algorithm is used to classify the features, resulting in a classified feature subset. A structured feature storage system is constructed using the classified feature subsets and stored in the decision feature library.

[0031] Furthermore, based on the evolved decision-making characteristics, the optimization of the initial decision-making model for the agent group includes: Highly adaptive features are extracted from the post-evolutionary decision feature library and injected into low-performance agents through a propagation mechanism to determine the feature integration effect; If the feature integration effect is higher than the preset threshold, the overall performance is optimized based on long-term evolution simulation to obtain the optimized group decision model.

[0032] Furthermore, if the feature integration effect exceeds a preset threshold, the overall performance is optimized based on long-term evolution simulation to obtain an optimized group decision-making model, including: If the feature integration effect is higher than the preset threshold, the injected data is processed by the feature extraction method to obtain the feature integration result; Based on the feature integration results, long-term evolutionary simulation parameters were adjusted to obtain the group behavior patterns. By analyzing the patterns of group behavior, the feature weight allocation is optimized to obtain the adjusted weight set; If the adjusted weight set meets the performance requirements of the decision model, the population decision model is updated using a genetic algorithm to obtain a temporary optimized model. Based on the temporary optimization model, the number of simulation iterations is used to control the long-term evolution simulation to obtain the evolutionary stable state; By evolving to a stable state, the model optimization strategy is updated to obtain the optimized model; If the performance of the optimized model is higher than the preset threshold, the optimized model is output; otherwise, the feature weight allocation is repeatedly adjusted to obtain the final optimized model.

[0033] Specifically, in this embodiment, for example, when extracting high-fitness features from the evolved decision feature library, a feature selection method based on a genetic algorithm can be used. Assume the feature library contains 1000 features, each represented by a fitness score ranging from 0 to 1. First, calculate the performance of each feature in historical decision tasks. For example, a feature might have an average accuracy of 0.85 and a standard deviation of 0.1 in 100 tasks. Using a genetic algorithm, with a population size of 50, a crossover rate of 0.8, a mutation rate of 0.01, and 100 generations, select features with fitness scores higher than 0.8, thus filtering out 200 high-fitness features. Next, inject these features into low-performance agents through a propagation mechanism. A gradient-based propagation algorithm can be used, based on a decision tree, with a learning rate of 0.1, a tree depth of 5, and 200 iterations. Assuming the initial accuracy of the low-performance agent is 0.6, after feature injection, its decision model parameters are updated using the formula w_new = w_old + η·▽L, where η is the learning rate and ▽L is the loss gradient. After injection, the agent's accuracy on the test set improves to 0.75, and the standard deviation decreases to 0.08. To evaluate the feature integration effect, cross-validation is used, with 5-fold cross-validation, to calculate accuracy, F1 score, and AUC. Assuming the integrated agent's F1 score on the test set is 0.78 and AUC is 0.82, a significant improvement compared to 0.65 and 0.70 before injection, respectively, a t-test analysis (p < 0.05) confirms the statistical significance of the integration effect.

[0034] For example, in the feature integration stage, assuming the input dataset contains 1000 samples, each with a 10-dimensional feature vector, principal component analysis (PCA) is first used to reduce the dimensionality of the features, retaining the principal components with a cumulative variance contribution rate of 95%. The first three principal components are calculated, resulting in a 3-dimensional feature vector, thus reducing computational complexity. Subsequently, the Pearson correlation coefficient between features is calculated, and feature pairs with an absolute correlation coefficient greater than 0.8 are selected to remove redundant features, ultimately retaining 2-dimensional features. The integration effect is evaluated using the feature information gain ratio. Assuming the gain ratio reaches 0.85, which is higher than the preset threshold of 0.8, the next optimization step is performed. Long-term evolutionary simulation uses a genetic algorithm, initializing the population size to 100, with individuals representing the decision model parameter set. The fitness function is the model's F1 score on the validation set, with an initial F1 score of 0.75. The genetic algorithm iterates for 50 generations, with a crossover probability of 0.9 and a mutation probability of 0.1. The top 20% of individuals in each generation are selected for crossover and mutation to simulate natural selection. After iteration, the optimal individual's F1 score increases to 0.82. The optimized parameters were used to construct a group decision-making model, employing a weighted voting mechanism. The weights were based on the accuracy of each sub-model on the validation set (e.g., sub-model 1 had an accuracy of 0.9, with a weight of 0.45; sub-model 2 had an accuracy of 0.85, with a weight of 0.4). The final group decision-making model achieved an accuracy of 0.88 on the test set, a 13.33% improvement over the initial model.

[0035] Specifically, in this embodiment, the optimized group decision-making model is deployed and tested in a dynamic environment to obtain feedback data and determine the final adaptive decision-making strategy. This includes: For example, deploying the optimized group decision-making model in a dynamic environment involves first generating test data by simulating the dynamic environment. Assume the environment contains 100 decision nodes, each with 5 features (such as resource allocation and response time), and feature values ​​are randomly generated in the range [0, 100] to simulate market fluctuations. A genetic algorithm is used to optimize the group decision-making model, setting the population size to 200, crossover probability to 0.8, mutation probability to 0.01, and iterations to 100 generations. The objective function is to minimize the decision error (error is defined as the root mean square error between the actual and predicted results). During the deployment phase, a distributed computing framework (such as Apache Spark) is used to process 1000 sets of test data in real time on the cloud. Each set contains the decision results of 100 nodes, and the average response time (e.g., 0.5 seconds) and decision accuracy (e.g., 95%) for each set are calculated. When acquiring feedback data, a feedback collection module was designed to extract the error value and environmental parameters of each decision group from the test environment via an API interface, storing them in JSON format, with a data volume of approximately 10MB. When analyzing the feedback data, a K-means clustering algorithm (K=5) was used to classify the error data, identify high-error decision patterns (e.g., error > 10%), and calculate the mean and variance of each category (e.g., mean 5.2, variance 1.8). Based on the analysis results, the final adaptive decision-making strategy was determined.

[0036] like Figure 2 As shown, this embodiment also proposes an intelligent agent decision optimization system based on reinforcement learning, including: The building module is used to obtain current state data from multiple agents through environmental interaction and build an initial decision model for the agent group; The reinforcement learning module is used to simulate dynamic environmental changes based on the initial decision-making model of the agent group, update individual policy parameters using reinforcement learning, and determine the group collaboration parameters. The evolution module is used to perform strategy evolution based on the group coordination parameters and obtain the evolved decision characteristics; The optimization module optimizes the initial decision-making model of the agent group based on the evolved decision-making characteristics. The strategy output module deploys and tests the optimized initial decision-making model of the agent group in a dynamic environment, obtains feedback data, and determines the final adaptive decision-making strategy.

[0037] Furthermore, the evolution module includes: The fusion unit is used to determine that the group coordination parameter is low. If the group coordination parameter is lower than a preset threshold, the individual differences are fused through a competition and cooperation mechanism to obtain a diverse set of strategies. Genetic evolutionary units are used to perform genetic algorithm evolution on diverse policy sets, retaining excellent decision characteristics and obtaining evolved decision characteristics.

[0038] Furthermore, the optimization module includes: The extraction unit is used to extract highly adaptive features from the post-evolutionary decision feature library, inject them into low-performance agents through a propagation mechanism, and judge the feature integration effect. The simulation optimization unit is used to optimize the overall performance based on long-term evolution when the feature integration effect is higher than a preset threshold, and obtain the optimized group decision model.

[0039] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for optimizing intelligent agent decision-making based on reinforcement learning, characterized in that, include: By interacting with the environment, current state data is obtained from multiple agents to construct an initial decision-making model for the agent group. Based on the initial decision-making model of the agent group, dynamic environmental changes are simulated, and reinforcement learning is used to update individual policy parameters and determine the group collaboration parameters. Based on the aforementioned group collaboration parameters, strategy evolution is performed to obtain the evolved decision characteristics; Based on the evolved decision-making characteristics, the initial decision-making model of the agent group is optimized; The optimized initial decision-making model of the agent group is deployed and tested in a dynamic environment to obtain feedback data and determine the final adaptive decision-making strategy.

2. The intelligent agent decision optimization method based on reinforcement learning according to claim 1, characterized in that, Based on the initial decision-making model of the agent group, dynamic environmental changes are simulated, and reinforcement learning is used to update individual policy parameters to determine the group collaboration parameters, including: Based on the initial decision-making model of the agent group, simulate dynamic environmental changes, obtain the state update data of each agent in the environmental interaction, and obtain the set of state changes; Based on the set of state changes, a reinforcement learning algorithm is used to update the individual policy parameters of each agent, resulting in an adjusted set of individual policies. By analyzing the collaborative interaction behavior among agents through the adjusted set of individual strategies, collaborative behavior data is obtained. If the collaborative behavior data is inconsistent with the preset group collaboration parameters, the collaborative behavior data is grouped through cluster analysis to obtain the behavior grouping results; Based on the behavioral grouping results, a Markov decision process model is constructed to optimize the group collaboration parameters and obtain an optimized set of collaboration parameters.

3. The intelligent agent decision optimization method based on reinforcement learning according to claim 1, characterized in that, Based on the aforementioned group collaboration parameters, strategy evolution is performed to obtain the evolved decision characteristics, including: If the group collaboration parameter is lower than the preset threshold, then individual differences are integrated through a competition and cooperation mechanism to obtain a diverse set of strategies; Genetic algorithm evolution is performed on a diverse set of strategies to retain excellent decision characteristics and obtain the evolved decision characteristics.

4. The intelligent agent decision optimization method based on reinforcement learning according to claim 3, characterized in that, If the group coordination parameter is lower than a preset threshold, then individual differences are integrated through a competition and cooperation mechanism to obtain a diverse set of strategies, including: If the group coordination parameter is lower than the preset threshold, the group behavior data is used to obtain the group coordination parameter value. The group behavior data is processed using a difference analysis algorithm to determine the set of individual differences. A competition-cooperation mechanism is used to merge the set of individual differences to generate an initial strategy set; Based on the initial policy set, the policies are classified using a clustering algorithm to determine the classification policy set; The evaluation module performs collaborative optimization on the set of classification strategies to determine whether the group collaboration parameters have reached the preset threshold. If the group collaboration parameters are still below the preset threshold, the competition and cooperation mechanism parameters are adjusted through an iterative update mechanism to generate an updated set of diverse strategies.

5. The intelligent agent decision optimization method based on reinforcement learning according to claim 3, characterized in that, Genetic algorithm evolution is performed on a diverse set of strategies, preserving excellent decision characteristics to obtain the evolved decision characteristics, including: An initial policy population is obtained from a diverse policy set using a genetic algorithm, and the performance of each policy is evaluated using a fitness function to obtain a subset of superior policies. If the fitness value of the subset of superior strategies is higher than the preset threshold, the corresponding decision features are retained and stored in a temporary feature set. If the value is below the threshold, the corresponding strategy is eliminated. Based on the temporary feature set, crossover and mutation operations are used to generate a new policy population, resulting in an updated policy set. The updated policy set is evolved using an iterative genetic algorithm to obtain the optimal decision features of each generation and store them in the evolution feature set. If the number of features in the evolutionary feature set reaches a preset threshold, the iteration stops, and the final evolutionary policy features are obtained.

6. The intelligent agent decision optimization method based on reinforcement learning according to claim 1, characterized in that, Based on the evolved decision-making characteristics, the optimization of the initial decision-making model for the agent group includes: Highly adaptive features are extracted from the post-evolutionary decision feature library and injected into low-performance agents through a propagation mechanism to determine the feature integration effect; If the feature integration effect is higher than the preset threshold, the overall performance is optimized based on long-term evolution simulation to obtain the optimized group decision model.

7. The intelligent agent decision optimization method based on reinforcement learning according to claim 6, characterized in that, If the feature integration effect is higher than a preset threshold, the overall performance is optimized based on long-term evolution simulation to obtain the optimized group decision-making model, which includes: If the feature integration effect is higher than the preset threshold, the injected data is processed by the feature extraction method to obtain the feature integration result; Based on the feature integration results, long-term evolutionary simulation parameters were adjusted to obtain the group behavior patterns. By analyzing the patterns of group behavior, the feature weight allocation is optimized to obtain the adjusted weight set; If the adjusted weight set meets the performance requirements of the decision model, the population decision model is updated using a genetic algorithm to obtain a temporary optimized model. Based on the temporary optimization model, the number of simulation iterations is used to control the long-term evolution simulation to obtain the evolutionary stable state; By evolving to a stable state, the model optimization strategy is updated to obtain the optimized model; If the performance of the optimized model is higher than the preset threshold, the optimized model is output; otherwise, the feature weight allocation is repeatedly adjusted to obtain the final optimized model.

8. An intelligent agent decision optimization system based on reinforcement learning, characterized in that, The system is used to implement the reinforcement learning-based intelligent agent decision optimization method as described in any one of claims 1-7, the system comprising: The building module is used to obtain current state data from multiple agents through environmental interaction and build an initial decision model for the agent group; The reinforcement learning module is used to simulate dynamic environmental changes based on the initial decision-making model of the agent group, update individual policy parameters using reinforcement learning, and determine the group collaboration parameters. The evolution module is used to perform strategy evolution based on the group coordination parameters and obtain the evolved decision characteristics; The optimization module optimizes the initial decision-making model of the agent group based on the evolved decision-making characteristics. The strategy output module deploys and tests the optimized initial decision-making model of the agent group in a dynamic environment, obtains feedback data, and determines the final adaptive decision-making strategy.

9. The intelligent agent decision optimization system based on reinforcement learning according to claim 8, characterized in that, The evolution module includes: The fusion unit is used to determine that the group coordination parameter is low. If the group coordination parameter is lower than a preset threshold, the individual differences are fused through a competition and cooperation mechanism to obtain a diverse set of strategies. Genetic evolutionary units are used to perform genetic algorithm evolution on diverse policy sets, retaining excellent decision characteristics and obtaining evolved decision characteristics.

10. The intelligent agent decision optimization system based on reinforcement learning according to claim 8, characterized in that, The optimization module includes: The extraction unit is used to extract highly adaptive features from the post-evolutionary decision feature library, inject them into low-performance agents through a propagation mechanism, and judge the feature integration effect. The simulation optimization unit is used to optimize the overall performance based on long-term evolution when the feature integration effect is higher than a preset threshold, and obtain the optimized group decision model.

Images