93 results about "Markov game" patented technology

The invention discloses a virtual power plant scheduling method based on a large language model and deep reinforcement learning, and belongs to the technical field of virtual power plant scheduling. Comprising the following steps: constructing a virtual power plant multi-agent cloud edge collaborative scheduling framework based on large language model driving; predicting wind power, photovoltaic power and load power based on a large language model; constructing a mathematical model of virtual power plant optimization scheduling; converting the virtual power plant optimization scheduling model into a Markov game process in combination with a large language model; performing initialization training on the strategy network of the edge layer intelligent agent by adopting imitation learning to obtain a pre-trained edge layer intelligent agent strategy network; and based on the pre-training strategy network of the boundary layer intelligent agent, combining with a large language model and adopting an improved multi-agent near-end strategy optimization algorithm to solve a scheduling strategy.

The invention relates to the technical field of wireless communication, and discloses an unmanned aerial vehicle mobile edge computing resource allocation method based on deep reinforcement learning, which constructs a mixed integer non-convex optimization problem, and the objective of the optimization problem is to collaboratively optimize the flight path of each unmanned aerial vehicle, a mobile user scheduling strategy and a task unloading proportion. Therefore, the time delay-energy consumption minimization is realized; and solving the optimization problem by adopting a multi-agent double-delay depth deterministic strategy gradient algorithm: converting the optimization problem into a Markov game model, and realizing the solution of the optimization problem through an experience playback pool and a double-evaluation network updating strategy. According to the method provided by the invention, the flight path and the unloading proportion of the unmanned aerial vehicle can be adaptively adjusted, so that calculation resource allocation for ground users is efficiently completed. Compared with a global unloading scheme, an average unloading scheme and other deep reinforcement learning methods, the method provided by the invention has excellent performance of lower energy consumption and shorter time delay while completing tasks.

The invention belongs to the technical field of power systems, and particularly relates to a transmission and distribution cooperative scheduling optimization method based on multi-agent deep reinforcement learning. The method comprises the following steps: firstly, establishing a transmission and distribution cooperative scheduling distributed framework, and respectively establishing distributed optimization models taking the minimization of the operation cost of a power transmission network and the minimization of the operation cost of each active power distribution network as targets; secondly, converting a transmission and distribution collaborative scheduling problem into a Markov game model with a multi-agent benefit balance characteristic, and designing a collaborative multi-agent architecture; then interacting with a transmission and distribution interconnection environment through each agent to obtain historical interaction information, storing the historical interaction information into an experience pool, and training by using an MADDPG algorithm to realize iterative updating and optimization of an Actor network and a Critic network of each agent; and finally, realizing local optimization control of the power transmission network and each active distribution network by using an Actor network.

The invention provides a honey array defense resource allocation optimization method based on self-game reinforcement learning, and relates to the technical field of network security defense. The optimization method comprises the following steps: deploying defense resources based on network topology, and establishing a mapping relationship between a host and the defense resources; performing attack and defense confrontation simulation based on the mapping relation to obtain an initial attack strategy and an initial defense strategy; constructing a Markov game model based on the initial attack strategy and the initial defense strategy; performing self-game reinforcement learning on attack and defense training of the Markov game model based on a depth deterministic strategy gradient algorithm; and carrying out value evaluation on the attack and defense training based on an Actor-Critic algorithm to obtain an optimal defense resource allocation strategy. By implementing the optimization method provided by the invention, the defense party and the attack party can perform self-game reinforcement learning, the defense strategy is dynamically adjusted in confrontation, and the defense effect is improved.

The invention relates to a Markov game-based satellite cluster observation resource allocation method and system. Through a two-stage decomposition strategy, a multi-target balance problem of a task integrity rate, a resource utilization rate and a cost-efficiency ratio is effectively solved. In the first stage, a genetic algorithm (GA) is adopted to complete satellite-task-time window three-dimensional matching, and optimal distribution of limited visible windows is achieved. And in the second stage, a distributed decision-making mechanism is implemented based on an improved Improved-MADDPG framework, and after an intelligent agent adopts a random game strategy to execute exploration in a distributed manner, optimal dynamic configuration of observation resources is achieved through global information sharing and local strategy iteration, and key calculation indexes such as an average reward value and the like are remarkably improved.

The invention provides an active power distribution network multi-time scale rolling optimization method driven by multi-agent graph reinforcement learning. The method comprises the following steps: dividing slow time scale equipment and fast time scale equipment; the slow time scale is optimized and modeled into a multi-agent Markov game, and a global state space and a combined action space corresponding to switch actions, reactive compensation equipment gears and voltage regulation equipment gears are defined; switching action candidate vectors are generated based on a basic loop of the power distribution network, and action space dimensions are compressed through a scheme of excluding conflicting actions of the same common branch; fitting an action value function of each agent by adopting a graph neural network, generating a dimensionality-reduced discrete action strategy according to the global state space, and carrying out collaborative optimization through a distributed decision and a global reward mechanism; and on the basis of the slow time scale optimization result, the fast time scale optimization problem is converted into a mixed integer second-order cone programming model, and an instruction is solved in a rolling manner.

The invention discloses a multi-user cooperative anti-interference strategy based on a multi-agent discrete flexible actor-commentator algorithm (MA-DSAC), and belongs to the technical field of wireless communication. The invention provides a multi-user cooperative anti-interference strategy based on a multi-agent discrete flexible actor-commentator algorithm in combination with a frequency domain, a power domain and a modulation coding domain in order to simultaneously solve external malicious interference and same-frequency interference between internal communication receiving and transmitting pairs in a multi-user scene. In the strategy, firstly, a Markov game is introduced to carry out modeling and analysis on a multi-user anti-interference problem; secondly, discretizing a flexible actor-commentator algorithm to cope with a multi-dimensional discrete action space; and finally, through information interaction between communication receiving and transmitting pairs, based on a framework of centralized training and distributed execution, the algorithm is expanded into a multi-agent deep reinforcement learning algorithm, so that efficient multi-user cooperative anti-interference is realized.

The invention discloses a mountain tourism route planning method based on weather and natural disasters, and the method comprises the steps: 1, building a Markov game model based on POMDP, combining Mama multi-modal feature compression and RAG semantic enhancement, designing a dynamic risk matrix integrating geological disaster probability, weather trend and ecological sensitivity, and providing partial observable environment state perception; 2, establishing a Nash equilibrium constraint optimization model, sharing an extreme weather strategy through federal learning, introducing an NSGA-II-Actor-Critic architecture to optimize unmanned aerial vehicle inspection and path adjustment, and triggering dynamic communication when rainfall exceeds a threshold value; 3, fusing a security and experience income design reward function, processing continuous action planning by using improved DDPG, and accelerating multi-target Pareto frontier convergence in combination with NSGA-II; 4, constructing a user weather sensitivity portrait, and dynamically adjusting weight balance income and AR simulation preview preference; and 5, based on a GIS closed-loop decision engine, performing agent and environment feature complementation, recording an emergency log by a block chain, and constructing a disaster case library by meta-learning.

The invention discloses a double-layer non-cooperative demand response method based on asynchronous multi-agent reinforcement learning, and the method comprises the steps: S1, constructing a double-layer non-cooperative game framework: constructing a double-layer dynamic game model between a public utility company and a consumer; s2, completing modeling of the partially observable Markov game, and converting the double-layer non-cooperative game into the partially observable Markov game; s3, designing an asynchronous multi-agent reinforcement learning algorithm as a game equilibrium solving engine; and S4, implementing centralized training and priority experience playback to realize asynchronous multi-agent reinforcement learning algorithm efficiency optimization. The method is suitable for a dynamic game scene between a public utility company and a consumer group in an intelligent power grid environment, and can realize efficient consumption and load fluctuation suppression of distributed energy in a power grid.

The invention discloses a control method for efficient utilization of energy in a green station area for a micro-grid group. The method comprises the following steps: constructing an active and reactive power coordinated optimization scheduling model of the micro-grid group in the green station area, and taking the minimum network loss in a scheduling period as an objective function of the model; taking constraints of power flow, photovoltaic output, a static var compensator, branch transmission power, voltage amplitude and energy storage, and limits of action times of an on-load tap-changer and a circuit breaker as constraint conditions of the model; the method comprises the following steps: defining an agent reward function by determining an agent observation space and an agent action space, and describing an active and reactive coordinated optimization scheduling model of a microgrid group in a green radio station area as a Markov game problem; and an improved distributed multi-agent soft actor-commentator algorithm and a migration reinforcement learning method are used to train the green station area microgrid group active and reactive coordination optimization scheduling model converted into the Markov game problem. According to the method, the intelligent agent can be trained to learn the optimal scheduling strategy in the process of interaction with the environment, and the dependence on the model is overcome.

The invention discloses a multi-mechanical-arm accurate assembly method for double-population evolution reinforcement learning, and the method comprises the steps: constructing a partially observable Markov game model, designing a mixed reward function, and building a double-population mechanism: an Exploit main population is responsible for optimizing the performance, and an Explore auxiliary population maintains the diversity; extracting unified feature representation through a shared observation embedded network, optimizing a main population by adopting conjugate gradient differential evolution, and optimizing an auxiliary population by adopting novelty driven evolution; promoting information interaction between the double populations through an elite exchange mechanism, and utilizing parameter migration fusion to reinforce learning and evolutionary optimization results; and finally, an optimal strategy set for balancing accumulated rewards and novelty is selected through Pareto optimization, and the optimal strategy set is deployed to a multi-mechanical-arm system to execute a cooperative task after convergence conditions are met. According to the method, the population diversity is effectively maintained and the cooperation robustness is improved while the high performance of the strategy is ensured; and a powerful technical support is provided for automatic assembly in a complex industrial scene.

The invention discloses an expandable unmanned ship cluster cooperative obstacle avoidance decision-making method, and the method comprises the steps: determining a joint state space, an action space and a reward function of an attacker unmanned ship cluster based on a Markov game model through building an unmanned ship cluster cooperative penetration task scene, and building a prediction network of an attacker unmanned ship, therefore, the accelerated speed and the angular speed of the unmanned ship of the attacking side are obtained, and the cooperative obstacle avoidance decision of the unmanned ship cluster of the attacking side is completed. The prediction model can predict the acceleration and the angular velocity of the unmanned ship of the attacking side based on multi-dimensional feature input, the prediction model has generalization and expansibility in combination with an attention mechanism, and the problem that the cluster scale changes due to collision of the unmanned ship is solved. And the remaining unmanned ships of the attacking party cannot continue to execute the defense penetration task.

The invention discloses a multi-agent collaborative power distribution network planning and operation integrated optimization method and system. The method comprises the steps of dividing main bodies of a power distribution network source, a power distribution network grid, a power distribution network load and a power distribution network storage into independent agents; constructing a double-layer multi-agent game model by combining a Markov game process based on the competition-cooperation relationship of each agent; the double-layer multi-agent game model comprises a planning layer game model and an operation layer game model; solving the planning layer game model to obtain the optimal configuration capacity of each agent, and calculating the constraint boundary in the action space of each agent of the operation layer; and solving the operation layer game model by using an MATD3 algorithm in which an enhanced action projection mechanism is introduced to obtain an optimal operation strategy and operation income of each agent, and feeding back the operation income to the planning layer game model to calculate a reward function of the planning layer game model. According to the invention, multi-time scale and multi-main-body cooperation characteristics can be fused, so that the system operation efficiency is improved.

The embodiment of the invention provides a multi-unmanned aerial vehicle data acquisition task planning method and device considering communication interruption. The method comprises the steps that a working scene is set, the working scene comprises a plurality of unmanned aerial vehicles and a plurality of to-be-collected data points (PoI), and the unmanned aerial vehicles need to plan routes to go to the PoI to execute data collection tasks; under the working scene, an unmanned aerial vehicle data acquisition model and an unmanned aerial vehicle communication interruption model are established, an optimization target of a multi-unmanned aerial vehicle data acquisition task planning problem considering communication interruption is set as data quality sum maximization, and corresponding constraint conditions are set as position constraint, energy constraint and communication interruption risk constraint. Based on this, a partially observable Markov game model of a multi-unmanned aerial vehicle data acquisition task planning problem considering communication interruption is established, and a multi-agent reinforcement learning algorithm is adopted to solve the partially observable Markov game model. In this way, multi-unmanned aerial vehicle data acquisition task planning can be performed in consideration of communication interruption.

The invention discloses an electric vehicle charging management method based on an HA-SAC algorithm, and the method comprises the steps: firstly, carrying out the modeling of an electric vehicle charging management optimization model through fully considering the life loss of a transformer and the charging preference of a user; modeling a charging management optimization problem as Markov Game (MG for short), taking each electric vehicle as an independent intelligent agent, constructing a state space according to information such as the charging state, the electricity price and the user preference, and taking the charging power as an action variable; then, on the basis of an HA-SAC algorithm, a local action network and a centralized evaluation network are constructed for each agent, and centralized training is carried out through global information; and finally, the trained strategy is deployed to each electric vehicle agent node, so that the electric vehicle agent node can autonomously decide a charging behavior only by depending on local information in actual operation, and distributed and privacy-protected cooperative charging control is realized.

The invention provides a power distribution network voltage control method based on multi-agent deep reinforcement learning, and relates to the technical field of power distribution networks, and the method comprises the steps: dividing a power distribution network into a plurality of regions, and carrying out the centralized management and optimization of the voltage of each region; a cooperative voltage control problem between regions is converted into a Markov game so as to realize effective cooperation between intelligent agents; a centralized training and decentralized execution framework is adopted, so that the intelligent agent fully utilizes global information in a training stage and independently makes a decision according to local observation in an execution stage; a self-attention mechanism is integrated, so that the agents are more focused on information related to self rewards, a multi-agent reinforcement learning technology can be combined with the self-attention mechanism, the problem of voltage fluctuation in a power distribution network is effectively solved, stable control and optimal management of voltage are realized, and the power distribution network quality is improved. The voltage regulation precision and response speed of the power distribution network are improved, and the stability and reliability of a power system are ensured.

The invention discloses an energy storage cluster participation power grid peak regulation optimization method and system and a related device, and belongs to the technical field of energy storage. The method comprises the following steps: establishing an optimization model for an energy storage cluster to participate in power grid peak regulation; converting the optimization model of the energy storage cluster participating in the peak regulation of the power grid into a Markov game model; constructing a depth deterministic strategy gradient model; obtaining typical daily system load and energy storage cluster historical data of a certain region, training a depth deterministic strategy gradient model offline, solving a Markov game model based on the depth deterministic strategy gradient model until awards converge, and obtaining an energy storage cluster participation power grid peak regulation optimization strategy; and the energy storage cluster participates in the power grid peak regulation optimization strategy, and the charging and discharging behaviors of the energy storage cluster participating in the power grid peak regulation are optimized online. The method adaptively optimizes the charging and discharging strategy of each energy storage power station to realize peak load shifting, and the peak load regulation capability of the power grid is effectively improved.

The invention discloses an electric vehicle charging method and device based on multi-agent deep reinforcement learning, and the method comprises the steps: building a charging control model based on a Markov game framework and a real-time charging scene, and the charging control model comprises a state variable, an action variable, a reward function and a state transfer function; constructing a control strategy network based on the charging control model, and constructing a parameter updating model of the control strategy network; and constructing an evaluation network, and optimizing the control strategy network based on the evaluation network and the parameter updating model to obtain an optimal control strategy. A charging control model is constructed through a Markov game framework, so that multi-agent collaborative optimization of vehicle charging control is constructed, charging resources of each electric vehicle are dynamically allocated, it is ensured that under limited power grid resources, the charging requirements of all vehicles are met to the maximum extent, and the charging efficiency and economic benefits in different scenes are planned as a whole.

The invention discloses a multi-ship cooperative autonomous collision avoidance method based on PMA-MADDPG. The method comprises the following steps that ship agents and the environment where the ship agents are located are modeled into a multi-agent Markov game system; designing a reward function of the multi-agent Markov game system into a multi-target comprehensive reward function; establishing a PMA-MADDPG model based on a policy-value network structure, and introducing a long and short-term memory network and a priority experience playback mechanism into the model; the PMA-MADDPG model is trained, and after a specified training round is reached, the PMA-MADDPG model after training is completed is obtained; and inputting the current local observation information into the trained PMA-MADDPG model by each ship intelligent body, generating an optimal collision avoidance action at the next moment, and executing a collision avoidance task according to the optimal collision avoidance action. According to the method, through multi-agent collaborative decision and time sequence state perception, the ship collision avoidance safety and the navigation rule conformity in a complex maritime scene are remarkably improved.

The invention discloses a multi-agent collaborative optimization control method and system for regional energy interconnection, and belongs to the technical field of multi-agent collaborative optimization control. According to the method, through a constructed Markov game model, each single agent is regarded as a node in a GNN graph neural network graph; capturing a coupling relationship among a plurality of single agents through a GNN graph neural network; introducing a federated learning framework, training the PPO algorithm of each single agent, and obtaining PPO algorithm training model parameters of each single agent; and according to the coupling relationship between the single agents, aggregating and updating the obtained PPO algorithm training model parameters of the single agents, constructing a distributed PPO algorithm training model, updating the system parameters of the single agents of the energy interconnection system, and obtaining a multi-agent collaborative optimization control strategy. According to the method, cooperation among the intelligent agents of the multi-region energy interconnection system can be optimized, and efficient, stable and flexible system operation is achieved.

The invention discloses a distribution-microgrid coordination control method and system based on multi-agent deep reinforcement learning. The method comprises the following steps: firstly, constructing a distribution-microgrid coordination control optimization model; then, a multi-agent deep reinforcement learning framework is built, and the distribution-microgrid coordination control optimization model is converted into a Markov game; finally, an improved TD3 algorithm is used for solving the distribution-microgrid coordination control optimization model of the Markov game, an optimal distribution-microgrid coordination control strategy is generated, and the improved TD3 algorithm is based on updating rules of Taylor series optimization strategy network parameters. Coordination control optimization of the power distribution network and the micro-grid is converted into cooperation and competition of intelligent agents, the TD3 algorithm based on Taylor series optimization is used for generating the optimal power distribution-micro-grid coordination control strategy, learning efficiency and real-time performance are both considered, dynamic coordination of the power distribution network and the micro-grid is achieved, and the power distribution network and micro-grid coordination control method is suitable for large-scale popularization and application. The operation cost of the intelligent power distribution system and the multi-microgrid system is effectively reduced, and the stability of the whole system is improved.

The invention relates to a red-blue game confrontation-oriented constrained collaborative multi-agent reinforcement learning method, which comprises the following steps of: constructing a virtual game scene and a scenario confrontation blue target, and decomposing a game confrontation task into a target collaborative allocation problem and a maneuvering strategy generation problem; respectively establishing a Markov game process of a target collaborative allocation problem and a Markov game process of a maneuvering strategy generation problem; establishing a constraint collaborative multi-agent reinforcement learning model used for solving a Markov game process of a target collaborative allocation problem, and inputting game situation information to obtain a target allocation scheme; and establishing a constraint collaborative experience-first multi-agent reinforcement learning model of a Markov game process for solving a maneuvering strategy generation problem, and inputting game situation information and maneuvering state information to obtain an optimal path of an attack target. The method can improve the utilization rate of the training sample, reduce the training time, improve the sample efficiency, and enhance the environment learning ability of the intelligent agent.

The invention discloses an interference frequency domain resource scheduling method and device for multi-unmanned aerial vehicle communication. Firstly, a distributed frequency domain blocking type interference model for multi-unmanned aerial vehicle frequency hopping communication of multiple jammers based on an interference-to-signal ratio criterion is established; then taking a distributed frequency domain blocking type interference model as a starting point, taking a plurality of jammers as an agent group, constructing a process of interfering multi-unmanned aerial vehicle communication into a partially observable Markov game process, and constructing a multi-agent reliability distribution model based on a reliability distribution criterion; and finally, an improved anti-fact multi-agent strategy gradient algorithm and a finite-length elite trajectory experience playback mechanism are utilized to optimize a reliability allocation strategy in the process that the multiple jammers confront the multiple unmanned aerial vehicles, reasonable and effective interference frequency domain resource scheduling is realized, the reliability allocation efficiency is effectively improved, and the robustness of the system is improved. The problems of low reliability distribution credibility and poor distribution efficiency of a conventional multi-jammer confrontation multi-unmanned aerial vehicle frequency hopping communication system are solved.

The invention discloses a micro-grid optimization operation method based on an elastic step length depth Q network. The micro-grid optimization operation method comprises the following steps: constructing a multi-objective optimization model taking the minimization of total operation cost and environmental cost as an objective; a micro-grid multi-objective optimization problem is converted into a Markov game process; a hidden feature clustering driving mechanism and a neighbor-based variable density clustering algorithm are combined, an elastic step length adjusting mechanism is designed, and the step length of deep reinforcement learning is adjusted according to state similarity so as to relieve the problem of Q value overestimation; and optimizing the micro-grid intelligent agent by using an ES-DQN method. According to the method provided by the invention, the state similarity is effectively judged by combining an elastic step length mechanism and deep reinforcement learning and utilizing clustering analysis, so that the optimization efficiency is improved while the problem of Q value over-estimation in the traditional DQN is relieved, and the micro-grid optimization scheduling is more accurate.

The invention discloses a virtual power plant lightweight cloud edge collaborative optimization scheduling method and device, and relates to the field of virtual power plant optimization scheduling. The method comprises the following steps: dividing a virtual power plant cluster at a cloud according to the energy coupling strength of a virtual power plant; establishing a virtual power plant optimization scheduling model; modeling the virtual power plant optimization scheduling model as a Markov game model; multi-agent neural network parameters in the Markov game process are trained; configuring an edge computing unit for each cluster according to a cluster division result, and enabling each edge computing unit to receive optimal parameters of a Markov game model and a multi-agent neural network of the cloud; and each edge computing unit acquires state information of the respective cluster by using a self-adaptive data compression and acquisition method, and performs Markov game by using the received model and the optimal parameter to obtain a power scheduling result of each node in each cluster. According to the method, the virtual power plant scheduling decision accuracy under limited time and limited observation information is improved.

This invention discloses a collaborative game control method for swarm spacecraft based on a CTDE architecture, comprising the following steps: Step S1: Constructing a swarm-based relatively nonlinear dynamic model containing multiple tracking spacecraft; Step S2: Obtaining a configuration evaluation index for guiding a collaborative encirclement configuration; Step S3: Obtaining a Markov game environment framework; Step S4: Obtaining a dense reward signal for policy optimization within the Markov game environment framework; Step S5: Obtaining a convergent collaborative game control policy based on the CTDE architecture; Step S6: Training the collaborative game control policy using a phased learning mechanism. This invention achieves stable convergence in a multi-agent environment, provides dense reward signals to alleviate reward sparsity, forms a swarm encirclement configuration, smoothly integrates hard engineering constraints, and improves the robustness of the policy under high-dynamic adversarial conditions through phased learning.

The invention relates to the technical field of power systems, and particularly discloses a medium-low voltage collaborative optimal control method for an active power distribution network based on double-layer reinforcement learning. Historical data of distributed photovoltaic output, key node voltage, power flow reverse transmission rate of a transformer in a transformer area, tap state of a medium-low voltage transformer, port power of a medium-voltage intelligent soft switch and network loss are collected through a measuring device; training a two-stage neural network based on historical data; taking the two-stage neural network as a reinforcement learning off-line training environment, and constructing a double-layer Markov game model; and deploying the trained agent parameters to medium and low voltage control terminals to realize medium and low voltage multi-target cooperative control containing multiple types of adjusting devices. The method has the advantages that the double-layer reinforcement learning algorithm integrated with the attention mechanism is proposed to solve the multi-time scale equipment regulation and control problem, the sum of lower-layer agent rewards serves as a part of an upper-layer agent, and active power distribution network medium-low voltage collaborative optimal control is achieved.

The invention discloses a multi-agent deep reinforcement learning-based secondary frequency modulation method for a network-constructing converter, and belongs to the field of converter intelligent frequency modulation, and the method comprises the following steps: constructing a GFC dynamic model based on the control logic and physical model of the GFC; constructing a Markov game process based on the GFC dynamic model; constructing a multi-agent deep reinforcement learning framework based on the Markov game process; and training the agents based on the multi-agent deep reinforcement learning framework, and updating strategy parameters of the agents to obtain a GFC secondary frequency modulation strategy. According to the invention, autonomous learning and collaborative decision-making of the intelligent agent are realized, and the secondary frequency modulation performance of the GFC is remarkably improved.

The invention discloses a large-model-driven multi-subject trajectory generation and city-level behavior simulation method. According to the method, a multi-agent generation adversarial imitation learning framework is adopted, an MAGAIL-VL model is constructed, and synchronous simulation of vehicle movement and road section traffic state evolution is realized. An urban traffic system is abstracted into a Markov game, two types of agents including a vehicle agent and a road section agent are included, and expert demonstration is carried out by utilizing a real track and speed data. The model generator generates an action according to a local state, and the discriminator provides a reward signal by distinguishing generated data and expert data to guide strategy network updating. According to the framework, centralized training and distributed execution are adopted, the calculation complexity is reduced, meanwhile, the interaction influence between main bodies can be accurately described, and vivid urban-level traffic tracks and simulation scenes are generated.

The invention relates to a voltage reactive power control method and device, control equipment and a storage medium, and the method comprises the steps: obtaining the real-time state data of a power distribution network, the real-time state data comprises the reactive power output, node voltage and load information of a photovoltaic inverter in each sub-region divided in advance; inputting the real-time state data into a pre-trained multi-agent cooperative control model, and outputting a reactive power control instruction through the multi-agent cooperative control model, the reactive power control instruction being used for indicating the reactive power output of the photovoltaic inverter of each sub-region; wherein the multi-agent cooperative control model is obtained by performing set training on a Markov game model constructed by taking a sub-region as an agent, and the sub-region is obtained by dividing the power distribution network through node modularity determined by an electrical distance between nodes of the power distribution network. Therefore, the real-time performance and the control precision of voltage reactive power control can be improved.