Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

211 results about "Partially observable Markov decision process" patented technology

A partially observable Markov decision process (POMDP) is a generalization of a Markov decision process (MDP). A POMDP models an agent decision process in which it is assumed that the system dynamics are determined by an MDP, but the agent cannot directly observe the underlying state. Instead, it must maintain a probability distribution over the set of possible states, based on a set of observations and observation probabilities, and the underlying MDP.

Multi-agent unmanned aerial vehicle cluster collaborative search method based on semantic communication technology

The invention discloses a multi-agent unmanned aerial vehicle cluster collaborative search method based on a semantic communication technology. The method comprises the following steps: defining a partially observable Markov decision process of an unmanned aerial vehicle cluster multi-agent system according to a search task demand; initializing a semantic communication protocol of interaction between the unmanned aerial vehicle cluster and the environment; summarizing and abstracting semantic information of the unmanned aerial vehicle cluster; unmanned aerial vehicle cluster collaborative search task allocation and path planning are carried out by using semantic information; dynamic adjustment and real-time task optimization of the multi-agent system are realized in combination with semantic information; and evaluating a task completion effect. According to the method, the problem of incomplete search coverage of the unmanned aerial vehicle cluster in the complex environment is effectively solved, the search efficiency and accuracy of the unmanned aerial vehicle cluster in the complex environment can be effectively improved, the search efficiency and task completion quality of the unmanned aerial vehicle cluster are remarkably improved, and the intelligent level and robustness of the system are enhanced.
Owner:NANJING UNIV OF AERONAUTICS & ASTRONAUTICS

Complex product process route flexible planning system based on deep cycle Q network, planning method and application

The invention discloses a complex product process route flexible planning method based on a deep cycle Q network. The method is suitable for efficient process route optimization in a dynamic manufacturing environment. The method comprises the following steps: firstly, modeling process characteristics, processing operation, resources and constraint relationships of a part, and constructing a state space and reward mechanism based on a partial observable Markov decision process; and then constructing a long short-term memory network to extract time sequence characteristics in a processing state, and constructing a DRQN on the basis to realize an optimal decision of processing operation. In order to adapt to variable process targets and environments, a self-adaptive reward function adjustment strategy is introduced, and dynamic optimization is realized by adjusting weights such as processing quality, time and energy consumption; and meanwhile, a selective forgetting mechanism is constructed, and model parameters are suppressed and updated by utilizing a Fisher information matrix, so that efficient forgetting of outdated process knowledge is realized, and the migration ability and generalization performance of the model are improved.
Owner:SHENYANG AEROSPACE UNIVERSITY

Highly adaptive unmanned aerial vehicle cluster collaborative target search method based on deep reinforcement learning

The invention discloses a highly adaptive unmanned aerial vehicle cluster collaborative target search method based on deep reinforcement learning. According to the method, the problem of large-scale moving target search in a complex environment is solved through a revisit time compensation mechanism, wherein the problem is caused by target movement and the environmental cognitive information graph is unreliable. The method adopts a distributed partially observable Markov decision process to model layered collaborative search of an unmanned aerial vehicle group, and learns a joint strategy based on a deep reinforcement learning algorithm of a value decomposition network. Therefore, the unmanned aerial vehicle group can make full use of sensor performance characteristics at different heights, the risk of collision with adjacent unmanned aerial vehicles and obstacles is reduced, and the capability of cooperatively searching dynamic targets is improved. Simulation results show that compared with a baseline algorithm and an existing traditional algorithm, the algorithm has superiority and expandability.
Owner:NANJING UNIV OF AERONAUTICS & ASTRONAUTICS

Unmanned aerial vehicle trajectory planning method based on noise dual-depth Q network

The invention relates to the field of intelligent unmanned aerial vehicle autonomous control and decision making, in particular to an unmanned aerial vehicle trajectory planning method based on a noise dual-depth Q network, which comprises the following steps: acquiring unmanned aerial vehicle sensor data; establishing a forest fire model according to unmanned aerial vehicle sensor data; establishing an unmanned aerial vehicle constraint condition and an optimization function according to the forest fire model, and converting the unmanned aerial vehicle constraint condition and the optimization function into a partially observable Markov decision process; solving the partial observable Markov decision process by adopting a noise double-depth Q network to obtain an optimal unmanned aerial vehicle path planning strategy; according to the invention, the node areas are taken as decision objects, and value grading is carried out on different node areas, so that the decision is more in line with the reality; a noise bias item is introduced into a neural network weight to balance learning stability and exploration performance, so that the network can dynamically adapt to exploration behaviors in different environments; and a trajectory smoothness constraint is designed to ensure that the flight path is continuous, and the success rate of unmanned aerial vehicle trajectory planning is further improved.
Owner:CHONGQING UNIV OF POSTS & TELECOMM

Mobile charging station load situation awareness and intelligent scheduling method for electric vehicles

The invention relates to an electric vehicle-oriented mobile charging station load situation awareness and intelligent scheduling method, which comprises the following steps of: firstly, training historical load data of a fixed charging station, and acquiring a predicted value of a future load and uncertainty probability distribution of the predicted value by using a time sequence prediction model; and then, taking the prediction result as part of observation information, constructing a scheduling model based on a POMDP (Partial Observable Markov Decision Process), solving an optimal strategy of the POMDP, generating a scheduling decision, and sending a scheduling task to the mobile charging station to realize efficient distribution of charging resources.
Owner:NANJING UNIV OF POSTS & TELECOMM

Multi-agent cluster cooperative attack and defense game method based on near-end strategy optimization

The invention discloses a multi-agent cluster cooperative attack and defense game method based on near-end strategy optimization, and relates to the technical field of automatic control, and the method comprises the steps: constructing a partial considerable Markov decision process, and generating a multi-agent decision model; establishing a cooperative attack and defense game simulation framework including an attacker agent, a defender agent, a target area and irregular obstacles in the preset task area; parallel trajectory sampling is executed, an observation-action-reward sequence of each agent is collected, and a training data set is generated; a centralized state is constructed by cascading and splicing local observation of each agent, a time sequence difference error and a dominant function are calculated, network parameter updating is carried out by adopting cutting strategy gradient and value function regression, and a trained cooperative attack and defense game strategy network is generated when a reward curve converges. And the multi-agent cluster realizes intelligent cooperative attack and defense game and autonomous obstacle avoidance in a task environment with irregular obstacles.
Owner:BEIHANG UNIV

Microgrid group low-carbon optimization operation method based on improved MAPPO algorithm

The invention discloses a micro-grid group low-carbon optimization operation method based on an improved MAPPO algorithm, and the method comprises the steps: building a micro-grid group optimization operation model with the minimum total cost in a single micro-grid operation period as an objective function; constructing a partially observable Markov decision process of micro-grid optimization operation, and establishing an observation space, an action space and a reward function corresponding to each micro-grid agent; non-linear constraints of carbon treatment cost and collaborative rewards among micro-grids are introduced into a reward function, and a negative sampling experience sharing framework is introduced into an MAPPO algorithm; the LSTM network is embedded into the MAPPO algorithm, and a differential learning rate attenuation strategy is designed to further improve the training speed of the LSTM-MAPPO model; and training the intelligent agent based on an improved MAPPO algorithm to obtain an optimal microgrid group low-carbon optimization operation scheme. The optimization method provided by the invention can effectively reduce the operation cost of each micro-grid and reduce the carbon emission of the system at the same time.
Owner:CHINA THREE GORGES UNIV

Multi-Agent Reinforcement Learning Rolling Scheduling Method, Device, Equipment and Storage Medium

The present invention discloses a multi-agent reinforcement learning rolling scheduling method, device, equipment and storage medium. The method constructs a rolling scheduling model for the intraday active power corresponding to a high-proportion new energy power system; performs multi-agent decentralized partially observable Markov decision process modeling on the rolling scheduling model to obtain a multi-agent scheduling architecture; obtains an attention network for the improved region feature aggregation graph of the multi-agent scheduling architecture, and obtains a multi-agent reinforcement learning algorithm that supports spatio-temporal multi-dimensional feature aggregation. According to the attention network and the multi-agent reinforcement learning algorithm, a training architecture of a distributed intraday rolling scheduling algorithm based on multi-agent reinforcement learning is constructed, with fast modeling and solving speed and a simple training process, which conforms to the actual application scenario of power grid scheduling, improves the accuracy of multi-agent reinforcement learning rolling scheduling, and enhances the speed and efficiency of multi-agent reinforcement learning rolling scheduling.
Owner:WUHAN UNIV

Multi-objective optimization partially observable markov decision process system utilizing hierarchical adaptive probabilistic information manifestation mechanism

PendingJP2025109811ABiological modelsState spaceBelief state
To provide an efficient system that overcomes information incompleteness in the Partially Observable Markov Decision Processes (POMDP) and simultaneously optimizes multiple competing objective functions.SOLUTION: The present invention provides a multi-objective optimization system using a hierarchical adaptive manifestation mechanism in a POMDP. The system hierarchically clusters a state space to construct a hierarchical belief support MDP, and calculates a strategy that optimizes a weighted sum of a plurality of objective functions. Additionally, the system includes an adaptive manifestation mechanism that dynamically switches between exploration actions and goal-achieving actions according to uncertainty of belief states, and a continuous manifestation mechanism that corresponds to continuous state and action spaces. This enables improvement in computational efficiency and realization of multi-objective optimization.SELECTED DRAWING: None
Owner:NYU-YO-KU ZENERAL GURU-PU INKU

Harmonic suppression method and system for power distribution network

The invention discloses a power distribution network harmonic suppression method and system, and the method comprises the steps: building an objective function with the minimum total harmonic distortion rate of nodes of a whole network, building a power flow constraint, a harmonic distortion rate constraint and an active filter device related constraint, and obtaining a power distribution network harmonic suppression model according to the objective function and constraint conditions. A distributed partially observable Markov decision process is established based on the MADDPG algorithm, and harmonic suppression is performed on the power distribution network based on the distributed partially observable Markov decision process by using the PRE-combined MADDPG algorithm, so that the problem of low sample efficiency of a reinforcement learning algorithm is effectively solved, the stability of a training process and the convergence speed of the algorithm are promoted, and the robustness of the power distribution network is improved. According to the method, the effectiveness of the finally obtained harmonic suppression strategy is ensured, based on a centralized training-distributed execution architecture, it can be ensured that a plurality of active filter devices can make a cooperative control strategy giving consideration to the harmonic problem of the whole power distribution network in real time only by means of distributed observation, and therefore efficient whole network harmonic cooperative governance is achieved.
Owner:STATE GRID FUJIAN ELECTRIC POWER CO LTD +1

Multi-microgrid distributed control method and device based on multi-agent reinforcement learning

The invention relates to a multi-microgrid distributed control method and device based on multi-agent reinforcement learning. The method comprises the following steps: modeling a power network system, and modeling a multi-microgrid forming process into a distributed partially observable Markov decision process; deploying a multi-agent reinforcement learning algorithm according to a partial observable Markov decision process, performing environment interaction sampling information between each agent and the power network system, and storing the sampling information into respective experience playback pools; based on a DQN algorithm, training a strategy network corresponding to each agent to learn an optimal behavior by sampling experience samples in an experience replay pool according to an action mask technology and an experience screening technology; and utilizing the trained strategy network model to respectively control the switch action of each region of the power system to form a micro-grid, and recovering load power supply. By adopting the method, the toughness of the power system can be effectively improved, and rapid formation and recovery of the multi-microgrid can be realized.
Owner:NAT UNIV OF DEFENSE TECH

Edge computing collaborative task unloading method and system based on dynamic reputation value

The invention discloses an edge computing collaborative task unloading method and an edge computing collaborative task unloading system based on a dynamic reputation value, and the edge computing collaborative task unloading method is based on a task model for constructing collaborative task unloading. A task unloading decision optimization problem with task success rate maximization as a target is converted into a decentralized partially observable Markov decision process; according to the three-factor reputation evaluation method based on beta distribution, the reputation value of an edge node is efficiently and accurately evaluated, and the reputation value of the edge node is stored by adopting a block chain technology. And the reputation value is used for setting a reward obtained after the edge node makes a task unloading decision. Based on interaction of edge nodes and the environment in a decentralized partially observable Markov decision process, an experience fragment composed of observation, actions and rewards is constructed to train a near-end strategy optimization algorithm; and using the trained near-end strategy optimization algorithm to obtain an optimal task unloading strategy according to observation of the edge nodes.
Owner:HANGZHOU NORMAL UNIVERSITY

Planning trajectory generation method and device, electronic equipment and storage medium

The invention provides a planned trajectory generation method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining a state space and an action space of a vehicle; according to the potential collision risk and the action space, classifying the predicted trajectories of the side vehicle to obtain a risk trajectory set and a safety trajectory set; and through a partially observable Markov decision process, modeling is performed on a state space, an action space, a risk trajectory set and a safety trajectory set to obtain a reference trajectory. The decision planning process under the uncertainty of the side vehicle behavior is modeled as POMDP, so that the uncertain behavior of the side vehicle is fully predicted, and the driving safety of the vehicle is ensured. Besides, in consideration of too high dimensionality of a multi-traffic-participant prediction trajectory cluster in a complex scene and easy falling into dimensionality disaster of the POMDP model, the prediction trajectory of the side vehicle is classified into a risk trajectory set and a safe trajectory set through a potential collision risk, so that state space dimensionality reduction is realized, and decision planning efficiency and decision stability are effectively improved.
Owner:SAIC GM WULING AUTOMOBILE CO LTD

Unmanned aerial vehicle dynamic task allocation and path planning method and device for cluster cooperation

The invention relates to a cluster cooperation-oriented unmanned aerial vehicle dynamic task allocation and path planning method and device, and the method comprises the steps: modeling a monitoring region comprising an unmanned aerial vehicle cluster, a plurality of to-be-tracked targets and a specified moving target as a partially observable Markov decision process, and enabling each unmanned aerial vehicle to obtain local observation information based on relative parameters; then, each unmanned aerial vehicle outputs a flight orientation adjustment instruction through a strategy neural network, and the network is trained and optimized under the guidance of a value network through a multi-agent near-end strategy optimization algorithm; the unmanned aerial vehicle cluster executes a joint action and generates a comprehensive reward signal fusing global task rewards, individual behavior rewards and anti-collision punishment; and finally, driving the strategy network to synchronously complete implicit task allocation and collaborative path planning through continuous feedback of the reward signal. According to the invention, integrated cooperative solution of task allocation and path planning is realized, so that the unmanned aerial vehicle cluster has adaptive cooperative capability and high robustness under a distributed decision framework.
Owner:TAODIAN CHAIN (GUANGZHOU) INFORMATION TECH CO LTD

Intelligent wireless control method and system for cloud robot

The invention discloses an intelligent wireless control method and system for a cloud robot. The method comprises the following steps: constructing a reconfigurable intelligent metasurface assisted cloud robot wireless control system; based on each preset constraint condition, constructing an optimization model comprehensively considering control errors and communication jitter by taking maximization of long-term stability as a target; reconstructing the optimization model by adopting a preset part observable Markov decision process model to obtain a target optimization model; based on observation state information, collected in real time, of each cloud robot in the cloud robot wireless control system, a multi-agent migration reinforcement learning model based on near-end strategy optimization is adopted to optimize the target optimization model, and target parameter values corresponding to the to-be-optimized parameters are obtained, and performing intelligent control on each cloud robot in the cloud robot wireless control system based on the target parameter value. According to the method, the strategy migration efficiency and the system stability of wireless control of the cloud robot can be improved.
Owner:SHENYANG INST OF AUTOMATION - CHINESE ACAD OF SCI

Agricultural unmanned aerial vehicle heterogeneous cluster cooperative task allocation and path planning method

The invention relates to a heterogeneous cluster cooperative task allocation and path planning method for an agricultural unmanned aerial vehicle, and the method comprises the following steps: S1, carrying out the system modeling and initialization, and carrying out the modeling of a cooperative data collection task of an unmanned aerial vehicle mechanism cluster in an agricultural environment as a decentralized partially observable Markov decision process; s2, defining a multi-dimensional space, wherein the decentralized part observes a global state space, a joint action space and a local observation space of a Markov decision process; s3, designing a mixed reward function; s4, performing cluster strategy training; and S5, performing online distributed execution. The method effectively solves the problems that an existing method is poor in environment dynamic change adaptability, does not fully consider the heterogeneous characteristics of the unmanned aerial vehicles and does not fully consider decision-making and execution separation, and remarkably improves the operation efficiency, safety and collaborative intelligence level of an agricultural unmanned aerial vehicle cluster in a complex unstructured environment.
Owner:QINGDAO AGRI UNIV

Intelligent radar and jammer dynamic waveform gaming method based on curiosity driving

The invention provides an intelligent radar and jammer dynamic waveform gaming method based on curiosity driving, and belongs to the field of intelligent radar and intelligent jammer waveform gaming. The problems that the intelligent agent is difficult to effectively learn from limited environment feedback due to lack of a waveform-level dynamic game mechanism close to an actual electronic warfare and a sparse reward problem, and the ability imbalance of the intelligent agent is easy to occur in a training process are solved. The method comprises the following steps: constructing a confrontation environment model, namely modeling a dynamic game of a radar and a jammer into a partially observable Markov decision process, and defining a state space, an observation space, an action space and a reward function; designing a complex field feature extraction network; establishing a decision-evaluation network based on an MADDPG framework; a curiosity exploration strategy is introduced; implementing an opponent agent sampling strategy; and alternately training the radar and jammer agents. The device is mainly used in the electronic warfare field.
Owner:HARBIN INST OF TECH

Distributed online AI service collaborative awareness method for unmanned aerial vehicle cluster

The invention provides an unmanned aerial vehicle cluster distributed online AI service collaborative awareness method. The method comprises a data and model joint scheduling decision algorithm based on centralized offline training, a distributed online reasoning framework and deep reinforcement learning. According to the invention, a complex situation awareness task is divided into a target identification-target tracking-target positioning AI service chain; an AI service is cooperatively provided by using an unmanned aerial vehicle cluster; a partially observable Markov decision process is adopted for modeling environment-action-reward of the intelligent agent; and finally, training a decision model based on a deep reinforcement learning algorithm, and deploying the model to each unmanned aerial vehicle node for independent data scheduling and model dynamic activation decision. According to the method, the problem of dynamic coordination of chained AI service tasks (such as situation awareness tasks and micro-services) is effectively solved, higher flexibility and expansibility are achieved, the resource utilization rate is also remarkably improved, and meanwhile, stable operation of the system can still be kept when a node fault occurs.
Owner:UNIV OF ELECTRONICS SCI & TECH OF CHINA

Unmanned aerial vehicle moving edge calculation trajectory planning method based on multi-target reinforcement learning

The invention discloses an unmanned aerial vehicle moving edge calculation trajectory planning method based on multi-target reinforcement learning. Aiming at the problems of limited energy and uneven task distribution of unmanned aerial vehicles in a multi-unmanned aerial vehicle assisted mobile edge computing system, the method comprises the following steps: collecting real-time state data of the unmanned aerial vehicles, evaluating system target preferences (minimizing service interruption time or maximizing computing task load), constructing a distributed partially observable Markov decision process model, and calculating the target preferences of the unmanned aerial vehicles in the distributed partially observable Markov decision process model. And iteratively updating model parameters in combination with a crowding distance optimization experience playback mechanism so as to dynamically generate an optimal flight path. Experiments show that the method can adaptively balance and calculate the task load and the service continuity, effectively improves the system stability and the resource utilization rate, and is suitable for dynamic scenes such as disaster rescue and intelligent traffic.
Owner:WUHAN UNIV OF TECH

Caching, migration and unloading collaborative optimization method in satellite-assisted ocean network

The invention particularly relates to a cache, migration and unloading collaborative optimization method in a satellite-assisted ocean network, which comprises the following steps: constructing a satellite-assisted ocean network architecture, mapping task completion delay to system utility according to the satellite-assisted ocean network architecture, and constructing a target function with the purpose of maximizing the system utility under multi-dimensional constraints; converting an objective function optimization problem into a partially observable Markov decision process; and solving an objective function optimization problem based on attention-enhanced multi-agent reinforcement learning to obtain an optimal solution. According to the method, the problem of low-delay and efficient task calculation under the satellite-assisted ocean network architecture is solved.
Owner:NORTHWESTERN POLYTECHNICAL UNIV

Vulnerability intelligent collaborative detection method and system based on multi-agent reinforcement learning

The invention discloses a vulnerability intelligent collaborative detection method and system based on multi-agent reinforcement learning. The method comprises the following steps: independently executing detection according to local observation information and a strategy generation model through an intelligent agent deployed at an edge node; the strategy generation model is constructed based on a decentralized partial observable Markov decision process, and is obtained through cloud centralized training and global shared reward function optimization. When the agent obtains a forward reward, packaging a corresponding vulnerability context and related strategy parameters into an experience package; and the experience packet is diffused in the cooperative detection network in a mode of randomly selecting neighbor nodes and forwarding step by step. And after each agent receives the data, analyzing the content of the experience package and comparing the content with the local environment so as to dynamically adjust local strategy parameters and realize collaborative optimization. According to the vulnerability intelligent collaborative detection method and system based on multi-agent reinforcement learning provided by the invention, the problems that network security vulnerability detection is poor in expandability and lacks knowledge generalization ability are solved.
Owner:SHIZUISHAN POWER SUPPLY COMPANY OF STATE GRID NINGXIA ELECTRIC POWER +1

Main distribution coordinated voltage optimization control method and equipment under communication and electrical coupling

The present invention discloses a method and device for optimizing voltage control of main distribution coordination under communication and electrical coupling. The method is based on a model prediction rolling optimization framework, takes voltage deviation minimization as the goal, and constructs a distributed voltage control model under communication and electrical coupling; decomposes the distributed voltage control model into sub-problems of adjacent partition information interaction and multiple partition parallel control optimization and reconstructs it into a partially observable Markov decision process; uses a multi-agent reinforcement learning algorithm to quickly solve the reconstructed model and determine the voltage control method. The present invention can improve the voltage regulation accuracy and real-time performance, reduce the communication bandwidth requirements, and make the voltage under main distribution coordination run more safely and stably.
Owner:HEFEI UNIV OF TECH

Proximity information associated multi-unmanned aerial vehicle dynamic task allocation method in emergency scene

The invention discloses a multi-unmanned aerial vehicle dynamic task allocation method for adjacent information association in an emergency scene. The method comprises the following steps: constructing a multi-unmanned aerial vehicle system model in the emergency scene; constructing a dynamic task model; constructing a multi-unmanned aerial vehicle dynamic task allocation problem into a maximized multi-unmanned aerial vehicle task coverage optimization problem, and providing a maximized task coverage objective function; the maximized unmanned aerial vehicle task coverage is expressed as a partially observable Markov decision process; a multi-unmanned aerial vehicle depth deterministic strategy gradient algorithm based on adjacent information association is utilized, a system state is formed through information exchange between different unmanned aerial vehicles, and the maximum coverage of a task point in an emergency scene within limited time is found out. According to the multi-unmanned aerial vehicle depth deterministic strategy gradient collaboration method based on adjacent information association, the efficiency of multi-unmanned aerial vehicle collaboration under a dynamic task is improved, and the requirement of the multi-unmanned aerial vehicle for maximizing task coverage within limited time in an emergency scene is met.
Owner:HOHAI UNIV

Distributed energy storage aggregation scheduling method and system considering user fatigue effect

The invention discloses a distributed energy storage aggregation scheduling method and system considering a user fatigue effect, and the method comprises the steps: building a user response behavior model considering the long-term response preference and short-term fatigue of a user based on a partial observable Markov decision process; establishing a user response state online inference strategy based on maximum likelihood estimation, and inferring an unobservable state of a user; and based on the user response behavior model and the user response state online inference strategy, establishing an online learning aggregation scheduling method combined with the running state constraint of the distributed energy storage unit, and realizing distributed energy storage aggregation scheduling. According to the method, the user response uncertainty, the fatigue level and the actual response capacity of the energy storage equipment are comprehensively considered, a reliable model basis is provided for aggregation scheduling, effective tracking of the target power is achieved, and therefore the accuracy and reliability of aggregation scheduling are improved.
Owner:NANJING NORMAL UNIVERSITY

Graph reinforcement learning reactive voltage control method considering domain knowledge

The invention discloses a graph reinforcement learning reactive voltage control method considering domain knowledge, and the method comprises the steps: (I) enabling a reactive voltage control problem to be built into a partially observable Markov decision process, and enabling an intelligent agent corresponding to each inverter to only observe the local state quantity of a limited number of nodes around; (2) constructing a space-time diagram in which domain knowledge is embedded; (3) utilizing a graph attention neural network to carry out domain knowledge-based feature reinforcement learning of the multi-agent; and (IV) performing multi-inverter action decision making based on a GAMARL method and the like. According to the invention, by integrating domain knowledge and an advanced graph reinforcement learning technology, the voltage regulation capability of the novel power distribution system under the condition of high-proportion distributed photovoltaic access is remarkably improved, and the intelligent and sustainable development of a power distribution network is promoted.
Owner:TIANJIN UNIV

Communication lightweight multi-micro energy network cooperative control method for large model assisted reward recovery

The invention belongs to the technical field of multi-micro-energy-network cooperative control, and particularly relates to a communication lightweight multi-micro-energy-network cooperative control method for reward recovery assisted by a large model, and the method comprises the following steps: S1, constructing a mathematical model of distributed energy in a micro-energy network, and enabling the mathematical model to be used for describing the dynamic behaviors of energy conversion, storage and constraint; s2, modeling a micro energy network cooperative control problem as a partially observable Markov decision process POMDP; s3, solving the POMDP of the step S2 by using an optimized MAPPO method to obtain a cooperative control scheme; and issuing to each micro-energy network terminal for execution. The method can break through the limitation of the communication bottleneck on the premise of ensuring the learning quality of the control strategy.
Owner:CHONGQING UNIV

Distributed heterogeneous workshop scheduling method and equipment based on greedy heuristic enhancement

The invention belongs to the related technical field of distributed workshop dynamic scheduling, and discloses a distributed heterogeneous workshop scheduling method and device based on greedy heuristic enhancement, and the method comprises the steps: inputting the state characteristics of a to-be-scheduled distributed heterogeneous workshop into a double-layer end-to-end strategy network; the double-layer end-to-end strategy network outputs an optimal workpiece to be processed, and then a final scheduling scheme is obtained; wherein the training environment of the double-layer end-to-end policy network is constructed based on a partial observable Markov decision process scheduling model; the state features are obtained based on a partial observable Markov decision process scheduling model; the partial observable Markov decision process scheduling model is obtained by modeling a dynamic distributed heterogeneous workshop scheduling problem by adopting a decentralized multi-agent architecture. According to the invention, the expansibility and the flexibility are improved.
Owner:HUAZHONG UNIV OF SCI & TECH

Micro-grid distributed robust optimization scheduling method considering source charge uncertainty and electricity-hydrogen-carbon coupling

The invention belongs to the technical field of power system operation and control, and particularly provides a microgrid distributed robust optimization scheduling method considering source load uncertainty and electricity-hydrogen-carbon coupling. Comprising the following steps: constructing an electricity-hydrogen-carbon-heat four-dimensional collaborative micro-grid energy flow framework comprising a multi-port alternating current and direct current bus, a hydrogen production storage and transportation subsystem, a carbon capture methanation subsystem and a cross-stage thermodynamic energy cascade network; establishing an IGDT-based source-load bilateral uncertainty dynamic pricing model, and selecting a risk avoidance or opportunity seeking pricing model as required; a multi-agent game double-layer interaction model with a micro-grid operator as a leader and a load aggregator and a charging station as followers is constructed, and a nested part can observe a Markov decision process; and a privacy protection multi-agent deep reinforcement learning algorithm in which a maximum entropy framework and a CTDE mechanism are introduced is adopted for solving. According to the method, the problem of low-carbon operation of the micro-grid in a source load uncertainty environment is solved, and collaborative improvement of economical efficiency, low-carbon performance and robustness is realized.
Owner:CHINA THREE GORGES UNIV

MARL-based flexible distributed energy network multi-agent collaborative optimization method

The invention provides a flexible distributed energy network multi-agent collaborative optimization method based on MARL, and the method comprises the steps: building a distributed energy network multi-agent interaction framework which comprises a local energy transaction model for distinguishing an intelligent soft switching path, a network cost model based on a power transmission distribution factor, and a power distribution network operator and user income model; a physical system operation constraint model is established and covers intelligent soft switch operation characteristics, network topology radial constraint, energy storage charging and discharging dynamic states, controllable distributed power supply and photovoltaic output limitation, translational load scheduling and DistFlow power flow constraint. And finally, constructing a collaborative optimization mechanism based on a multi-agent near-end strategy optimization algorithm, modeling the system into a partially observable Markov decision process, adopting a centralized training distributed execution framework, and guaranteeing transaction security and data privacy through block chain evidence storage, RSA encryption and an intelligent contract. The system economy and the operation flexibility are improved.
Owner:INST OF ELECTRICAL ENG CHINESE ACAD OF SCI

Electric vehicle charging scheduling method based on average field multi-agent reinforcement learning

The invention discloses an electric vehicle charging scheduling method based on mean field multi-agent reinforcement learning. Charging scheduling is carried out on a mobile charging station and an electric vehicle in a large-scale mobile charging scene at the same time. A charging scheduling problem is modeled as a partially observable Markov decision process, a mobile charging station and an electric vehicle serve as two different intelligent agents, a reinforcement learning scheduling network is trained respectively to recommend a scheduling position, and meanwhile, the average field theory is utilized to simplify the mutual influence between the intelligent agents; the trained scheduling network is distributed to the mobile charging station and the electric vehicle, so that the mobile charging station and the electric vehicle can quickly make a scheduling decision only by obtaining local observation, and complete distributed charging scheduling is realized. According to the scheduling method, the dynamically changing charging demand is met, the charging income of the mobile charging station is obviously increased, and the successful charging proportion of the electric vehicle is effectively improved.
Owner:NANJING UNIV OF POSTS & TELECOMM