Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

253 results about "Partially observable Markov decision process" patented technology

A partially observable Markov decision process (POMDP) is a generalization of a Markov decision process (MDP). A POMDP models an agent decision process in which it is assumed that the system dynamics are determined by an MDP, but the agent cannot directly observe the underlying state. Instead, it must maintain a probability distribution over the set of possible states, based on a set of observations and observation probabilities, and the underlying MDP.

Agent policy learning method with privacy protection in mobile edge computing

An agent policy learning method with privacy protection in mobile edge computing, which belongs to the technical field of mobile communications. The method comprises: establishing an edge collaborative computing unloading model which comprises a service caching model, a task unloading model and a system cost model; on the basis of the edge collaborative computing unloading model, using the minimization of the processing cost of a task as a target to establish an optimization problem of joint task unloading, service caching, computing power resource allocation and transmission power control; abstracting an optimal problem into a partially observable Markov decision process; and on the basis of the Markov decision process, using a multi-agent deep reinforcement learning algorithm based on a federated learning method to autonomously learn a task unloading, service caching, computing power resource allocation and transmission power control policy. In the present invention, an edge collaborative computing unloading model is established, such that an agent network protects privacy security of data and sensitive information of users by means of training based on federated learning.
Owner:CHONGQING UNIV OF POSTS & TELECOMM

Multi-agent unmanned aerial vehicle cluster collaborative search method based on semantic communication technology

The invention discloses a multi-agent unmanned aerial vehicle cluster collaborative search method based on a semantic communication technology. The method comprises the following steps: defining a partially observable Markov decision process of an unmanned aerial vehicle cluster multi-agent system according to a search task demand; initializing a semantic communication protocol of interaction between the unmanned aerial vehicle cluster and the environment; summarizing and abstracting semantic information of the unmanned aerial vehicle cluster; unmanned aerial vehicle cluster collaborative search task allocation and path planning are carried out by using semantic information; dynamic adjustment and real-time task optimization of the multi-agent system are realized in combination with semantic information; and evaluating a task completion effect. According to the method, the problem of incomplete search coverage of the unmanned aerial vehicle cluster in the complex environment is effectively solved, the search efficiency and accuracy of the unmanned aerial vehicle cluster in the complex environment can be effectively improved, the search efficiency and task completion quality of the unmanned aerial vehicle cluster are remarkably improved, and the intelligent level and robustness of the system are enhanced.
Owner:NANJING UNIV OF AERONAUTICS & ASTRONAUTICS

Voltage out-of-limit control system and method based on multi-agent deep reinforcement learning

The invention provides a voltage out-of-limit control system based on multi-agent deep reinforcement learning, and the system comprises a power grid region division module which is used for dividing a power grid into a plurality of regions, and each region at least comprises one piece of photovoltaic equipment; the photovoltaic equipment control strategy model building module is used for carrying out modeling by adopting a partially observable Markov decision process aiming at photovoltaic equipment in each region of the power grid to obtain a photovoltaic equipment control strategy model; and the photovoltaic equipment control strategy output module is used for inputting the current operation parameters of the power grid in each region into the photovoltaic equipment control strategy model, and solving the photovoltaic equipment control strategy model by using a double-delay depth deterministic strategy gradient algorithm. And obtaining reactive power of each photovoltaic inverter and outputting the reactive power to a power grid of a corresponding area to control the voltage of each node to be within a preset range. According to the method, the problems of voltage fluctuation and out-of-limit caused by distributed photovoltaic access to the power grid are effectively solved, and the control efficiency and the control effect are improved.
Owner:STATE GRID JIBEI ELECTRIC POWER COMPANY +1

Complex product process route flexible planning system based on deep cycle Q network, planning method and application

The invention discloses a complex product process route flexible planning method based on a deep cycle Q network. The method is suitable for efficient process route optimization in a dynamic manufacturing environment. The method comprises the following steps: firstly, modeling process characteristics, processing operation, resources and constraint relationships of a part, and constructing a state space and reward mechanism based on a partial observable Markov decision process; and then constructing a long short-term memory network to extract time sequence characteristics in a processing state, and constructing a DRQN on the basis to realize an optimal decision of processing operation. In order to adapt to variable process targets and environments, a self-adaptive reward function adjustment strategy is introduced, and dynamic optimization is realized by adjusting weights such as processing quality, time and energy consumption; and meanwhile, a selective forgetting mechanism is constructed, and model parameters are suppressed and updated by utilizing a Fisher information matrix, so that efficient forgetting of outdated process knowledge is realized, and the migration ability and generalization performance of the model are improved.
Owner:SHENYANG AEROSPACE UNIVERSITY

Highly adaptive unmanned aerial vehicle cluster collaborative target search method based on deep reinforcement learning

The invention discloses a highly adaptive unmanned aerial vehicle cluster collaborative target search method based on deep reinforcement learning. According to the method, the problem of large-scale moving target search in a complex environment is solved through a revisit time compensation mechanism, wherein the problem is caused by target movement and the environmental cognitive information graph is unreliable. The method adopts a distributed partially observable Markov decision process to model layered collaborative search of an unmanned aerial vehicle group, and learns a joint strategy based on a deep reinforcement learning algorithm of a value decomposition network. Therefore, the unmanned aerial vehicle group can make full use of sensor performance characteristics at different heights, the risk of collision with adjacent unmanned aerial vehicles and obstacles is reduced, and the capability of cooperatively searching dynamic targets is improved. Simulation results show that compared with a baseline algorithm and an existing traditional algorithm, the algorithm has superiority and expandability.
Owner:NANJING UNIV OF AERONAUTICS & ASTRONAUTICS

Unmanned aerial vehicle trajectory planning method based on noise dual-depth Q network

The invention relates to the field of intelligent unmanned aerial vehicle autonomous control and decision making, in particular to an unmanned aerial vehicle trajectory planning method based on a noise dual-depth Q network, which comprises the following steps: acquiring unmanned aerial vehicle sensor data; establishing a forest fire model according to unmanned aerial vehicle sensor data; establishing an unmanned aerial vehicle constraint condition and an optimization function according to the forest fire model, and converting the unmanned aerial vehicle constraint condition and the optimization function into a partially observable Markov decision process; solving the partial observable Markov decision process by adopting a noise double-depth Q network to obtain an optimal unmanned aerial vehicle path planning strategy; according to the invention, the node areas are taken as decision objects, and value grading is carried out on different node areas, so that the decision is more in line with the reality; a noise bias item is introduced into a neural network weight to balance learning stability and exploration performance, so that the network can dynamically adapt to exploration behaviors in different environments; and a trajectory smoothness constraint is designed to ensure that the flight path is continuous, and the success rate of unmanned aerial vehicle trajectory planning is further improved.
Owner:CHONGQING UNIV OF POSTS & TELECOMM

Mobile charging station load situation awareness and intelligent scheduling method for electric vehicles

The invention relates to an electric vehicle-oriented mobile charging station load situation awareness and intelligent scheduling method, which comprises the following steps of: firstly, training historical load data of a fixed charging station, and acquiring a predicted value of a future load and uncertainty probability distribution of the predicted value by using a time sequence prediction model; and then, taking the prediction result as part of observation information, constructing a scheduling model based on a POMDP (Partial Observable Markov Decision Process), solving an optimal strategy of the POMDP, generating a scheduling decision, and sending a scheduling task to the mobile charging station to realize efficient distribution of charging resources.
Owner:NANJING UNIV OF POSTS & TELECOMM

Multi-agent cluster cooperative attack and defense game method based on near-end strategy optimization

The invention discloses a multi-agent cluster cooperative attack and defense game method based on near-end strategy optimization, and relates to the technical field of automatic control, and the method comprises the steps: constructing a partial considerable Markov decision process, and generating a multi-agent decision model; establishing a cooperative attack and defense game simulation framework including an attacker agent, a defender agent, a target area and irregular obstacles in the preset task area; parallel trajectory sampling is executed, an observation-action-reward sequence of each agent is collected, and a training data set is generated; a centralized state is constructed by cascading and splicing local observation of each agent, a time sequence difference error and a dominant function are calculated, network parameter updating is carried out by adopting cutting strategy gradient and value function regression, and a trained cooperative attack and defense game strategy network is generated when a reward curve converges. And the multi-agent cluster realizes intelligent cooperative attack and defense game and autonomous obstacle avoidance in a task environment with irregular obstacles.
Owner:BEIHANG UNIV

Mobile robot path planning method based on improved long short-term memory network

The invention discloses a mobile robot path planning method based on an improved long short-term memory network. The method comprises the following steps: step 1, carrying out partial observable Markov decision process modeling on a mobile robot in a time state; 2, adding a long-short-term memory network optimization reinforcement learning path planning algorithm and a preliminary optimization algorithm to complete basic navigation path planning, and 3, providing probability compensation for the long-short-term memory network; 4, applying a self-adaptive compensation function aiming at the strategy function so that the model can more effectively utilize the existing information and improve the decision and navigation capability of the robot in a complex environment in a changeable environment, and 5, training the modified SAC algorithm model, and deploying the finally trained model to the mobile robot, so as to improve the navigation capability of the robot in a complex environment. And path planning can be better completed. According to the method, the long-short-term memory network is introduced into the reinforcement learning algorithm, so that the performance of path planning is effectively improved.
Owner:ZHEJIANG RUNCHEN TECH CO LTD

Microgrid group low-carbon optimization operation method based on improved MAPPO algorithm

The invention discloses a micro-grid group low-carbon optimization operation method based on an improved MAPPO algorithm, and the method comprises the steps: building a micro-grid group optimization operation model with the minimum total cost in a single micro-grid operation period as an objective function; constructing a partially observable Markov decision process of micro-grid optimization operation, and establishing an observation space, an action space and a reward function corresponding to each micro-grid agent; non-linear constraints of carbon treatment cost and collaborative rewards among micro-grids are introduced into a reward function, and a negative sampling experience sharing framework is introduced into an MAPPO algorithm; the LSTM network is embedded into the MAPPO algorithm, and a differential learning rate attenuation strategy is designed to further improve the training speed of the LSTM-MAPPO model; and training the intelligent agent based on an improved MAPPO algorithm to obtain an optimal microgrid group low-carbon optimization operation scheme. The optimization method provided by the invention can effectively reduce the operation cost of each micro-grid and reduce the carbon emission of the system at the same time.
Owner:CHINA THREE GORGES UNIV

Multi-unmanned aerial vehicle cooperative confrontation decision-making method and system

The invention relates to a multi-unmanned aerial vehicle cooperative confrontation decision-making method and system. The method comprises the following steps: constructing a multi-unmanned aerial vehicle cooperative air combat confrontation decision-making environment by establishing a multi-unmanned aerial vehicle air combat confrontation motion model and an air combat situation evaluation model; establishing a distributed partially observable Markov decision process model of the multi-unmanned aerial vehicle cooperative confrontation decision problem according to the action space and the local observation and state of each unmanned aerial vehicle in the confrontation decision environment; designing a multi-machine collaborative adversarial reward function and an HASAC algorithm network space; and training and generating a multi-unmanned-aerial-vehicle cooperative confrontation strategy model based on interaction between a network space and a multi-unmanned-aerial-vehicle cooperative confrontation decision-making environment of an HASAC algorithm. According to the method, for the multi-aircraft cooperative air combat confrontation problem, the specific global state is designed for the multi-aircraft cooperative confrontation decision-making unmanned aerial vehicle, and compared with direct splicing of observation vectors of all unmanned aerial vehicles, the global state dimension is reduced, and the training efficiency is improved.
Owner:BEIHANG UNIV

Multi-Agent Reinforcement Learning Rolling Scheduling Method, Device, Equipment and Storage Medium

The present invention discloses a multi-agent reinforcement learning rolling scheduling method, device, equipment and storage medium. The method constructs a rolling scheduling model for the intraday active power corresponding to a high-proportion new energy power system; performs multi-agent decentralized partially observable Markov decision process modeling on the rolling scheduling model to obtain a multi-agent scheduling architecture; obtains an attention network for the improved region feature aggregation graph of the multi-agent scheduling architecture, and obtains a multi-agent reinforcement learning algorithm that supports spatio-temporal multi-dimensional feature aggregation. According to the attention network and the multi-agent reinforcement learning algorithm, a training architecture of a distributed intraday rolling scheduling algorithm based on multi-agent reinforcement learning is constructed, with fast modeling and solving speed and a simple training process, which conforms to the actual application scenario of power grid scheduling, improves the accuracy of multi-agent reinforcement learning rolling scheduling, and enhances the speed and efficiency of multi-agent reinforcement learning rolling scheduling.
Owner:WUHAN UNIV

Multi-objective optimization partially observable markov decision process system utilizing hierarchical adaptive probabilistic information manifestation mechanism

PendingJP2025109811ABiological modelsState spaceBelief state
To provide an efficient system that overcomes information incompleteness in the Partially Observable Markov Decision Processes (POMDP) and simultaneously optimizes multiple competing objective functions.SOLUTION: The present invention provides a multi-objective optimization system using a hierarchical adaptive manifestation mechanism in a POMDP. The system hierarchically clusters a state space to construct a hierarchical belief support MDP, and calculates a strategy that optimizes a weighted sum of a plurality of objective functions. Additionally, the system includes an adaptive manifestation mechanism that dynamically switches between exploration actions and goal-achieving actions according to uncertainty of belief states, and a continuous manifestation mechanism that corresponds to continuous state and action spaces. This enables improvement in computational efficiency and realization of multi-objective optimization.SELECTED DRAWING: None
Owner:NYU-YO-KU ZENERAL GURU-PU INKU

Harmonic suppression method and system for power distribution network

The invention discloses a power distribution network harmonic suppression method and system, and the method comprises the steps: building an objective function with the minimum total harmonic distortion rate of nodes of a whole network, building a power flow constraint, a harmonic distortion rate constraint and an active filter device related constraint, and obtaining a power distribution network harmonic suppression model according to the objective function and constraint conditions. A distributed partially observable Markov decision process is established based on the MADDPG algorithm, and harmonic suppression is performed on the power distribution network based on the distributed partially observable Markov decision process by using the PRE-combined MADDPG algorithm, so that the problem of low sample efficiency of a reinforcement learning algorithm is effectively solved, the stability of a training process and the convergence speed of the algorithm are promoted, and the robustness of the power distribution network is improved. According to the method, the effectiveness of the finally obtained harmonic suppression strategy is ensured, based on a centralized training-distributed execution architecture, it can be ensured that a plurality of active filter devices can make a cooperative control strategy giving consideration to the harmonic problem of the whole power distribution network in real time only by means of distributed observation, and therefore efficient whole network harmonic cooperative governance is achieved.
Owner:STATE GRID FUJIAN ELECTRIC POWER CO LTD +1

Multi-microgrid distributed control method and device based on multi-agent reinforcement learning

The invention relates to a multi-microgrid distributed control method and device based on multi-agent reinforcement learning. The method comprises the following steps: modeling a power network system, and modeling a multi-microgrid forming process into a distributed partially observable Markov decision process; deploying a multi-agent reinforcement learning algorithm according to a partial observable Markov decision process, performing environment interaction sampling information between each agent and the power network system, and storing the sampling information into respective experience playback pools; based on a DQN algorithm, training a strategy network corresponding to each agent to learn an optimal behavior by sampling experience samples in an experience replay pool according to an action mask technology and an experience screening technology; and utilizing the trained strategy network model to respectively control the switch action of each region of the power system to form a micro-grid, and recovering load power supply. By adopting the method, the toughness of the power system can be effectively improved, and rapid formation and recovery of the multi-microgrid can be realized.
Owner:NAT UNIV OF DEFENSE TECH

Unmanned aerial vehicle network intelligent trusted routing method based on blockchain enabling

The invention discloses an unmanned aerial vehicle network intelligent trusted routing method based on blockchain enabling, which aims at minimizing routing total end-to-end time delay, constructs a trusted network environment through a trust management mechanism based on a blockchain, and emphasizes a trusted routing problem as a distributed partially observable Markov decision process. And a multi-agent deep reinforcement learning algorithm is used for solving, and multi-dimensional observable information of the nodes is comprehensively considered. The method does not depend on global information which is difficult to obtain, can adapt to a dynamic environment, is low in calculation cost, can obtain a trusted routing path in a short time, and effectively solves the trusted routing problem that malicious nodes exist in an unmanned aerial vehicle network due to deliberate attacks and the like.
Owner:NANJING UNIV OF AERONAUTICS & ASTRONAUTICS

Edge computing collaborative task unloading method and system based on dynamic reputation value

The invention discloses an edge computing collaborative task unloading method and an edge computing collaborative task unloading system based on a dynamic reputation value, and the edge computing collaborative task unloading method is based on a task model for constructing collaborative task unloading. A task unloading decision optimization problem with task success rate maximization as a target is converted into a decentralized partially observable Markov decision process; according to the three-factor reputation evaluation method based on beta distribution, the reputation value of an edge node is efficiently and accurately evaluated, and the reputation value of the edge node is stored by adopting a block chain technology. And the reputation value is used for setting a reward obtained after the edge node makes a task unloading decision. Based on interaction of edge nodes and the environment in a decentralized partially observable Markov decision process, an experience fragment composed of observation, actions and rewards is constructed to train a near-end strategy optimization algorithm; and using the trained near-end strategy optimization algorithm to obtain an optimal task unloading strategy according to observation of the edge nodes.
Owner:HANGZHOU NORMAL UNIVERSITY

Electricity-carbon transaction method based on multi-agent reinforcement learning

The invention relates to the technical field of power systems, and discloses an electricity-carbon transaction method based on multi-agent reinforcement learning, and the method comprises the following steps: constructing a deep coupling P2P transaction system architecture containing electricity and carbon transaction participants, and determining the transaction variety of a carbon electricity market; providing a network fee strategy for the power distribution company, and allowing the clearing electricity price of a buyer corresponding to the power distribution company to be discounted adaptively according to the transaction volume; a decomposable reward including the overall transaction fairness of the electricity distribution and sale market and the transaction competitiveness of each market participant is constructed; and a partially observable Markov decision-making process oriented to the deep coupling transaction is provided, and a multi-agent deep deterministic strategy gradient algorithm is adopted to optimize the deep coupling property transaction of the distribution level. The method has the advantage that the competitiveness of a power distribution company can be improved.
Owner:SICHUAN UNIV

Planning trajectory generation method and device, electronic equipment and storage medium

The invention provides a planned trajectory generation method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining a state space and an action space of a vehicle; according to the potential collision risk and the action space, classifying the predicted trajectories of the side vehicle to obtain a risk trajectory set and a safety trajectory set; and through a partially observable Markov decision process, modeling is performed on a state space, an action space, a risk trajectory set and a safety trajectory set to obtain a reference trajectory. The decision planning process under the uncertainty of the side vehicle behavior is modeled as POMDP, so that the uncertain behavior of the side vehicle is fully predicted, and the driving safety of the vehicle is ensured. Besides, in consideration of too high dimensionality of a multi-traffic-participant prediction trajectory cluster in a complex scene and easy falling into dimensionality disaster of the POMDP model, the prediction trajectory of the side vehicle is classified into a risk trajectory set and a safe trajectory set through a potential collision risk, so that state space dimensionality reduction is realized, and decision planning efficiency and decision stability are effectively improved.
Owner:SAIC GM WULING AUTOMOBILE CO LTD

Unmanned aerial vehicle dynamic task allocation and path planning method and device for cluster cooperation

The invention relates to a cluster cooperation-oriented unmanned aerial vehicle dynamic task allocation and path planning method and device, and the method comprises the steps: modeling a monitoring region comprising an unmanned aerial vehicle cluster, a plurality of to-be-tracked targets and a specified moving target as a partially observable Markov decision process, and enabling each unmanned aerial vehicle to obtain local observation information based on relative parameters; then, each unmanned aerial vehicle outputs a flight orientation adjustment instruction through a strategy neural network, and the network is trained and optimized under the guidance of a value network through a multi-agent near-end strategy optimization algorithm; the unmanned aerial vehicle cluster executes a joint action and generates a comprehensive reward signal fusing global task rewards, individual behavior rewards and anti-collision punishment; and finally, driving the strategy network to synchronously complete implicit task allocation and collaborative path planning through continuous feedback of the reward signal. According to the invention, integrated cooperative solution of task allocation and path planning is realized, so that the unmanned aerial vehicle cluster has adaptive cooperative capability and high robustness under a distributed decision framework.
Owner:TAODIAN CHAIN (GUANGZHOU) INFORMATION TECH CO LTD

Intelligent wireless control method and system for cloud robot

The invention discloses an intelligent wireless control method and system for a cloud robot. The method comprises the following steps: constructing a reconfigurable intelligent metasurface assisted cloud robot wireless control system; based on each preset constraint condition, constructing an optimization model comprehensively considering control errors and communication jitter by taking maximization of long-term stability as a target; reconstructing the optimization model by adopting a preset part observable Markov decision process model to obtain a target optimization model; based on observation state information, collected in real time, of each cloud robot in the cloud robot wireless control system, a multi-agent migration reinforcement learning model based on near-end strategy optimization is adopted to optimize the target optimization model, and target parameter values corresponding to the to-be-optimized parameters are obtained, and performing intelligent control on each cloud robot in the cloud robot wireless control system based on the target parameter value. According to the method, the strategy migration efficiency and the system stability of wireless control of the cloud robot can be improved.
Owner:SHENYANG INST OF AUTOMATION - CHINESE ACAD OF SCI

Agricultural unmanned aerial vehicle heterogeneous cluster cooperative task allocation and path planning method

The invention relates to a heterogeneous cluster cooperative task allocation and path planning method for an agricultural unmanned aerial vehicle, and the method comprises the following steps: S1, carrying out the system modeling and initialization, and carrying out the modeling of a cooperative data collection task of an unmanned aerial vehicle mechanism cluster in an agricultural environment as a decentralized partially observable Markov decision process; s2, defining a multi-dimensional space, wherein the decentralized part observes a global state space, a joint action space and a local observation space of a Markov decision process; s3, designing a mixed reward function; s4, performing cluster strategy training; and S5, performing online distributed execution. The method effectively solves the problems that an existing method is poor in environment dynamic change adaptability, does not fully consider the heterogeneous characteristics of the unmanned aerial vehicles and does not fully consider decision-making and execution separation, and remarkably improves the operation efficiency, safety and collaborative intelligence level of an agricultural unmanned aerial vehicle cluster in a complex unstructured environment.
Owner:QINGDAO AGRI UNIV

Intelligent radar and jammer dynamic waveform gaming method based on curiosity driving

The invention provides an intelligent radar and jammer dynamic waveform gaming method based on curiosity driving, and belongs to the field of intelligent radar and intelligent jammer waveform gaming. The problems that the intelligent agent is difficult to effectively learn from limited environment feedback due to lack of a waveform-level dynamic game mechanism close to an actual electronic warfare and a sparse reward problem, and the ability imbalance of the intelligent agent is easy to occur in a training process are solved. The method comprises the following steps: constructing a confrontation environment model, namely modeling a dynamic game of a radar and a jammer into a partially observable Markov decision process, and defining a state space, an observation space, an action space and a reward function; designing a complex field feature extraction network; establishing a decision-evaluation network based on an MADDPG framework; a curiosity exploration strategy is introduced; implementing an opponent agent sampling strategy; and alternately training the radar and jammer agents. The device is mainly used in the electronic warfare field.
Owner:HARBIN INST OF TECH

Distributed online AI service collaborative awareness method for unmanned aerial vehicle cluster

The invention provides an unmanned aerial vehicle cluster distributed online AI service collaborative awareness method. The method comprises a data and model joint scheduling decision algorithm based on centralized offline training, a distributed online reasoning framework and deep reinforcement learning. According to the invention, a complex situation awareness task is divided into a target identification-target tracking-target positioning AI service chain; an AI service is cooperatively provided by using an unmanned aerial vehicle cluster; a partially observable Markov decision process is adopted for modeling environment-action-reward of the intelligent agent; and finally, training a decision model based on a deep reinforcement learning algorithm, and deploying the model to each unmanned aerial vehicle node for independent data scheduling and model dynamic activation decision. According to the method, the problem of dynamic coordination of chained AI service tasks (such as situation awareness tasks and micro-services) is effectively solved, higher flexibility and expansibility are achieved, the resource utilization rate is also remarkably improved, and meanwhile, stable operation of the system can still be kept when a node fault occurs.
Owner:UNIV OF ELECTRONICS SCI & TECH OF CHINA

Unmanned aerial vehicle moving edge calculation trajectory planning method based on multi-target reinforcement learning

The invention discloses an unmanned aerial vehicle moving edge calculation trajectory planning method based on multi-target reinforcement learning. Aiming at the problems of limited energy and uneven task distribution of unmanned aerial vehicles in a multi-unmanned aerial vehicle assisted mobile edge computing system, the method comprises the following steps: collecting real-time state data of the unmanned aerial vehicles, evaluating system target preferences (minimizing service interruption time or maximizing computing task load), constructing a distributed partially observable Markov decision process model, and calculating the target preferences of the unmanned aerial vehicles in the distributed partially observable Markov decision process model. And iteratively updating model parameters in combination with a crowding distance optimization experience playback mechanism so as to dynamically generate an optimal flight path. Experiments show that the method can adaptively balance and calculate the task load and the service continuity, effectively improves the system stability and the resource utilization rate, and is suitable for dynamic scenes such as disaster rescue and intelligent traffic.
Owner:WUHAN UNIV OF TECH

Caching, migration and unloading collaborative optimization method in satellite-assisted ocean network

The invention particularly relates to a cache, migration and unloading collaborative optimization method in a satellite-assisted ocean network, which comprises the following steps: constructing a satellite-assisted ocean network architecture, mapping task completion delay to system utility according to the satellite-assisted ocean network architecture, and constructing a target function with the purpose of maximizing the system utility under multi-dimensional constraints; converting an objective function optimization problem into a partially observable Markov decision process; and solving an objective function optimization problem based on attention-enhanced multi-agent reinforcement learning to obtain an optimal solution. According to the method, the problem of low-delay and efficient task calculation under the satellite-assisted ocean network architecture is solved.
Owner:NORTHWESTERN POLYTECHNICAL UNIV

Vulnerability intelligent collaborative detection method and system based on multi-agent reinforcement learning

The invention discloses a vulnerability intelligent collaborative detection method and system based on multi-agent reinforcement learning. The method comprises the following steps: independently executing detection according to local observation information and a strategy generation model through an intelligent agent deployed at an edge node; the strategy generation model is constructed based on a decentralized partial observable Markov decision process, and is obtained through cloud centralized training and global shared reward function optimization. When the agent obtains a forward reward, packaging a corresponding vulnerability context and related strategy parameters into an experience package; and the experience packet is diffused in the cooperative detection network in a mode of randomly selecting neighbor nodes and forwarding step by step. And after each agent receives the data, analyzing the content of the experience package and comparing the content with the local environment so as to dynamically adjust local strategy parameters and realize collaborative optimization. According to the vulnerability intelligent collaborative detection method and system based on multi-agent reinforcement learning provided by the invention, the problems that network security vulnerability detection is poor in expandability and lacks knowledge generalization ability are solved.
Owner:SHIZUISHAN POWER SUPPLY COMPANY OF STATE GRID NINGXIA ELECTRIC POWER +1

Main distribution coordinated voltage optimization control method and equipment under communication and electrical coupling

The present invention discloses a method and device for optimizing voltage control of main distribution coordination under communication and electrical coupling. The method is based on a model prediction rolling optimization framework, takes voltage deviation minimization as the goal, and constructs a distributed voltage control model under communication and electrical coupling; decomposes the distributed voltage control model into sub-problems of adjacent partition information interaction and multiple partition parallel control optimization and reconstructs it into a partially observable Markov decision process; uses a multi-agent reinforcement learning algorithm to quickly solve the reconstructed model and determine the voltage control method. The present invention can improve the voltage regulation accuracy and real-time performance, reduce the communication bandwidth requirements, and make the voltage under main distribution coordination run more safely and stably.
Owner:HEFEI UNIV OF TECH

Machine learning and data classification for operating a device such as a vehicle

A computer that includes a processor and a memory, the memory including instructions executable by the processor to input an image to a first neural network to generate a first detected object and input the image to a second neural network to generate a reconstructed image which is input to a third neural network to generate a second detected object. The image can be divided into portions and the portions input to respective fourth neural networks to generate portions of a third detected object. The first detected object, the second detected object, the portions of the third detected object, and context data can be input to a partially observable Markov decision process to generate a high confidence detected object.
Owner:FORD GLOBAL TECH LLC +1

Proximity information associated multi-unmanned aerial vehicle dynamic task allocation method in emergency scene

The invention discloses a multi-unmanned aerial vehicle dynamic task allocation method for adjacent information association in an emergency scene. The method comprises the following steps: constructing a multi-unmanned aerial vehicle system model in the emergency scene; constructing a dynamic task model; constructing a multi-unmanned aerial vehicle dynamic task allocation problem into a maximized multi-unmanned aerial vehicle task coverage optimization problem, and providing a maximized task coverage objective function; the maximized unmanned aerial vehicle task coverage is expressed as a partially observable Markov decision process; a multi-unmanned aerial vehicle depth deterministic strategy gradient algorithm based on adjacent information association is utilized, a system state is formed through information exchange between different unmanned aerial vehicles, and the maximum coverage of a task point in an emergency scene within limited time is found out. According to the multi-unmanned aerial vehicle depth deterministic strategy gradient collaboration method based on adjacent information association, the efficiency of multi-unmanned aerial vehicle collaboration under a dynamic task is improved, and the requirement of the multi-unmanned aerial vehicle for maximizing task coverage within limited time in an emergency scene is met.
Owner:HOHAI UNIV