Deep reinforcement learning-oriented strategy anomaly detection method and device

Abstract

The invention discloses a deep reinforcement learning-oriented strategy anomaly detection method and device. The method comprises the steps of performing reinforcement learning on a DDPG network by using a collected state sample; constructing an imitation learning network comprising an actor network and a discriminator, and training the imitation learning network by using the collected state samples and expert state actions; generating a state action pair based on an input state sample by using the parameter-optimized DDPG network, discriminating the state action pair by using the parameter-optimized discriminator, and when the discrimination result is 1, considering that the action is not attacked; when the judgment result is 0, considering that the shape action is abnormal; and when the action is abnormal and the action amplitude difference is out of the threshold range, indicating that the action is attacked, and using a state action pair generated by the simulation learning network for replacing a state action pair generated by the DDPG network so as to guide the DDPG network to make a correct decision in the reinforcement learning process of the subsequent stage.

Description

technical field [0001] The invention belongs to the field of security defense of deep reinforcement learning, and in particular relates to a policy anomaly detection method and device for deep reinforcement learning. Background technique [0002] With the continuous development of artificial intelligence technology, the Deep Reinforcement Learning (DRL) method has been valued and favored by experts and scholars since it was proposed. It has been deeply developed and widely used in the fields of automatic driving, robot control, game gaming, medical health and so on. As an indispensable and important technology in the field of artificial intelligence, the method of DRL is also constantly being expanded. RL is a key part of DRL, the core of which is that the agent obtains a relatively better strategy by continuously maximizing reward rewards. However, related studies have shown that the agent is vulnerable to adversarial attacks during the policy execution phase. The attacke...

AI Technical Summary

Problems solved by technology

[0005] The detection effect of the above two abnormal strategy detection methods is not good, and there is an urgent need for a better abnormal strategy detection method

Images