Industrial control cross-layer security decision method based on partial observable Markov decision

By applying the POMDP-based T-DRQN deep reinforcement learning method in industrial control systems, the problem of intrusion response decision-making in some considerable system scenarios is solved, and more efficient intrusion response strategy generation and industrial control system security improvement are achieved.

CN119945764AActive Publication Date: 2025-05-06BEIJING UNIV OF TECH
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202510067740.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-01-16
Publication Date
2025-05-06
Estimated Expiration
2045-01-16

AI Technical Summary

Technical Problem

Existing intrusion response decision-making methods are difficult to effectively guide intrusion response decisions in some considerable industrial control system scenarios, especially when the system state information is not fully known.

Method used

A deep reinforcement learning method based on partially considerable Markov decision-making process (POMDP) ​​is adopted, and a T-DRQN algorithm is combined with LSTM and deep Q networks, and a dual-branch neural network architecture is used to distinguish state value and action advantages to generate an intrusion response strategy.

Benefits of technology

It significantly improves the security of industrial control systems, can effectively resist cross-layer attacks, and is suitable for industrial control scenarios where system status information is considerable.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119945764A_ABST
    Figure CN119945764A_ABST
Patent Text Reader

Abstract

The invention discloses a partially observable Markov decision-based industrial control cross-layer security decision-making method, which comprises the following steps of: 1, performing state perception on an industrial control system with partially observable state information by using a monitoring system, and collecting training data through continuous interaction between a model and the industrial control system; and step 2, combining data obtained by interaction with historical observation information for training by using a T-DRQN deep reinforcement learning method based on a POMDP decision framework. And 3, evaluating the response effect of the intrusion response strategy by using the alarm information of the network layer equipment node and the reading of the physical layer sensor, and analyzing the difference of the response performance between different models by using the indexes. According to the method, multi-stage cross-layer attacks from a network space to a physical space can be effectively resisted, a double-branch network structure is adopted, state values and action advantages are distinguished more obviously, intrusion response decisions are guided more accurately and efficiently, and the method is more suitable for processing industrial control scenes with partially considerable system state information.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Patent Citations

  • Defense strategy generation method and system for industrial control system

    CN113132405A

  • 5G NR downlink scheduling time delay optimization system based on reinforcement learning

    CN113395723A

  • Layered collaborative deep reinforcement learning decision-making method for multi-agent path planning

    CN119148519A

  • Handover optimisation in a cellular network by multi-objective deep reinforcement learning applied by agents deployed in base-stations

    EP4391644A1