An abnormal response and recovery method for an industrial control system based on a behavior twin model

Abstract

This invention provides a method for anomaly response and recovery in industrial control systems based on a behavioral twin model, relating to the field of industrial control system security defense technology. The method includes: constructing a topology diagram of the industrial control system and building behavioral state vectors for each node; modeling based on a graph neural network and introducing a physical constraint evaluation term into the loss function to train a behavioral twin model. The model updates the state in real time and detects anomalies, invoking a reinforcement learning strategy to output a high-level response decision, which, after physical constraint verification, is mapped to a control adjustment quantity and transmitted to a PID controller for smooth adjustment. The behavioral twin model continuously monitors the system state, evaluates it, dynamically corrects the control parameters, and re-executes them by the PID controller, forming a closed loop until the system recovers to a safe operating range. This invention achieves intelligent response and rapid recovery under abnormal operating conditions while ensuring the stability of industrial process control.

Description

Technical Field

[0001] This invention relates to the field of industrial control system security defense technology, and in particular to a method for abnormal response and recovery of industrial control systems based on behavioral twin models. Background Technology

[0002] Industrial control systems (ICS) are core systems supporting the safe operation of critical infrastructures such as power, water treatment, petrochemicals, and transportation. Their typical characteristics include a large number of devices, complex topologies, continuous operation, and deep coupling between physical processes and control systems. With the continuous improvement of the Industrial Internet and information technology, ICS are gradually becoming deeply integrated with information networks, significantly expanding the attack surface while improving production efficiency. Once an ICS is subjected to network attacks, abnormal command injection, sensor malfunctions, actuator failures, or communication anomalies, it may not only lead to control logic errors but also cause physical process instability, equipment damage, or even safety accidents. Therefore, compared to traditional information system security protection, ICS security protection not only needs to identify anomalies but also needs to promptly generate safe, stable, and executable response and recovery strategies after anomalies occur, enabling the system to return to a safe operating range as quickly as possible.

[0003] Existing anomaly response and recovery schemes for industrial control systems mainly fall into three categories. The first category is based on human experience and predefined rules, pre-setting control actions for specific alarm events. In this type of scheme, the response strategy is manually pre-set. When an anomaly involves multiple devices or control loops, fixed rules are difficult to dynamically adjust the recovery sequence and objectives, resulting in a sequential or localized recovery process that struggles to achieve a system-level coordinated response. Furthermore, as the system scales up, the number of rules increases dramatically, leading to higher maintenance costs. The second category is based on physical models, digital twins, or model predictive control, predicting states and calculating control inputs by constructing a system model. In this type of scheme, industrial processes involve multiple stages, multiple loops, and nonlinear coupling relationships, making unified modeling difficult. Model accuracy is also difficult to maintain over the long term due to changes in operating conditions and equipment aging, resulting in high maintenance costs. The third category is based on deep learning or reinforcement learning, generating response strategies by learning the mapping from states to control actions. In these types of solutions, the control actions generated by data-driven methods sometimes do not conform to physical laws, leading to unexpected system responses during the recovery process. Some solutions directly output the underlying control quantities, which may result in sudden changes or frequent switching of control quantities, easily causing system oscillations. In addition, the strategy output format is incompatible with the PID controller input interface, making it difficult to connect with the underlying control devices. Therefore, there is an urgent need to develop a solution to address these problems. Summary of the Invention

[0004] The purpose of this invention is to provide an abnormal response and recovery method for industrial control systems based on behavioral twin models, which improves the problems of insufficient system-level modeling capabilities, low security of recovery strategies, and insufficient connection between intelligent decision-making and underlying control in the prior art.

[0005] The present invention provides a method for anomaly response and recovery in industrial control systems based on a behavioral twin model, which adopts the following technical solution:

[0006] A method for anomaly response and recovery in industrial control systems based on a behavioral twin model, specifically including:

[0007] Construct a topology graph of the industrial control system, where nodes represent devices or functional units in the industrial control system, edges represent the connection relationships between nodes, and construct a behavior state vector for each node.

[0008] The topology is modeled based on a graph neural network, with the behavior state vector as the node input feature. A physical constraint evaluation term is introduced into the loss function of the graph neural network to train a behavior twin model.

[0009] The behavioral twin model acquires the operating data of the industrial control system in real time and updates the behavioral state vector therein; when an anomaly is detected, an anomaly assessment information is determined based on the behavioral twin model, the anomaly assessment information includes the scope of the anomaly's impact, the propagation path, and the risk level; a trained reinforcement learning strategy is invoked, the updated behavioral state vector and the anomaly assessment information are input, and a high-level response decision is output;

[0010] The high-level response decision is verified based on the physical constraint evaluation terms, and the verified high-level response decision is converted into a control adjustment quantity; the control adjustment quantity is transmitted to the PID controller, and the PID controller performs the adjustment.

[0011] The behavioral twin model monitors the system state after the PID controller executes, corrects the control adjustment amount based on the system state, and transmits the corrected control adjustment amount to the PID controller, which then re-executes the adjustment until the system returns to a safe operating range.

[0012] This invention provides an anomaly response and recovery method for industrial control systems based on a behavioral twin model. It combines graph neural networks to characterize the overall operating state of the industrial control system and introduces physical constraint evaluation terms into the model training process, ensuring that the behavioral twin model maintains consistency with the physical laws of the industrial process during state evolution. Based on this, the behavioral twin model serves as the decision-making environment for a reinforcement learning agent, training the agent to output high-level response decisions. During the online execution phase, the behavioral twin model performs physical constraint verification on the high-level response decisions and maps them to the setpoint adjustment of the PID controller, which then performs smooth adjustment. Simultaneously, the behavioral twin model continuously monitors the system state and dynamically corrects the control objective, forming a collaborative closed loop of intelligent decision-making and low-level stable control. This improves upon the shortcomings of existing methods in system-level behavioral modeling, recovery strategy security, and the connection between intelligent decision-making and low-level control.

[0013] Optionally, when constructing the topology diagram of the industrial control system, the following are included:

[0014] Based on the process flow diagram, control logic, equipment connection relationships, and communication links of the industrial control system, determine the nodes and their connection relationships; map physical pipeline connections, electrical connections, control loops, communication links, material transfer relationships, and energy transfer relationships as the edges; assign different edge attributes to edges of different connection types.

[0015] Optionally, the behavioral twin model is constructed in the following manner:

[0016] The topology graph is constructed and a behavior state vector is established for each node. The behavior state vector includes key process parameters of the equipment, control status information, equipment operation status information, network and communication status information, historical change characteristics, and statistical characteristics.

[0017] The topology is modeled based on a graph neural network, with the behavior state vector as the node input feature. A physical constraint evaluation term is introduced into the loss function of the graph neural network. The physical constraint evaluation term includes mass conservation constraint residual, energy conservation constraint residual, key variable safety boundary penalty term, and process rule violation penalty term. The behavior twin model is then trained to obtain the model.

[0018] Optionally, the behavioral twin model performs a physical constraint check on the state update result of each node during the information transmission process of the graph neural network; if the node state exceeds the safety boundary, the updated node state is corrected according to the physical constraint evaluation term.

[0019] Optionally, the training process of the reinforcement learning strategy includes:

[0020] Using the behavioral twin model as an interactive environment, simulate abnormal or attack scenarios;

[0021] The state space of a reinforcement learning agent consists of the node behavior state, system-level behavior representation, anomaly type, anomaly node location, anomaly propagation path, node risk level, physical constraint evaluation result, and safety margin output by the behavioral twin model.

[0022] The action space of a reinforcement learning agent consists of the recovery target node, recovery priority, target safety interval, setpoint adjustment direction, setpoint adjustment range, recovery time window, or PID parameter correction suggestions.

[0023] The reward function of a reinforcement learning agent consists of system stability, recovery time, state deviation, degree of violation of physical constraints, and risk of anomaly propagation.

[0024] The trained reinforcement learning strategy is obtained through repeated training.

[0025] Optionally, when validating the high-level response decision based on the physical constraint evaluation item and converting the validated high-level response decision into a control adjustment quantity, the following steps are included:

[0026] The high-level response decision is input into the behavioral twin model to obtain the predicted behavior states of each node; the predicted behavior states are compared with the target behavior range and the physical constraint evaluation items; when the predicted behavior states exceed the target behavior range or do not meet the physical constraint evaluation items, the setpoint adjustment direction and setpoint adjustment magnitude in the high-level response strategy are corrected; the corrected setpoint adjustment direction and setpoint adjustment magnitude are mapped to the control adjustment amount;

[0027] The control adjustment quantities include target behavior range, setpoint adjustment suggestions, setpoint change rate, or PID parameter correction quantities.

[0028] Optionally, when the control adjustment is transmitted to the PID controller and the PID controller performs the adjustment, it includes:

[0029] The PID controller acquires the deviation between the current measured value and the target setpoint in the control adjustment; calculates the control output based on the proportional, integral, and derivative of the deviation; during the calculation of the control output, a limiting mechanism is used to limit the amplitude of the control output, a speed limiting mechanism is used to limit the rate of change of the control output, and an anti-integral saturation mechanism is used to limit the cumulative range of the integral term; and the control output after limiting, speed limiting, and anti-integral saturation processing is output to the actuator of the industrial control system.

[0030] Optionally, when the behavioral twin model monitors the system state after the PID controller executes, and corrects the control adjustment amount based on the system state, it includes:

[0031] The behavioral twin model obtains the current behavior state vector of each node after the PID controller is executed, and corrects the control parameters based on the deviation of the current behavior state vector from the target behavior range, the current safety margin, and the anomaly propagation path; it generates the corrected control adjustment amount based on the corrected control parameters and outputs it to the PID controller.

[0032] Optionally, the control parameters include the recovery target, adjustment priority, setpoint change rate, and PID parameter correction amount.

[0033] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0034] 1. This invention utilizes graph neural networks to model the topology, equipment states, and inter-equipment coupling relationships of industrial control systems. It also introduces physical constraints, process rules, and safety boundaries to construct a behavioral twin model oriented towards anomaly response and recovery. Compared to traditional methods based on local states or flattened state vectors, this model better reflects the overall system behavior and the impact of anomaly propagation, providing a reliable basis for response decisions.

[0035] 2. This invention uses a behavioral twin model as a reinforcement learning interactive environment, and combines information such as node behavior state, anomaly propagation path, physical constraint evaluation results and security margin to train response and recovery strategies. This enables the system to generate recovery targets, adjust directions or control reference values ​​under different anomaly or attack scenarios, thereby improving the adaptive capability of the response strategy and reducing the reliance on manual rules and static plans.

[0036] 3. This invention first verifies the high-level response decisions generated by reinforcement learning through a behavioral twin model for physical constraints and process rules, then maps them to executable recovery targets or control reference values, and finally executes continuous and smooth low-level control by a PID controller. Through the above-mentioned constraint verification, target mapping, and closed-loop correction mechanisms, problems such as control abrupt changes, overshoot, or violation of safety boundaries can be avoided, improving the safety, stability, and engineering feasibility of the anomaly recovery process. Attached Figure Description

[0037] Figure 1 The flowchart illustrates an abnormal response and recovery method for an industrial control system based on a behavioral twin model, as provided in this embodiment of the invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Unless otherwise defined, the technical or scientific terms used herein should have the ordinary meaning understood by those skilled in the art. The terms "comprising" and similar expressions used herein mean that the element or object preceding the word covers the element or object listed after the word and its equivalents, but does not exclude other elements or objects.

[0039] See Figure 1 This invention provides a method for anomaly response and recovery in industrial control systems based on a behavioral twin model, comprising the following steps:

[0040] S1. Construct the topology graph of the industrial control system. The nodes of the topology graph represent the equipment or functional units in the industrial control system, the edges of the topology graph represent the connection relationship between the nodes, and construct a behavior state vector for each node.

[0041] S2. Model the topology graph based on graph neural network, use the behavior state vector as the node input feature, introduce the physical constraint evaluation term into the loss function of graph neural network, and train to obtain the behavior twin model.

[0042] S3. The behavioral twin model acquires the operating data of the industrial control system in real time and updates the behavioral state vector therein. When an anomaly is detected, the anomaly assessment information is determined based on the behavioral twin model. The anomaly assessment information includes the scope of the anomaly's impact, the propagation path, and the risk level. The trained reinforcement learning strategy is invoked, the updated behavioral state vector and the anomaly assessment information are input, and the high-level response decision is output.

[0043] S4. Verify the high-level response decision based on the physical constraint evaluation items, and convert the verified high-level response decision into a control adjustment quantity; transmit the control adjustment quantity to the PID controller, and the PID controller executes the adjustment.

[0044] S5. The behavioral twin model monitors the system state after the PID controller executes, corrects the control adjustment amount according to the system state, and transmits the corrected control adjustment amount to the PID controller, which then re-executes the adjustment until the system returns to the safe operating range.

[0045] In some embodiments, when constructing the topology diagram of the industrial control system in step S1, nodes and their connection relationships are determined based on the process flow diagram, control logic, equipment connection relationships, and communication links of the industrial control system; physical pipeline connections, electrical connections, control loops, communication links, material transfer relationships, and energy transfer relationships are mapped as edges; and different edge attributes are assigned to edges of different connection types. The specific process is as follows:

[0046] S11. Obtain the structural description information of the industrial control system. The structural description information includes the process flow diagram, control logic diagram, equipment connection list, and communication network topology diagram of the industrial control system. The process flow diagram describes the material flow, energy transfer relationships, and instrument control points between various devices in the industrial control system. The control logic diagram describes the information flow and control dependencies between controllers, actuators, and sensors. The equipment connection list records the physical piping connections, electrical connections, and communication link information between various devices. The communication network topology diagram describes the logical relationships between various network nodes in communication networks such as industrial Ethernet and fieldbus. The above structural description information can be obtained from the industrial control system's design documents, as-built documentation, or system configuration files.

[0047] S12. Determine the node set V of the topology graph based on the structural description information. Each node in node set V corresponds to a physical device, control unit, or functional unit in the industrial control system. Physical devices include process equipment such as pumps, compressors, fans, valves, heat exchangers, reactors, storage tanks, pipeline nodes, motors, and frequency converters. Control units include control devices such as programmable logic controllers, distributed control system controllers, remote terminal units, industrial computers, and embedded controllers. Functional units include sensors, actuators, operator stations, engineer stations, and historical data servers. When determining nodes, each device, control unit, and functional unit appearing in the structural description information is treated as an independent node, and a unique node identifier is assigned to each node. The node identifier can be represented by a numerical code or device tag number, and a correspondence is established with the device number or functional location number in the structural description information to facilitate the mapping of subsequent operating data and behavioral state vectors.

[0048] S13. Determine the edge set E of the topology graph based on the structural description information. Each edge in edge set E represents a connection or interaction relationship between two nodes. Edges are mapped according to the actual connection relationships between devices in the industrial control system. Edge types include physical connection edges, control connection edges, communication connection edges, and material / energy transfer edges. Physical connection edges correspond to physical pipeline connections or electrical connections between devices. When there is a material conveying pipeline between two devices, a physical connection edge is established, with the direction of the edge indicating the material flow direction; when there is a power cable connection between two devices, a physical connection edge is established, representing the power transfer relationship. Control connection edges correspond to control signal connections between devices. When the output channel of the controller is connected to the control input terminal of the actuator, a control connection edge is established, with the direction from the controller to the actuator; when the measurement signal of the sensor is connected to the input channel of the controller, a control connection edge is established, with the direction from the sensor to the controller. Communication connection edges correspond to data communication connections between devices. When two devices exchange data through industrial Ethernet, fieldbus, serial communication, or wireless communication, a communication connection edge is established. Material / energy transfer edges correspond to the material or energy transfer relationships between devices. When the output material of one device becomes the input material of another device, a material transfer edge is established, with the direction from the upstream device to the downstream device. When one device transfers heat, mechanical energy, or electrical energy to another device, an energy transfer edge is established. When constructing the edge set E, corresponding edges are established item by item according to the connection relationships explicitly recorded in the structural description information. When there are multiple connection relationships between two nodes, an edge is established for each relationship, and each edge is distinguished by the edge type in the edge attributes.

[0049] S14. Assign edge attributes to each edge in the topology graph. Edge attributes describe the type and characteristics of the edge. Each edge's attributes include edge type, connection strength or weight, and direction identifier. When the edge type is a communication connection edge, it also includes the communication protocol type. The edge attributes identify the connection type of the edge, including physical connections, control connections, communication connections, and material / energy transfer connections. Connection strength or weight indicates the tightness of the coupling relationship between the two nodes corresponding to the edge: for material transfer edges, weight is assigned based on the material flow rate, with a larger flow rate resulting in a larger weight; for control connection edges, weight is assigned based on the response speed of the control loop, with a faster response speed resulting in a larger weight; in some preferred embodiments, the weight is represented by a normalized numerical value, ranging from [0,1]. The direction identifier indicates whether the edge has directionality. In physical pipe connections, the material flow direction, control signal flow direction, communication data flow direction, and material / energy transfer direction all have directionality and are represented by directed edges; non-directional connections such as electrical connections are represented by undirected edges. For the communication connection edge, the communication protocol type can also be set to identify the protocol type used for the communication connection, including Modbus, Profibus, OPC UA, IEC 61850, and MQTT;

[0050] S15. Based on the node set V, edge set E, and edge attributes, generate the topology graph G=(V,E) of the industrial control system. This topology graph G can be a directed graph, undirected graph, or a hybrid graph, where the directionality is determined by the direction identifier of each edge. The topology graph G is used to represent the topological relationships between various devices, control units, and functional units in the industrial control system, providing graph structure input for subsequent behavioral twin modeling based on graph neural networks. The generated topology graph G is stored in the form of an adjacency matrix and a node feature matrix, where the adjacency matrix represents the connection relationships and edge attributes between nodes, and the node feature matrix stores the initial behavioral state vector of each node.

[0051] S16. Construct a behavior state vector for each node in the topology diagram. The behavior state vector describes the operating state of the corresponding device or functional unit at a given moment. It is constructed as follows: Based on the node type, determine the set of state variables corresponding to that node. The set of state variables includes: key process parameters of the equipment, including flow rate, pressure, temperature, liquid level, rotational speed, and concentration; control status information, including valve opening degree, on / off status, setpoint, control mode, and actuator output; equipment operating status information, including start / stop status, fault status, alarm status, and health status; and network and communication status information, including communication latency, data packet loss rate, and abnormal command flags. Arrange the above state variables in a preset order to form the behavior state vector of that node. The dimensions of the behavior state vectors of each node can be the same or different, determined according to the actual number of monitored variables of the corresponding device or functional unit.

[0052] In some embodiments, in step S2, the topology is modeled based on a graph neural network, using behavioral state vectors as node input features. A physical constraint evaluation term is introduced into the loss function of the graph neural network. When training to obtain the behavioral twin model, the topology is constructed and a behavioral state vector is established for each node. The behavioral state vector includes key process parameters of the equipment, control state information, equipment operating state information, network and communication state information, historical change features, and statistical features. The topology is modeled based on a graph neural network, using behavioral state vectors as node input features. A physical constraint evaluation term is introduced into the loss function of the graph neural network. The physical constraint evaluation term includes mass conservation constraint residuals, energy conservation constraint residuals, key variable safety boundary penalty terms, and process rule violation penalty terms. The behavioral twin model is then trained. The specific process is as follows:

[0053] S21. Obtain the topology graph G=(V,E) and the behavior state vectors of each node. The behavior state vectors of each node are constructed according to the method in step S1, including key process parameters of the equipment, control status information, equipment operating status information, network communication status information, historical change characteristics, and statistical characteristics. Among them, key process parameters of the equipment include flow rate, pressure, temperature, liquid level, speed, and concentration. Control status information includes valve opening degree, on / off status, set value, control mode, and actuator output. Equipment operating status information includes start / stop status, fault status, alarm status, and health status. Network communication status information includes communication delay, data packet loss, and abnormal command flags;

[0054] S22. Construct a graph neural network model. Specifically, the topological graph G=(V,E) obtained in step S21 is used as the structural input of the graph neural network, and the time of each node is... The behavior state vector is used as the node input feature. For the ... There are nodes, and their behavior state vectors are denoted as... To eliminate the differences in the units of measurement of different variables, we first... The process parameters, control status information, equipment operating status information, and network communication status information are normalized; then, the normalized behavior state vector is mapped to an initial hidden representation of a unified dimension through a node feature mapping layer, specifically as follows:

[0055] ,

[0056] in, Represents a node At any moment The initial hidden representation, and These represent the node feature mapping parameters, This represents a nonlinear activation function. Through the above processing, node state variables of different types and scales can be mapped to a unified feature space, facilitating subsequent information transfer and state updates between nodes.

[0057] Encode the edge attributes of the edges in the topological graph. For nodes... Pointing to node edge Generate an edge attribute vector based on the edge's connection type, direction identifier, connection strength or weight, and communication protocol type. Connection types include physical connections, control connections, communication connections, and material or energy transfer connections. Edge attribute vectors. Used to describe nodes With nodes The coupling relationship between them enables graph neural networks to distinguish the impact of different connection relationships on node state changes during information transmission;

[0058] In the information transmission process of the l-th layer of a graph neural network, for nodes... First, start from its set of neighboring nodes. Obtain the hidden representation of adjacent nodes and combined with edge attribute vectors Generate neighbor node j pairs of nodes The message is specifically expressed as follows:

[0059] ,

[0060] in, Represents the nodes in the l-th layer Passed to node The news, and Representing nodes respectively and nodes The hidden representation at layer l, This represents the message generation function of the l-th layer, which can be implemented by a linear transformation function or a multilayer perceptron. This message generation process can simultaneously consider the current node state, the neighboring node states, and the connection relationship between the two nodes, thereby reflecting the physical coupling, control dependency, or communication association between devices in the industrial control system.

[0061] To further differentiate the importance of different neighboring nodes to the current node's state update, neighboring nodes are calculated based on node hidden representation and edge attributes. For nodes The attention score is specifically expressed as:

[0062] ,

[0063] in, Represents the nodes in the l-th layer Relative to node Attention score This represents the attention scoring function. Subsequently, for the node... The attention scores of all neighboring nodes are normalized to obtain the attention weights:

[0064] ,

[0065] in, Represents the nodes in the l-th layer Relative to node Attention score Represents the nodes in the l-th layer For nodes The influence weights, and satisfy:

[0066] ,

[0067] By calculating the attention weights as described above, the graph neural network can automatically increase the influence of key neighbor nodes, key connecting edges, or highly coupled relationships during state updates, while reducing the interference of weakly related nodes on the state update results.

[0068] Subsequently, for the nodes The neighbor messages are weighted and aggregated to obtain the node. The neighborhood aggregation information at layer l is specifically represented as follows:

[0069] ,

[0070] in, Represents a node The neighborhood aggregation information received at layer l. This neighborhood aggregation information integrates information from nodes. Changes in the operating status of adjacent devices or functional units, as well as the connection attributes and coupling strength between nodes;

[0071] After obtaining neighborhood aggregation information, combined with nodes Its own hidden representation at layer l For nodes Performing a state update is specifically represented as follows:

[0072] ,

[0073] in, Represents a node The hidden representation at layer l+1, and These are the trainable parameters for the l-th layer. Through this state update process, the node... When updating its own behavior state, it retains its current operating state information and integrates the operating states of adjacent devices or functional units and the connection relationship between them, thereby realizing the modeling of the dynamic coupling relationship of the industrial control system.

[0074] Graph neural networks include at least one information passing layer. When multiple information passing layers are used, the first layer aggregates information from first-order neighbor nodes, and subsequent layers aggregate information from higher-order neighbor nodes, enabling the model to characterize the multi-level propagation effects of anomalies, disturbances, or attacks in the topology. For directional edges, message aggregation is performed based on both incoming and outgoing edges to distinguish the influence of upstream devices on downstream devices and the influence in the direction of control feedback. For edges with different connection types, corresponding edge type weights or edge type transformation parameters can be set to improve the model's ability to express different coupling relationships.

[0075] After completing the L-layer information transmission, the final hidden representation of each node is obtained. Based on this final hidden representation, nodes are predicted through the output layer. The behavioral state at the next moment is specifically represented as follows:

[0076] ,

[0077] in, Indicates the predicted nodes The behavioral state at the next moment and This represents the output layer parameters. If it's necessary to predict the state of multiple steps within a preset time window, it can be represented as:

[0078] ,

[0079] in, Indicates the prediction step size. Indicates the length of the prediction time window. and They represent the first Output layer parameters corresponding to each prediction step size;

[0080] Simultaneously, the final hidden representations of all nodes are pooled to obtain a system-level behavioral representation, specifically:

[0081] ,

[0082] in, Indicates that the industrial control system is at any time The system-level behavior representation, This represents the average pooling, max pooling, or weighted pooling function. Used to characterize the overall operating status of the industrial control system, and as basic data for subsequent anomaly assessment, reinforcement learning strategy input, and physical constraint evaluation;

[0083] The graph neural network model described above can integrate the topology, node operating status, and inter-node connection relationships of the industrial control system into the behavior modeling process. This allows the behavior twin model to not only describe the local state changes of a single device, but also reflect the linkage effects and anomaly propagation trends between devices, thereby improving the accuracy, scalability, and engineering applicability of system-level behavior modeling.

[0084] S23. Constructing Physical Constraint Evaluation Terms. To ensure that the prediction results of the behavioral twin model conform to the actual operating laws of the industrial process, physical constraint evaluation terms are introduced during model training. These terms are used to constrain and verify the predicted states output by the graph neural network, enabling the model to learn both data features and physical laws during training. Physical constraint evaluation terms can be constructed based on the node states or system states predicted by the behavioral twin model, including mass conservation constraint residuals, energy conservation constraint residuals, key variable safety boundary penalty terms, and process rule violation penalty terms.

[0085] Specifically, suppose the graph neural network outputs nodes based on step S22. The predicted behavioral state at the next moment is .from Predictive variables related to physical constraints are extracted, including predicted flow rate, predicted pressure, predicted temperature, predicted liquid level, predicted energy input / output, predicted equipment status, and predicted control status. Based on these predictive variables, residuals of mass conservation constraints, residuals of energy conservation constraints, safety boundary penalty terms for key variables, and penalty terms for violations of process rules are constructed.

[0086] First, construct the mass conservation constraint residuals. This applies to material balance nodes or material balance units in industrial control systems. Based on the predicted material flow rate entering the unit, the predicted material flow rate leaving the unit, and the change in the cumulative material amount within the unit, the mass conservation constraint residual is calculated, specifically expressed as follows:

[0087] ,

[0088] in, Material balance unit exist The residuals constrained by the conservation of mass at any given moment; Indicates to unit Input the set of upstream nodes of the material; Indicates from unit Output the set of downstream nodes for the material; Indicates from node Inflow unit Predicted material flow; Indicates from unit Flow to Node Predicted material flow; Representation unit The change in the cumulative amount of predicted material between adjacent sampling times; Indicates the sampling time interval. When The larger the absolute value, the greater the deviation of the predicted state from the law of mass conservation;

[0089] Based on the residuals of the mass conservation constraints in each material balance unit, a mass conservation constraint evaluation term is constructed:

[0090] ,

[0091] in, This represents the evaluation item for the mass conservation constraint. This represents the set of material balance units that require mass conservation verification. This evaluation term can suppress predicted states in the model output that do not satisfy material balance relationships, thus improving the accuracy of the behavioral twin model's description of the material flow process.

[0092] Secondly, construct energy conservation constraint residuals. This applies to energy balance units that exhibit heat transfer, power input / output, or energy conversion relationships. Based on the predicted energy input, predicted energy output, predicted power input and output, and the cumulative energy change within the unit, the energy conservation constraint residual is calculated, specifically expressed as follows:

[0093] ,

[0094] in, Represents an energy balance unit exist The residual is constrained by the energy conservation principle at any given moment; This represents the set of upstream nodes that are input to the energy balance unit c; This represents the set of downstream nodes output from energy balance unit c; Indicates input to unit Predicted energy; Indicates by unit The predicted energy output; Indicates input to unit Predicted power or external work; Representation unit The predicted output power or the work done externally; Representation unit The change in the internally predicted cumulative energy amount; Indicates the sampling time interval;

[0095] Based on the energy conservation constraint residuals of each energy balance unit, an energy conservation constraint evaluation term is constructed:

[0096] ,

[0097] in, This represents the evaluation term for the energy conservation constraint. This represents the set of energy balance units that require energy conservation verification. This evaluation item helps prevent model predictions from exhibiting state changes that do not conform to the laws of heat transfer, energy conversion, or power balance.

[0098] Next, construct the safety boundary penalty term for key variables. For nodes... Key process variables Pre-set its allowed lower security limit and safety limit A safety boundary penalty is incurred when the predicted variable value exceeds the safety boundary; the penalty is zero when the predicted variable value is within the safety boundary. The safety boundary penalty term for key variables is expressed as follows:

[0099] ,

[0100] in, This represents the safety boundary penalty term for key variables; Represents a node The corresponding set of key process variables; Represents a node Key process variables exist The predicted value at any given time; and These represent the lower and upper safety limits of the key process variable, respectively. This penalty term helps the behavioral twin model reduce predictions that exceed safety boundaries such as pressure, temperature, liquid level, flow rate, and rotational speed during training.

[0101] Then, penalties for violating process rules are constructed. Process rules include equipment start-up and shutdown rules, interlock protection rules, control mode switching rules, valve opening limit rules, upstream and downstream equipment coordinated operation rules, and communication status constraint rules. For the r-th process rule, it is represented as a rule constraint function. .when When, it indicates that the predicted state meets the process rule; when When this occurs, it indicates that the predicted state violates the process rule. The penalty for violating the process rule is represented as follows:

[0102] ,

[0103] in, This indicates a penalty for violating process rules; Represents a set of process rules; Indicates that all nodes are in The set of predicted behavioral states at any given time; This indicates the current control input or control state. This penalty term ensures that the behavioral twin model satisfies process logic and field operation rules when predicting state evolution, avoiding situations such as equipment state conflicts, interlock condition violations, or control actions that do not meet process requirements.

[0104] After obtaining the above-mentioned constraint evaluation terms, construct the physical constraint evaluation terms:

[0105] ,

[0106] in, express Physical constraint evaluation items at any given time; and These represent the weighting coefficients for the quality conservation constraint evaluation item, energy conservation constraint evaluation item, critical variable safety boundary penalty item, and process rule violation penalty item, respectively. These weighting coefficients can be set based on the process type, equipment importance, safety level requirements, or historical operating data of the industrial control system.

[0107] When it is necessary to constrain the prediction results of multiple steps within a preset time window, the physical constraint evaluation items corresponding to each prediction step can be accumulated to obtain the physical constraint evaluation items within the time window. :

[0108] ,

[0109] in, Indicates the length of the prediction time window; express The physical constraint evaluation term at each moment. Through this multi-step physical constraint evaluation term, the state evolution trend of the behavioral twin model can be constrained in the continuous prediction process, avoiding deviations from the physical law at subsequent prediction moments due to the model only satisfying the constraints at a single moment.

[0110] The physical constraint evaluation terms constructed in the above manner can be used as a component of the loss function during the training of the behavioral twin model, and can also serve as an evaluation basis for high-level response decision verification and control adjustment correction during the online anomaly response phase. Therefore, the behavioral twin model can not only learn the state change patterns of the industrial control system based on data, but also maintain consistency with the physical laws, safety boundaries, and process rules of the industrial process during prediction and decision-making, thereby improving the safety, stability, and engineering feasibility of the anomaly response and recovery process.

[0111] S24. Train to obtain the behavioral twin model. Specifically, after completing the graph neural network model construction in step S22 and the physical constraint evaluation item construction in step S23, the graph neural network model is trained using historical operating data, normal operating condition data and abnormal or disturbance scenario data of the industrial control system to obtain the behavioral twin model;

[0112] First, a training sample set is constructed. According to a preset sampling period, the operational data of each node at continuous time points are obtained from the historical database, real-time database, or simulation system of the industrial control system, and mapped to the corresponding node's behavior state vector. For each time point... The behavior state vectors of all nodes are combined into a node state set. The node state set is formed by combining the actual behavior state vectors of the next moment. The training samples can be represented as:

[0113] ,

[0114] in, Represents the training sample set, This represents the topology diagram of an industrial control system. Indicates time The set of node behavior states, Indicates time The set of real node behavior states. For cases requiring multi-step prediction, the training samples can be expanded to:

[0115] ,

[0116] Where T represents the prediction time window length. Through the above sample construction method, the model can learn the state evolution relationship of the industrial control system under different operating conditions;

[0117] Secondly, the training samples are preprocessed. Continuous variables in the behavior state vectors of each node are normalized, while discrete variables such as device start / stop status, control mode, alarm status, and communication status are encoded. For missing data, interpolation between adjacent time points, historical mean filling, or removal of outlier samples are used. Variables with different dimensions are uniformly mapped to a preset numerical range. The preprocessed node behavior state vectors are then used as input to the graph neural network model.

[0118] Then, the training samples are input into the graph neural network model constructed in step S22. For each training sample, using... and As input, the predicted behavior state of each node in the next time step is obtained through node feature mapping, edge attribute encoding, neighbor message generation, attention weight calculation, neighborhood message aggregation, and node state update. Among them, nodes The predicted behavioral state at the next moment is ;

[0119] To ensure that the predicted results closely approximate the actual operating state, a data prediction loss term is constructed. The data prediction loss term measures the deviation between the predicted behavioral state and the actual behavioral state, and is specifically expressed as follows:

[0120] ,

[0121] in, express The data prediction loss term at each time point, Represents a node exist The actual behavioral state at any given moment Represents a node exist Predictable behavioral state at any given moment;

[0122] Based on the data prediction loss term, the physical constraint evaluation term constructed in step S23 is introduced. Furthermore, a model parameter regularization term can be introduced. This is to suppress model overfitting. Model parameter regularization term. It can be represented as:

[0123] ,

[0124] Where θ represents the set of trainable parameters in the graph neural network model;

[0125] Therefore, the total training loss function for constructing the behavioral twin model :

[0126] ,

[0127] in, Indicates the prediction step size. This indicates the length of the prediction time window. This represents the weighting coefficient of the physical constraint evaluation item. This represents the weight coefficients of the regularization term in the model parameters. Multi-step prediction training can improve the behavioral twin model's ability to characterize continuous state evolution processes and reduce state shifts caused by error accumulation during long-term prediction.

[0128] During training, the backpropagation algorithm is used to calculate the total training loss function. Relative to model parameters The gradient is calculated, and gradient descent-type optimization algorithms are used to update the model parameters. The parameter update process can be represented as:

[0129] ,

[0130] in, Indicates the first Model parameters at the next iteration Indicates the first Model parameters after the next iteration Indicates the learning rate. This represents the gradient of the total training loss function with respect to the model parameters. Gradient descent optimization algorithms can include stochastic gradient descent, Adam, or other adaptive optimization algorithms.

[0131] Training parameters include the learning rate, batch size, number of training epochs, prediction time window length, hidden layer dimension, number of information propagation layers, physical constraint weight coefficients, and regularization weight coefficients. Specifically, the learning rate can be adjusted based on the reduction in training loss; the batch size is set according to computational resources and real-time requirements; the number of training epochs is determined based on the convergence of the validation set loss; the number of information propagation layers is set based on the anomaly propagation range and topological complexity; and the physical constraint weight coefficients are set according to the safety level of the industrial control system and the importance of the constraints.

[0132] In each training round, training samples are input into the graph neural network model in batches. The data prediction loss term, physical constraint evaluation term, and regularization term are calculated, and the model parameters are updated based on the total training loss function. Training stops when both the training loss and validation loss stabilize, or when the validation loss no longer decreases within a preset number of rounds. The training stopping condition can be expressed as:

[0133] ,

[0134] in, and Indicates the first Wheel and First The validation set loss after each round of training. This indicates the preset convergence threshold;

[0135] After training, the model parameters with the lowest validation set loss and those meeting the preset physical constraint violation requirements are selected as the final model parameters, resulting in a behavioral twin model. The final behavioral twin model can predict the behavioral state of each node in the next time step or within a preset time window based on the current industrial control system topology and node behavior state vectors, and output a system-level behavioral representation. This model is used for subsequent anomaly detection, anomaly impact range assessment, anomaly propagation path analysis, reinforcement learning strategy training, and online response decision verification.

[0136] Through the above training process, the behavioral twin model can simultaneously learn the state evolution law in the historical operation data of the industrial control system and the physical constraint relationship in the industrial process, so that its prediction results not only conform to the data change trend, but also meet the requirements of mass conservation, energy conservation, safety boundary and process rules, thus providing a reliable system-level behavioral model for abnormal response and collaborative recovery.

[0137] After training, the final model parameters, topology graph structure, node feature mapping rules, edge attribute encoding rules, and physical constraint evaluation terms are saved together to form an online-running behavioral twin model. During the online operation phase, the behavioral twin model receives real-time operational data collected by the industrial control system and, following the same node mapping relationships and feature preprocessing methods as in the training phase, converts the real-time operational data into current behavioral state vectors for each node, updating the node state representations in the model. Thus, the behavioral twin model obtained in the training phase can be integrated with the real-time state synchronization process in subsequent step S31, providing the current system behavioral state for anomaly detection, anomaly assessment, and response decision-making.

[0138] In some embodiments, the reinforcement learning strategy is trained in the following manner:

[0139] The behavioral twin model serves as the interactive environment for the reinforcement learning agent, simulating anomaly or attack scenarios. The reinforcement learning agent is a policy network or decision model used to generate high-level response decisions, taking the state information output by the behavioral twin model as input and the high-level response decisions as output. Specifically, in each training round, the reinforcement learning agent reads the current node's behavioral state, system-level behavioral representation, anomaly assessment information, and physical constraint evaluation results. Based on the current policy, it outputs suggestions for restoring the target node, restoration priority, target safety interval, setpoint adjustment direction, setpoint adjustment magnitude, restoration time window, or PID parameter correction. The behavioral twin model simulates system state evolution based on these high-level response decisions and calculates reward values ​​based on system stability, restoration time, state deviation, physical constraint violation degree, and anomaly propagation risk. The reinforcement learning agent updates its policy network parameters based on the reward values, enabling it to gradually learn to generate safe, stable, and efficient high-level response decisions under different anomaly or attack scenarios.

[0140] In one specific implementation, the reinforcement learning agent is trained using a proximal policy optimization algorithm; in other implementations, it can also be trained using a deep Q-network algorithm, a deep deterministic policy gradient algorithm, or a soft actor-commentator algorithm, with the specific algorithm selected based on whether the action space is continuous, the training stability requirements, and the real-time requirements of the industrial control system.

[0141] S31. The behavioral twin model, through its data interface with the industrial control system, collects real-time operational data from various devices, controllers, sensors, and actuators. This operational data includes equipment process parameters, control status information, equipment operating status information, and network communication status information. The behavioral twin model maps the collected operational data to the corresponding node's behavioral state vector, replacing the previous behavioral state vector in the model, thus completing the synchronous update of the behavioral state. The behavioral twin model stores the updated behavioral state vector in an in-memory database or a real-time database and updates the state representation of each node in the topology graph based on the updated behavioral state vector, ensuring that the topology graph always reflects the current operational state of the industrial control system.

[0142] S32. After the behavior state vector is updated, the behavior twin model detects whether there are any anomalies in the industrial control system based on the updated behavior state vectors of each node. Anomaly detection includes:

[0143] Threshold-based anomaly detection: The current value of the key process parameters of each node is compared with the preset safety threshold range. When the current value of any key parameter exceeds its preset safety upper limit or falls below its preset safety lower limit, the node is determined to be abnormal.

[0144] Anomaly detection based on behavior state vector prediction bias: Input the historical behavior state vector sequence within a preset time window before the current moment into the behavior twin model. The behavior twin model predicts the expected behavior state vector of each node at the current moment. Calculate the deviation between the expected behavior state vector and the actual updated behavior state vector. When the deviation exceeds a preset deviation threshold, it is determined that the node is abnormal.

[0145] Anomaly detection based on physical constraint evaluation terms: Substitute the updated behavior state vector or the behavior state predicted by the behavior twin model into the physical constraint evaluation terms defined in step S23. When any constraint residual or penalty term exceeds the preset constraint violation threshold, it is determined that there is an anomaly in the system.

[0146] Anomaly detection based on graph structure inconsistency: When the behavior state vector of a node changes significantly while the states of its neighboring nodes do not change accordingly, the node is determined to be abnormal.

[0147] S33. When an anomaly is detected, the behavioral twin model determines anomaly assessment information based on the current topology and the updated behavioral state vector. Anomaly assessment information includes the scope of the anomaly's impact, its propagation path, and its risk level.

[0148] Specifically, the scope of anomaly impact is determined by starting with the node whose behavior state vector is abnormal, traversing along the edges of the topology graph to adjacent nodes, and checking whether the behavior state vectors of adjacent nodes deviate from their normal operating range. Nodes whose behavior state vectors deviate from the normal operating range are included in the scope of anomaly impact, and the traversal continues outward until the behavior state vectors of adjacent nodes are all within the normal operating range or a preset traversal depth is reached. The propagation path of anomaly is determined based on the temporal order of the deviations of the behavior state vectors of each node within the scope of anomaly impact from the normal operating range, the connection relationships between nodes, and the attributes of the edges, determining the sequence of nodes and edges through which the anomaly propagates from the starting node to each affected node. The risk level of a node is determined based on the functional importance of each affected node in the industrial control system, its technological stage, the degree of deviation of the anomaly from its behavior state vector, and whether the node is located at the intersection of other propagation paths, classifying the affected nodes into risk levels. Risk levels can be represented by three levels: high, medium, and low, or by a numerical scoring method.

[0149] S34. After determining the anomaly assessment information, the behavioral twin model invokes the trained reinforcement learning policy, inputs the updated behavior state vector and the anomaly assessment information, and outputs a high-level response decision. The reinforcement learning policy takes the updated behavior state vectors of each node and the anomaly assessment information at the current moment as input, performs forward inference calculations through the policy network, and outputs a high-level response decision under the current anomaly scenario. The high-level response decision is output in the aforementioned action space format, including the target node to be restored, the restoration priority, the target safety interval, the direction of setpoint adjustment, the magnitude of setpoint adjustment, the restoration time window, and PID parameter correction suggestions. The high-level response decision is not directly sent as a control command to the field actuators, but rather serves as the input for the behavioral twin model to perform physical constraint verification and mapping in subsequent steps.

[0150] In some embodiments, in step S4, the high-level response decision is verified based on the physical constraint evaluation term, and the verified high-level response decision is converted into a control adjustment quantity. The control adjustment quantity is then passed to the PID controller. When the PID controller executes the adjustment, firstly, the high-level response decision is input into the behavioral twin model to obtain the predicted behavior states of each node. The predicted behavior states are compared with the target behavior range and the physical constraint evaluation term. When the predicted behavior states exceed the target behavior range or do not meet the physical constraint evaluation term, the setpoint adjustment direction and setpoint adjustment magnitude in the high-level response strategy are corrected. The corrected setpoint adjustment direction and setpoint adjustment magnitude are mapped to the control adjustment quantity. Then, the PID controller obtains the deviation between the current measured value and the target setpoint in the control adjustment quantity. The control output quantity is calculated based on the proportion, integral, and derivative of the deviation. During the calculation of the control output quantity, a limiting mechanism is used to limit the amplitude of the control output quantity, a rate limiting mechanism is used to limit the rate of change of the control output quantity, and an anti-integral saturation mechanism is used to limit the cumulative range of the integral term. The control output quantity after limiting, rate limiting, and anti-integral saturation processing is output to the actuator of the industrial control system. The specific process is as follows:

[0151] S41. After inputting the high-level response decision into the behavioral twin model, the behavioral twin model uses the current behavioral state vector of each node as its initial state and the setpoint adjustment direction and magnitude in the high-level response decision as control inputs. Based on the topology graph and graph neural network, it performs forward inference to simulate the evolution of the system state after the application of the high-level response decision. Specifically, the behavioral twin model applies the setpoint adjustment direction and magnitude in the high-level response decision to the corresponding recovery target node, updating the setpoint or control reference value in the node's behavioral state vector. Then, starting from the updated node's behavioral state vector, it updates the behavioral state vectors of each neighboring node in the topology graph layer by layer through multi-layer information transmission in the graph neural network, predicting the behavioral state of each node after the application of the high-level response decision. The predicted behavioral states of each node include the predicted values ​​of key process parameters, control states, and equipment operating states of each node at the next time step or multiple future time steps.

[0152] S42. Verify the predicted behavior states of each node based on physical constraints and process rules. Determine whether the predicted system state meets the physical constraints. The verification process uses the physical constraint evaluation items for quantitative judgment, including: comparing the predicted key process parameters with preset safety boundaries and calculating safety boundary penalty items; calculating the mass conservation constraint residual based on the predicted material flow rate and cumulative amount; calculating the energy conservation constraint residual based on the predicted energy input and output; comparing the predicted equipment start-up and shutdown states, interlock signals, etc., with preset process rules and calculating process rule violation penalty items. When any constraint residual or penalty item exceeds a preset threshold, it is determined that the predicted system state does not meet the constraints.

[0153] S43. When it is determined that the constraints are not met, the behavioral twin model corrects the adjustment direction and magnitude of the setpoint, so that the corrected high-level response decision, after re-deduction, can meet the physical constraints. The correction strategies include: when exceeding the safety boundary, reversing the adjustment direction and reducing the magnitude; when violating mass or energy conservation, compensating for the adjustment magnitude based on the conservation deviation; when violating process rules, adjusting the direction or magnitude to meet the rules. The corrected adjustment direction and magnitude are re-input into the behavioral twin model for deduction and verification, repeating this process until the predicted system state meets all constraints, or reaches the preset maximum number of correction iterations. If the maximum number of iterations is reached but the constraints are still not met, a preset conservative safety action is adopted, including maintaining the current setpoint or triggering a safety interlock.

[0154] S44. Once the predicted system state meets the constraints, the behavioral twin model converts the validated high-level response decision into control adjustment information. The control adjustment is a reference input directly usable by the PID controller, including the target setpoint, setpoint rate limit, PID parameter correction, or target behavior range. The mapping method is as follows: When the high-level response decision includes a target safety range and a setpoint adjustment direction, the behavioral twin model calculates the corresponding target setpoint based on the target safety range and determines the setpoint rate limit based on the setpoint adjustment magnitude and recovery time window; when the high-level response decision includes a PID parameter correction suggestion, the behavioral twin model converts the PID parameter correction suggestion into adjustments for proportional gain, integral time, or derivative time; when the high-level response decision includes a target behavior range, the behavioral twin model uses the center or boundary value of the target behavior range as the target setpoint.

[0155] S45. After receiving the control adjustment, the PID controller obtains the current measured value of the controlled variable and calculates the deviation between the current measured value and the target setpoint in the control adjustment. The PID controller calculates the control output based on the proportional, integral, and derivative of the deviation.

[0156] S46. In calculating the control output, the PID controller employs limiting, speed limiting, or anti-integral saturation mechanisms to constrain the control output. Specifically, the control output is limited to a preset output range to prevent the actuator from exceeding physical limits; the rate of change of the control output is limited to a preset upper limit to prevent the actuator from moving too fast and causing shocks; when the control output reaches the limiting boundary and the deviation direction has not changed, the integral term stops accumulating to avoid regulation lag or overshoot caused by integral saturation.

[0157] S47. The control output quantity after being processed by limiting amplitude, limiting speed and anti-integral saturation is output to the actuator of the industrial control system. The actuator includes regulating valve, pump, frequency converter and switching equipment.

[0158] In some embodiments, in step S5, the behavioral twin model monitors the system state after the PID controller executes, corrects the control adjustment amount based on the system state, and transmits the corrected control adjustment amount to the PID controller, which then re-executes the adjustment until the system recovers to a safe operating range. The specific process is as follows:

[0159] S51. After the PID controller performs adjustments, the behavioral twin model continuously acquires the operating data of each node through the data interface with the industrial control system, updates the behavioral state vector, and forms a real-time representation of the current system state. Based on the updated current behavioral state vectors of each node, the behavioral twin model performs a multi-dimensional evaluation of the system state after the PID controller execution. Evaluation indicators include whether key variables have returned to the safe range, whether the risk of anomaly propagation has decreased, whether the evaluation results of physical constraints have improved, whether the control action is smooth, and whether the recovery time meets the requirements.

[0160] In one specific implementation, the method for determining the safe range of key variable regression is as follows: Check the current values ​​of key variables at each node within the scope of the anomaly's influence. If all key variables are within the preset safe upper and lower limits, the indicator is deemed to meet the requirements. The method for evaluating the anomaly propagation risk indicator is as follows: Calculate the anomaly propagation risk value based on the changing trends of the number of nodes and risk levels within the current scope of the anomaly's influence. If the risk value is lower than a preset threshold, the indicator is deemed to meet the requirements. The method for determining the physical constraint evaluation results is as follows: Calculate the weighted sum of the residuals of mass conservation constraints, energy conservation constraints, variable safety boundary penalties, and process rule violation penalties under the current state. If the weighted sum is lower than a preset threshold, the indicator is deemed to meet the requirements. The method for determining the smoothness of control actions is as follows: Detect the change between two consecutive outputs of the PID controller. If the change is lower than a preset upper limit for the rate of change, the indicator is deemed to meet the requirements.

[0161] S52. Based on the evaluation results of step S51, the behavioral twin model comprehensively judges whether the system state after the PID controller execution meets the preset recovery conditions. The preset recovery conditions include: all key variables returning to the safe range; anomaly propagation risk reduced to below the preset safe level; all physical constraint evaluation results improved to the preset allowable range; control action smoothness meeting preset requirements; and cumulative recovery time not exceeding the recovery time window. When all the above conditions are met, the behavioral twin model determines that the system has recovered to the safe operating range, and the coordinated response and recovery process ends. When any of the above conditions are not met, the behavioral twin model determines that the system has not yet recovered to the safe operating range, and proceeds to step S53.

[0162] S53. When it is determined in step S52 that the system has not yet recovered to the safe operating range, the behavioral twin model corrects the control parameters based on the current deviation, safety margin, and abnormal propagation path. Specifically, based on the deviation between the current value of the key variable and the target safe range, the setpoint adjustment amount or the correction value of the setpoint change rate is calculated; based on the safety margin, i.e., the distance between the current value of the key variable and the safety boundary, the urgency of the adjustment and the allowable adjustment range are determined. When the safety margin is small, a smaller adjustment range is used to avoid overshoot; when the safety margin is large, the adjustment range can be appropriately increased to speed up the recovery; based on the changes in the abnormal propagation path, the recovery priority is adjusted to prioritize the adjustment needs of nodes on the critical propagation path or at high risk levels.

[0163] S54. The corrected control adjustment is retransmitted to the corresponding PID controller via the data interface. The PID controller recalculates the control output and adjusts the actuator based on the corrected control adjustment. The behavioral twin model again acquires the system state after execution and updates the behavioral state vector for the next round of evaluation and correction. This cycle of monitoring, evaluation, correction, and re-execution continues until the system state meets all preset recovery conditions and the system returns to a safe operating range.

[0164] In the aforementioned closed-loop correction process, the behavioral twin model applies physical constraints and process rule verification to the control adjustment in each loop to ensure that the corrected control adjustment is always within a feasible range and to avoid introducing new constraint violations or safety risks during the correction process.

[0165] Through the synergistic mechanism of combining reinforcement learning, behavioral twin models and PID control, this invention achieves intelligent response and rapid recovery under abnormal operating conditions while ensuring the stability of industrial process control. It effectively avoids system oscillations and safety hazards caused by traditional reinforcement learning methods directly participating in the underlying control.

[0166] While embodiments of the present invention have been described in detail above, it will be apparent to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations fall within the scope and spirit of the invention as set forth in the claims. Furthermore, the invention described herein may have other embodiments and can be implemented or carried out in various ways.

Claims

1. A method for anomaly response and recovery in industrial control systems based on a behavioral twin model, characterized in that, include: Construct a topology graph of the industrial control system, where nodes represent devices or functional units in the industrial control system, edges represent the connection relationships between nodes, and construct a behavior state vector for each node. The topology is modeled based on a graph neural network, with the behavior state vector as the node input feature. A physical constraint evaluation term is introduced into the loss function of the graph neural network to train a behavior twin model. The behavioral twin model acquires the operating data of the industrial control system in real time and updates the behavioral state vector therein; when an anomaly is detected, anomaly assessment information is determined based on the behavioral twin model, and the anomaly assessment information includes the scope of the anomaly's impact, the propagation path, and the risk level; The trained reinforcement learning policy is invoked, the updated behavior state vector and the anomaly evaluation information are input, and a high-level response decision is output. The high-level response decision is verified based on the physical constraint evaluation item, and the verified high-level response decision is converted into a control adjustment quantity. The control adjustment amount is transmitted to the PID controller, which then performs the adjustment. The behavioral twin model monitors the system state after the PID controller executes, corrects the control adjustment amount based on the system state, and transmits the corrected control adjustment amount to the PID controller, which then re-executes the adjustment until the system returns to a safe operating range.

2. The method as described in claim 1, characterized in that, When constructing the topology diagram of an industrial control system, the following should be included: Based on the process flow diagram, control logic, equipment connection relationships, and communication links of the industrial control system, determine the nodes and their connection relationships; map physical pipeline connections, electrical connections, control loops, communication links, material transfer relationships, and energy transfer relationships as the edges; assign different edge attributes to edges of different connection types.

3. The method as described in claim 1, characterized in that, The behavioral twin model is constructed in the following manner: The topology graph is constructed and a behavior state vector is established for each node. The behavior state vector includes key process parameters of the equipment, control status information, equipment operation status information, network and communication status information, historical change characteristics, and statistical characteristics. The topology is modeled based on a graph neural network, with the behavior state vector as the node input feature. A physical constraint evaluation term is introduced into the loss function of the graph neural network. The physical constraint evaluation term includes mass conservation constraint residual, energy conservation constraint residual, key variable safety boundary penalty term, and process rule violation penalty term. The behavior twin model is then trained to obtain the model.

4. The method as described in claim 1, characterized in that, The behavioral twin model performs physical constraint checks on the state update results of each node during the information transmission process of the graph neural network; if the node state exceeds the safety boundary, the updated node state is corrected according to the physical constraint evaluation terms.

5. The method as described in claim 1, characterized in that, The training process of the reinforcement learning strategy includes: Using the behavioral twin model as an interactive environment, simulate abnormal or attack scenarios; The state space of a reinforcement learning agent consists of the node behavior state, system-level behavior representation, anomaly type, anomaly node location, anomaly propagation path, node risk level, physical constraint evaluation result, and safety margin output by the behavioral twin model. The action space of a reinforcement learning agent consists of the recovery target node, recovery priority, target safety interval, setpoint adjustment direction, setpoint adjustment range, recovery time window, or PID parameter correction suggestions. The reward function of a reinforcement learning agent consists of system stability, recovery time, state deviation, degree of violation of physical constraints, and risk of anomaly propagation. The trained reinforcement learning strategy is obtained through repeated training.

6. The method as described in claim 1, characterized in that, When validating the high-level response decision based on the physical constraint evaluation terms, and converting the validated high-level response decision into a control adjustment variable, the following steps are included: The high-level response decision is input into the behavioral twin model to obtain the predicted behavior states of each node; the predicted behavior states are compared with the target behavior range and the physical constraint evaluation items; when the predicted behavior states exceed the target behavior range or do not meet the physical constraint evaluation items, the setpoint adjustment direction and setpoint adjustment magnitude in the high-level response strategy are corrected; the corrected setpoint adjustment direction and setpoint adjustment magnitude are mapped to the control adjustment amount; The control adjustment quantities include target behavior range, setpoint adjustment suggestions, setpoint change rate, or PID parameter correction quantities.

7. The method as described in claim 1, characterized in that, When the control adjustment is transmitted to the PID controller and the PID controller performs the adjustment, it includes: The PID controller acquires the deviation between the current measured value and the target setpoint in the control adjustment; calculates the control output based on the proportional, integral, and derivative of the deviation; during the calculation of the control output, a limiting mechanism is used to limit the amplitude of the control output, a speed limiting mechanism is used to limit the rate of change of the control output, and an anti-integral saturation mechanism is used to limit the cumulative range of the integral term; and the control output after limiting, speed limiting, and anti-integral saturation processing is output to the actuator of the industrial control system.

8. The method as described in claim 1, characterized in that, The behavioral twin model monitors the system state after the PID controller executes, and when correcting the control adjustment based on the system state, it includes: The behavioral twin model obtains the current behavior state vector of each node after the PID controller is executed, and corrects the control parameters based on the deviation of the current behavior state vector from the target behavior range, the current safety margin, and the anomaly propagation path; it generates the corrected control adjustment amount based on the corrected control parameters and outputs it to the PID controller.

9. The method as described in claim 8, characterized in that, The control parameters include the recovery target, adjustment priority, setpoint change rate, and PID parameter correction.

Images