An adaptive multi-type service scenario industrial network security service providing method

Abstract

The application provides an industrial network security service providing method adaptive to multiple types of business scenarios, comprising: constructing a knowledge graph based on the spatial and temporal correlation between the scene characteristics of production businesses; on the basis of the knowledge graph, combining a relational graph convolution network, a gated recurrent unit, a multi-head attention mechanism and a proximal policy optimization algorithm to perform model training, to obtain a basic network security service supply model; according to a meta-reinforcement learning algorithm, performing meta-training on the basic network security service supply model to obtain a standard network security service supply model; and providing industrial network security services for different types of business scenarios through the standard network security service supply model. The method of the application significantly improves the security and flexibility of the industrial network.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial network security, and particularly relates to a method for providing adaptive industrial network security services for multiple types of business scenarios. BACKGROUND

[0002] Industrial Internet of Things (IIoT) integrates Internet of Things technology into industrial environments, including smart objects, cyber-physical systems (CPSs), information technology, and cloud / edge computing platforms, collectively supporting the operational data needs of modern industry. With the development of 5G communication technology and the pursuit of flexible and intelligent manufacturing, IIoT has transformed into networked manufacturing by bridging the gap between traditional operational technology (OT) and information technology (IT). This paradigm, known as the emerging networked collaborative manufacturing model, has revolutionized the industry, enabling real-time control and coordination among different industrial devices, systems, and applications.

[0003] However, the popularity of interconnected entities not only extends to isolated IT infrastructures, but also encompasses core networks, data centers, and edge devices, significantly expanding the network security attack surface and increasing the attack surface for potential attackers. At the same time, the constantly changing operational conditions and entity roles in flexible and intelligent manufacturing require security mechanisms to be highly adaptable in order to tailor protection measures in the face of evolving manufacturing scenarios. These challenges will be even more prominent in the development of IIoT based on 6G, beyond the capabilities of traditional static security mechanisms. Therefore, there is an urgent need to develop advanced adaptive security provisioning frameworks that can dynamically adjust to the changing threats and manufacturing scenarios in industrial environments.

[0004] In recent years, the concept of security as a service (SECaaS) has gradually emerged as a promising approach, driving innovation in software-defined security (SDSec) by treating security as a manageability problem. This shift has prompted increasing efforts to enhance the adaptability and scalability of security services in IIoT environments. In this context, artificial intelligence and machine learning are widely applied to enhance the security capabilities of IIoT environments. These technologies not only automate the generation and adjustment of security policies, but also quickly identify potential threats through the analysis of large amounts of data. By combining these advanced technologies, more dynamic and adaptive security services can be achieved, effectively addressing the increasingly complex challenges of industrial network security. Deep reinforcement learning (DRL) combines the strengths of deep learning and reinforcement learning, enabling intelligent decision-making through interaction with the environment. In adaptive security service provision, DRL can optimize security policies in real time and dynamically respond to changing network threats.

[0005] Despite these researches significantly advancing the latest developments in adaptive security provisioning, existing solutions still face major challenges:

[0006] Lack of integration with manufacturing processes: Existing security mechanisms often overlook the complex manufacturing processes of IIoT, focusing mainly on isolated data points such as network traffic, and the methods in the prior art fail to provide a comprehensive view of the manufacturing ecosystem, limiting the effectiveness and robustness of security measures.

[0007] Challenge of environmental generalization: The dynamic nature of IIoT environments, characterized by frequent changes in operating conditions and threat environments, requires security solutions that can quickly adapt to new challenges without extensive retraining on new data. Traditional learning-based models, especially deep reinforcement learning, often struggle with generalization and require large amounts of data to adapt to new scenarios, which is impractical in rapidly evolving industrial environments. SUMMARY

[0008] Therefore, the present application provides an industrial network security service providing method adaptive to multiple types of business scenarios to solve the above problems.

[0009] The present application provides an industrial network security service providing method adaptive to multiple types of business scenarios, comprising: constructing a knowledge graph based on the spatial and temporal correlation between the scene characteristics of production business; based on the knowledge graph, combining a relational graph convolution network, a gated recurrent unit, a multi-head attention mechanism and a proximal policy optimization algorithm for model training to obtain a basic network security service supply model; according to a meta-reinforcement learning algorithm, meta-training the basic network security service supply model to obtain a standard network security service supply model; providing industrial network security services for different types of business scenarios through the standard network security service supply model.

[0010] In another implementation manner of the present application, the relational graph convolution network is used to model the static relationship and dynamic relationship in the knowledge graph; the static relationship is represented as:

[0011]

[0012] wherein, represents the prior knowledge feature vector of the entity node d on the l layer, R represents different types of relationships, is a set of neighbor nodes connected by the relationship r, is a trainable parameter specific to each relationship type.

[0013] The dynamic relationship is represented as:

[0014]

[0015] wherein, t represents a time step.

[0016] In another implementation of the present invention, the method further includes: connecting the static relationship and the dynamic relationship, which is expressed as:

[0017]

[0018] In another implementation of the present invention, the gated recurrent unit is used to model temporal dynamics and dependencies, maintaining a state memory of past entity interactions, and the predicted future security state is expressed as:

[0019]

[0020] in, is the hidden state at time step t.

[0021] In another implementation of the present invention, the loss function of the proximal policy optimization algorithm is expressed as:

[0022]

[0023] in, The new strategy πθ and the old strategy πθold in action and status The probability ratio of is the advantage function estimate at time t, ∈ PPO is a typically small hyperparameter that defines the clipping range.

[0024] The method for providing industrial network security services for adaptive multi-type business scenarios of the present invention aims to combine the adaptive security service provision technology of knowledge graphs, deep reinforcement learning and meta-learning, thereby significantly improving the security and flexibility of industrial networks; for adaptive security services, meta-learning can enable the system to quickly learn and adjust protection strategies when facing emerging security threats; the knowledge graph provides structured information support for adaptive security services. By representing entities and their relationships, the knowledge graph can enhance the system's understanding of the relationship between complex data, thereby improving the ability to identify security threats; by combining knowledge graphs, deep reinforcement learning and meta-learning, the system can not only achieve real-time detection and response when facing dynamically changing security threats, but also self-optimize according to environmental changes, thereby meeting the high security demands of the ever-changing modern industrial environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. By reading the detailed description of the embodiments below, the advantages and benefits of the solutions will become clear to those skilled in the art. The drawings are only for the purpose of illustrating preferred embodiments and are not to be considered as limiting the present invention.

[0026] In the drawings:

[0027] Figure 1 A method flowchart for adaptive multi-type service scenario industrial network security service of an embodiment of the present application.

[0028] Figure 2 A multi-type service scenario diagram under next-generation network collaborative manufacturing of an embodiment of the present application.

[0029] Figure 3 A production business material relationship (BOM) and flow diagram of an embodiment of the present application.

[0030] Figure 4 A security resource prediction and optimization deployment algorithm based on meta-reinforcement learning of an embodiment of the present application.

[0031] Figure 5 A space-time knowledge graph driven proximal policy optimization model STKG-PPO of an embodiment of the present application.

[0032] Figure 6 A meta-reinforcement learning method Reptile-CMDPs for quickly adapting to diverse manufacturing scenarios of an embodiment of the present application.

[0033] Figure 7 A comparison diagram of the STKG-PPO algorithm and other reinforcement learning baseline algorithms of an embodiment of the present application.

[0034] Figure 8 A comparison diagram of the Reptile-CMDPs algorithm and other meta-reinforcement learning baseline algorithms of an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the personnel in the art better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art should belong to the scope of protection of the embodiments of the present application.

[0036] Figure 1 A method flowchart for adaptive multi-type service scenario industrial network security service provided by an embodiment of the present application is shown as Figure 1 The present embodiment mainly includes:

[0037] S101, construct a knowledge graph based on the spatial and temporal correlation between the scene features of the production business.

[0038] As shown in the example, Figure 2 a space-time knowledge graph is constructed for the production business scene features. The graph models elements such as devices, network nodes, and data streams as nodes and uses edges to represent the spatial and temporal correlation between them by integrating domain knowledge in the industrial environment.

[0039] Specifically, each node is set as:

[0040] V=v1,v2,…,v n

[0041] where each node v i represents a specific device or data stream, and the weight w ij of the edge represents the degree of mutual influence between the nodes.

[0042] It should be understood that the construction of the knowledge graph not only provides background information for the deep reinforcement learning model, but also enhances the model's adaptability to dynamic environments, so that Figure 3 For example, the entire knowledge graph includes the interaction between various entities (AGV, robotic arm), the dependency between various product materials (product 1 / 2, parts), and the susceptibility of various entities to attack types, etc., thus forming a knowledge graph with multiple relationships.

[0043] S102, based on the knowledge graph, combine the relational graph convolution network, the gated recurrent unit, the multi-head attention mechanism, and the proximal policy optimization algorithm to train the model, and obtain the basic network security service supply model.

[0044] As shown in the example, Figure 4 based on the knowledge graph, a deep reinforcement learning algorithm based on proximal policy optimization (PPO) is used to train the basic network security service supply model, and meta-reinforcement learning technology is combined to achieve dynamic evaluation and optimization of security policies for adaptive multi-type business scenarios.

[0045] As shown in the example, Figure 5 the STKG-PPO model aims to capture the dynamic relationships between entities through the knowledge graph to enhance the allocation of security resources. To address this challenge, relational graph convolution networks (R-GCNs), gated recurrent units (GRUs), multi-head attention mechanisms, and PPO are combined, and each component plays a key role in capturing dynamic and temporal relationships between entities, enhancing feature fusion, and optimizing the policy network.

[0046] S103, according to the meta-reinforcement learning algorithm, meta-training is performed on the basic network security service supply model to obtain a standard network security service supply model.

[0047] S104, providing industrial network security services for different types of business scenarios through the standard network security service supply model.

[0048] The adaptive industrial network security service providing method of the present application aims to combine the adaptive security service providing technology of knowledge graph, deep reinforcement learning and meta-learning, and thus significantly improve the security and flexibility of industrial networks; for adaptive security services, meta-learning can enable the system to quickly learn and adjust the protection strategy when facing new security threats; the knowledge graph provides structured information support for adaptive security services, and through representing entities and their relationships, the knowledge graph can enhance the system's understanding of the relationships between complex data, thereby improving the identification ability of security threats; by combining knowledge graph, deep reinforcement learning and meta-learning, the system can not only achieve real-time detection and response when facing dynamic security threats, but also can self-optimize according to the changes in the environment, thereby meeting the high demand for security in the modern industrial environment.

[0049] In another implementation of the present application, R-GCNs are good at processing multi-relation data in graph structure, and in the knowledge graph, there are both prior knowledge and dynamic relationships, and the relationship graph convolution network is used to model the static relationships and dynamic relationships in the knowledge graph.

[0050] For prior knowledge, static relationships are extracted through R-GCN, and the static relationships are represented as:

[0051]

[0052] Wherein, represents the prior knowledge feature vector of the entity node d on the l layer, R represents different types of relationships, is a set of neighbor nodes connected by the relationship r, is a trainable parameter specific to each relationship type.

[0053] Dynamic relationships are extracted at each time step t, and the dynamic relationships are represented as:

[0054]

[0055] Wherein, t represents the time step.

[0056] These relationships are extracted through the same R-GCN structure and trainable parameters and .

[0057] In another implementation of the present application, further comprising: connecting the static relation and the dynamic relation, denoted as:

[0058]

[0059] This representation captures the evolution of network conditions and entity interactions.

[0060] In another implementation of the present application, the gated recurrent unit is used to model the timing dynamics and dependencies, keeping the state memory of past entity interactions, and the predicted future safety state is denoted as:

[0061]

[0062] where, is the hidden state at time step t.

[0063] Exemplarily, GRUs are used to model timing dynamics and dependencies, which keep the state memory of past entity interactions, which is crucial for predicting future safety states, and GRUs capture the timing effects in the manufacturing process, enhancing the ability of the model to predict future safety states based on past interactions.

[0064] In another implementation of the present application, in order to capture the interaction relationship between entities, a multi-head attention mechanism is further adopted, enabling the model to simultaneously focus on different aspects of entity interactions and context:

[0065]

[0066] where, represent the query, key and value in the attention mechanism, d k is the dimension of the key. The output of the multi-head attention mechanism is then passed into a linear layer to obtain the action probability or value required by PPO, specifically, the linear layer shares the same weights between different entities, ensuring the consistency of feature fusion and the scalability of the network.

[0067] In another implementation of the present application, the loss function of the policy network aims to maximize the expected return while avoiding significant deviation from the old policy, keeping the policy update within a safe range through a clipping mechanism. The loss function of the proximal policy optimization algorithm is denoted as:

[0068]

[0069] where, is the probability ratio of the new policy πθ and the old policy πθold at action and state at time t, is the advantage function estimate at time t, ∈ PPOis a typically small hyperparameter that defines the range of clipping.

[0070] Exemplarily, PPO is a policy gradient method, where the parameters of the policy network are θ, and the parameters of the value network are φ. The policy network is responsible for directly defining the policy, which outputs the distribution of actions (i.e., the allocation of defense resources corresponding to each threat) that the agent can take in a specific state. The value network is responsible for estimating the expected return of a given state The value function is evaluated from a specific state and the current policy.

[0071] The update formula of the policy parameters (θ) is:

[0072]

[0073] where αθ is the learning rate.

[0074] The value network is updated by the mean square error between the predicted value and the actual return.

[0075] By combining the above components, the STKG-PPO model can effectively perform dynamic allocation of security resources and optimize security strategies through the space-time knowledge graph, improving the response capability and defense effect of the system to adapt to changes in security requirements in different industrial application scenarios. Through the STKG-PPO model, the system can deeply understand the current security situation under different operating conditions and provide deeply embedded security deployment solutions.

[0076] In another implementation of the present application, meta-learning, or "learning to learn", focuses on the adaptability of the model in multiple tasks. As Figure 6 shown, for diversified business scenarios, the meta-reinforcement learning algorithm Reptile-CMDPs is used to improve the adaptability of the model between different tasks, and the algorithm process is as follows:

[0077] 1. Task distribution definition: First, define the task distribution, considering the context information of each task to reflect its complexity and diversity. In the context Markov decision process (CMDP), the state transition probability matrix of each task τ is modeled as a form dependent on the specific context .

[0078] 2. Meta-learning goal setting: The goal of meta-learning is to optimize the initial network parameters φ to quickly adapt in a multi-task environment, and the formula is as follows:

[0079]

[0080] where D(φ,Wτ) 2 represents the distance between the initial parameters and the optimal parameters of each task.

[0081] 3. Task training and updating: Perform multiple iterations of training for each task τ, and after each training, adjust the initial parameters by the following update rule:

[0082]

[0083] where, is the projection operation considering task-specific context, ∈ is a hyperparameter to control the update step size.

[0084] 4. Context information extraction: Extract the context information of each task by constructing a state compressor based on the autoencoder, forming a task-specific context vector The steady-state distribution of the framework is approximated by the autoencoder, reducing the computational complexity. The task complexity is estimated by the similarity of the extracted context to guide the adaptation process.

[0085]

[0086] The steady-state distribution of the framework is approximated by the autoencoder, reducing the computational complexity. The task complexity is estimated by the similarity of the extracted context to guide the adaptation process.

[0087] 5. Adaptive parameter adjustment: Estimate the task complexity according to the similarity of the context vector, and use the following formula to adaptively scale the learning parameters:

[0088]

[0089] In this way, more complex tasks will get a larger learning step, while simple tasks will get a relatively small step, to ensure the convergence efficiency of the model on new tasks.

[0090] Through the above steps, the Reptile-CMDPs algorithm can achieve efficient adaptive learning in a variety of manufacturing scenarios, ensuring that the model can quickly obtain generalization experience from multiple tasks and improve overall performance and safety.

[0091] In Figure 7 and Figure 8 , the comparison results of STKG-PPO algorithm and other reinforcement learning baseline algorithms are shown. Experiments show that STKG-PPO performs better in dynamic security resource management in terms of convergence speed and policy stability; in diversified tasks, Reptile-CMDPs algorithm significantly improves the adaptability and efficiency compared with traditional meta-reinforcement learning algorithms.

[0092] To cope with diversified industrial scenarios, the traditional Markov decision process is extended to a CMDP containing context information, reflecting the difficulty and diversity of task training; the Reptile-CMDPs method can make intelligent decisions on potential threats in different industrial environments by realizing rapid adaptation to diversified security tasks, and in each task iteration, the learning strategy is adjusted based on the context to enhance the response capability to new security threats.

[0093] A Monte Carlo simulation method is used to extract a stable state distribution from a state trajectory as task-specific context information to enhance the convergence efficiency of the model and guide the adaptive process of the model.

[0094] The method of the application can be applied in industrial control systems in manufacturing, energy and transportation industries, and can enhance the anti-attack capability of the system by providing customized security services.

[0095] In another aspect of the application, the electronic device comprises a processor, a memory, and a communication bus, a communication interface.

[0096] Wherein:

[0097] The processor, the memory and the communication interface complete mutual communication through the communication bus.

[0098] The communication interface is used for communication with other electronic devices or servers.

[0099] The processor is used for executing a program, and specifically can execute the steps of the adaptive multi-type service scenario industrial network security service providing method of any one of the above embodiments.

[0100] Specifically, the program can include program code including computer operation instructions.

[0101] The processor can be a central processing unit CPU, or an application specific integrated circuit ASIC, or one or more integrated circuits configured to implement one or more embodiments of the application. The one or more processors included in the smart device can be the same type of processor, such as one or more CPUs; or can be different types of processors, such as one or more CPUs and one or more ASICs.

[0102] a memory for storing a program. The memory can comprise a high speed RAM memory and can also include a non-volatile memory, such as at least one disk memory.

[0103] The program can specifically be used to make the processor perform the steps of the industrial network security service providing method for any adaptive multi-type traffic scenario described in the embodiments. The specific implementation of each step in the program can refer to the corresponding description of the steps and units performed by any adaptive multi-type traffic scenario industrial network security service providing method described above, and will not be described here. Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the foregoing method embodiments.

[0104] The exemplary embodiments of the present application also provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to make a computer execute the method of the embodiments of the present application.

[0105] The above-described method according to the embodiments of the present application can be implemented in hardware, firmware, or be implemented as software or computer code that can be stored in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk or an optical disk, or be implemented by computer code originally stored in a remote recording medium or a non-transitory machine-readable medium and downloaded to a local recording medium, so that the method described herein can be processed by such software on a recording medium using a general computer, a special purpose processor or programmable or special purpose hardware (such as ASIC or FPGA). It can be understood that the computer, processor, microprocessor controller or programmable hardware includes a storage component (for example, RAM, ROM, flash memory, etc.) that can store or receive software or computer code, when the software or computer code is accessed and executed by the computer, processor or hardware, the method described herein is implemented. In addition, when a general computer accesses the code for implementing the method shown herein, the execution of the code will convert the general computer into a special purpose computer for executing the method shown herein.

[0106] So far, specific embodiments of the present application have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0107] It should be noted that all directionality indications (such as up, down, left, right, back, etc.) in the embodiments of the present application are only used to explain the relative positional relationship, movement condition, etc. between components in a certain specific posture (as shown in the drawings), and if the specific posture changes, the directionality indications will also change accordingly.

[0108] In the description of the present application, the terms "first", "second" are only used for the convenience of describing different components or names, and cannot be understood as indicating or implying the order relationship, relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can be explicitly or implicitly included at least one of the features.

[0109] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used in the specification of the present application are only for the purpose of describing the specific embodiments and are not intended to limit the present application.

[0110] It should be noted that, although the specific embodiments of the present application are described in detail in combination with the drawings, it should not be understood as limiting the scope of protection of the present application. Various modifications and variations made by those skilled in the art within the scope described in the claims are still within the scope of protection of the present application.

[0111] The examples of the embodiments of the present application are intended to simply illustrate the technical features of the embodiments of the present application, so that those skilled in the art can intuitively understand the technical features of the embodiments of the present application, and are not improper limitations on the embodiments of the present application.

[0112] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for providing industrial network security services that is adaptive to multiple business scenarios, characterized in that: include: Build a knowledge graph based on the spatial and temporal relationships between the scene features of production business; Based on the knowledge graph, we train a model by combining a relational graph convolutional network, a gated recurrent unit, a multi-head attention mechanism, and a proximal policy optimization algorithm to obtain a basic network security service provision model. Performing meta-training on the basic network security service provision model according to a meta-reinforcement learning algorithm to obtain a standard network security service provision model; Providing industrial network security services for different business scenarios through the standard network security service provision model; The relationship graph convolutional network is used to model static and dynamic relationships in the knowledge graph; The static relationship is expressed as: in, represents the prior knowledge feature vector of entity node d on layer l, R represents different types of relationships, is a set of neighbor nodes connected by relationship r, is a trainable parameter specific to each relationship type; The dynamic relationship is expressed as: Where t represents the time step; The method further comprises: Connecting the static relationship and the dynamic relationship is expressed as: The gated recurrent unit is used to model temporal dynamics and dependencies, maintaining state memory of past entity interactions. The predicted future security state is expressed as: in, is the hidden state at time step t.

2. The method according to claim 1, characterized in that The loss function of the proximal policy optimization algorithm is expressed as: in, It's a new strategy With the old strategy In action and status The probability ratio of is the advantage function estimate at time t, is a typically small hyperparameter that defines the clipping range.

Images