Industrial network security service providing method adaptive to multiple types of business scenes

Abstract

The invention provides an industrial network security service providing method adaptive to multiple types of service scenes, which comprises the following steps of: constructing a knowledge graph based on a space and time association relationship between scene characteristics of production services; on the basis of the knowledge graph, performing model training in combination with a relational graph convolutional network, a gating loop unit, a multi-head attention mechanism and a near-end strategy optimization algorithm to obtain a basic network security service supply model; according to a meta reinforcement learning algorithm, performing meta training on the basic network security service supply model to obtain a standard network security service supply model; and providing industrial network security services for different types of business scenes through the standard network security service supply model. According to the method, the safety and the flexibility of the industrial network are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of industrial network security technology, and in particular to a method for providing industrial network security services that is adaptive to multiple types of business scenarios. Background Art

[0002] The Industrial Internet of Things (IIoT) integrates IoT technologies into industrial environments, including smart objects, cyber-physical systems (CPSs), information technology, and cloud / edge computing platforms, to jointly support the operational data needs of modern industry. With the development of 5G communication technology and the pursuit of flexible and intelligent manufacturing, IIoT has transformed into networked manufacturing by bridging the gap between traditional operational technology (OT) and information technology (IT). This paradigm, known as the emerging networked collaborative manufacturing model, has revolutionized the industry, enabling real-time control and coordination of different industrial equipment, systems, and applications.

[0003] However, the prevalence of connected entities extends not only to isolated IT infrastructure, but also to core networks, data centers, and edge devices, significantly expanding the cybersecurity attack surface and increasing the attack surface for potential attackers. At the same time, the ever-changing operating conditions and entity roles in flexible smart manufacturing require security mechanisms to be highly adaptable in order to customize protection measures in evolving manufacturing scenarios. These challenges will be more prominent in the development towards 6G-based IIoT, beyond the capabilities of traditional static security mechanisms. Therefore, there is an urgent need to develop advanced adaptive security provisioning frameworks that can dynamically adjust to cope with changing threats and manufacturing scenarios in industrial environments.

[0004] In recent years, the concept of security as a service (SECaaS) has gradually become a promising approach, driving the innovation of software-defined security (SDSec) to treat security as a manageability problem. This shift has prompted more and more efforts to focus on enhancing the adaptability and scalability of security services in IIoT environments. In this context, artificial intelligence and machine learning are widely used to enhance the security capabilities of IIoT environments. These technologies can not only automate the generation and adjustment of security policies, but also quickly identify potential threats by analyzing large amounts of data. By combining these advanced technologies, more dynamic and adaptive security services can be achieved, effectively addressing the increasingly complex challenges of industrial network security. Deep reinforcement learning (DRL) combines the advantages of deep learning and reinforcement learning to achieve intelligent decision-making through interaction with the environment. In the provision of adaptive security services, DRL can optimize security policies in real time and dynamically respond to changing network threats.

[0005] Although these studies have significantly advanced the state-of-the-art in adaptive security provisioning, existing solutions still face significant challenges:

[0006] Lack of integration with manufacturing processes: Existing security mechanisms often ignore the complex manufacturing processes of IIoT and focus mainly on isolated data points such as network traffic. Methods in existing technologies fail to provide a comprehensive view of the manufacturing ecosystem, limiting the effectiveness and robustness of security measures.

[0007] Challenges of generalization in the environment: The dynamic nature of the IIoT environment, characterized by frequent changes in operating conditions and threat environments, requires security solutions to be able to quickly adapt to new challenges without extensive retraining on new data. Traditional learning-based models (especially deep reinforcement learning) often have difficulties in generalization and require large amounts of data to adapt to new scenarios, which is impractical in a rapidly evolving industrial environment. Summary of the invention

[0008] In view of this, the present invention provides an industrial network security service provision method that is adaptive to multiple types of business scenarios to solve the above problems.

[0009] The present invention provides an industrial network security service provision method for adaptive multi-type business scenarios, including: constructing a knowledge graph based on the spatial and temporal correlation between scenario features of production business; on the basis of the knowledge graph, combining a relational graph convolutional network, a gated recurrent unit, a multi-head attention mechanism and a proximal strategy optimization algorithm to perform model training to obtain a basic network security service provision model; according to a meta-reinforcement learning algorithm, meta-training the basic network security service provision model to obtain a standard network security service provision model; and providing industrial network security services for different types of business scenarios through the standard network security service provision model.

[0010] In another implementation of the present invention, the relationship graph convolution network is used to model static relationships and dynamic relationships in the knowledge graph; the static relationship is expressed as:

[0011]

[0012] in, represents the prior knowledge feature vector of entity node d on layer l, R represents different types of relationships, is a set of neighbor nodes connected by relationship r, is a trainable parameter specific to each relation type.

[0013] The dynamic relationship is expressed as:

[0014]

[0015] Here, t represents the time step.

[0016] In another implementation of the present invention, the method further includes: connecting the static relationship and the dynamic relationship, which is expressed as:

[0017]

[0018] In another implementation of the present invention, the gated recurrent unit is used to model the temporal dynamics and dependencies, maintaining the state memory of past entity interactions, and the predicted future security state is expressed as:

[0019]

[0020] in, is the hidden state at time step t.

[0021] In another implementation of the present invention, the loss function of the proximal strategy optimization algorithm is expressed as:

[0022]

[0023] in, is the new strategy πθ and the old strategy πθold in action and status The probability ratio of is the advantage function estimate at time t, ∈ PPO is a typically small hyperparameter that defines the clipping range.

[0024] The method for providing industrial network security services for adaptive multi-type business scenarios of the present invention aims to combine the adaptive security service providing technology of knowledge graph, deep reinforcement learning and meta-learning, thereby significantly improving the security and flexibility of industrial networks; for adaptive security services, meta-learning can enable the system to quickly learn and adjust protection strategies when facing new security threats; the knowledge graph provides structured information support for adaptive security services, and by representing entities and their relationships, the knowledge graph can enhance the system's understanding of the relationships between complex data, thereby improving the ability to identify security threats; by combining knowledge graph, deep reinforcement learning and meta-learning, the system can not only achieve real-time detection and response when facing dynamically changing security threats, but also self-optimize according to changes in the environment, thereby meeting the high security demands of the ever-changing modern industrial environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. By reading the detailed description of the following implementations, the advantages and benefits of the solutions become clear to those skilled in the art. The drawings are only used to illustrate the preferred implementations and are not to be considered as limiting the present invention.

[0026] In the attached picture:

[0027] Figure 1 A flowchart of a method for providing industrial network security services for adaptive multi-type business scenarios according to an embodiment of the present invention.

[0028] Figure 2 This is a schematic diagram of multiple types of business scenarios under the next-generation network collaborative manufacturing according to an embodiment of the present invention.

[0029] Figure 3 A production business material relationship (BOM) and process diagram of an embodiment of the present invention.

[0030] Figure 4 A schematic diagram of a security resource prediction and optimization deployment algorithm based on meta-reinforcement learning according to an embodiment of the present invention.

[0031] Figure 5 A schematic diagram of a proximal policy optimization model STKG-PPO driven by a spatial-temporal knowledge graph according to an embodiment of the present invention.

[0032] Figure 6 Schematic diagram of Reptile-CMDPs, a meta-reinforcement learning method that can quickly adapt to various manufacturing scenarios according to an embodiment of the present invention.

[0033] Figure 7 A schematic diagram for comparing the STKG-PPO algorithm of an embodiment of the present invention and other reinforcement learning baseline algorithms.

[0034] Figure 8 The Reptile-CMDPs algorithm of one embodiment of the present invention and other meta-reinforcement learning baseline algorithms are comparative schematic diagrams. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be described clearly and in detail below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in the field based on the embodiments in the embodiments of the present invention should fall within the scope of protection of the embodiments of the present invention.

[0036] Figure 1 A flowchart of a method for providing an industrial network security service for adaptive multi-type business scenarios provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, this embodiment mainly includes:

[0037] S101. Construct a knowledge graph based on the spatial and temporal correlation between scene features of production business.

[0038] For example, Figure 2 As shown in the figure, a space-time knowledge graph is constructed based on the characteristics of production business scenarios. This graph integrates domain knowledge in the industrial environment, models elements such as equipment, network nodes, and data flows as nodes, and uses edges to represent the spatial and temporal relationships between them.

[0039] Specifically, set each node to:

[0040] V=v 1 ,v 2 ,…,v n

[0041] Among them, each node v i represents a specific device or data flow, and the edge weight w ij It represents the degree of mutual influence between nodes.

[0042] It should be understood that the construction of the knowledge graph not only provides background information for the deep reinforcement learning model, but also enhances the model's ability to adapt to dynamic environments. Figure 3 For example, the entire knowledge graph includes the interactive relationships between various entities (AGV, robotic arms), the dependencies between various product materials (product 1 / 2, parts), the susceptibility of various entities to attack types, etc., thus forming a knowledge graph with multiple relationships.

[0043] S102. Based on the knowledge graph, the relational graph convolutional network, gated recurrent unit, multi-head attention mechanism and proximal strategy optimization algorithm are combined to perform model training to obtain the basic network security service provision model.

[0044] For example, Figure 4 As shown in the figure, based on the knowledge graph, a deep reinforcement learning algorithm based on proximal policy optimization (PPO) is used to train the basic network security service provision model, and combined with meta-reinforcement learning technology, dynamic evaluation and optimization of security policies for adaptive multi-type business scenarios can be achieved.

[0045] like Figure 5 As shown in the figure, the STKG-PPO model aims to capture the dynamic relationships between entities through the knowledge graph to enhance the allocation of security resources. To address this challenge, the relational graph convolutional network (R-GCNs), gated recurrent units (GRUs), multi-head attention mechanism and PPO are combined. Each component plays a key role in capturing the dynamic and temporal relationships between entities, enhancing feature fusion, and optimizing the policy network.

[0046] S103. Perform meta-training on the basic network security service provision model according to the meta-reinforcement learning algorithm to obtain a standard network security service provision model.

[0047] S104. Provide industrial network security services for different types of business scenarios through a standard network security service provision model.

[0048] The method for providing industrial network security services for adaptive multi-type business scenarios of the present invention aims to combine the adaptive security service providing technology of knowledge graph, deep reinforcement learning and meta-learning, thereby significantly improving the security and flexibility of industrial networks; for adaptive security services, meta-learning can enable the system to quickly learn and adjust protection strategies when facing new security threats; the knowledge graph provides structured information support for adaptive security services, and by representing entities and their relationships, the knowledge graph can enhance the system's understanding of the relationships between complex data, thereby improving the ability to identify security threats; by combining knowledge graph, deep reinforcement learning and meta-learning, the system can not only achieve real-time detection and response when facing dynamically changing security threats, but also self-optimize according to changes in the environment, thereby meeting the high security demands of the ever-changing modern industrial environment.

[0049] In another implementation of the present invention, R-GCNs are good at processing multi-relation data in a graph structure. In the knowledge graph, there are both prior knowledge and dynamic relationships. The relational graph convolutional network is used to model the static and dynamic relationships in the knowledge graph.

[0050] For prior knowledge, static relations are extracted through R-GCN, and the static relations are expressed as:

[0051]

[0052] in, represents the prior knowledge feature vector of entity node d on layer l, R represents different types of relationships, is a set of neighbor nodes connected by relationship r, is a trainable parameter specific to each relation type.

[0053] The dynamic relationship is extracted at each time step t, and the dynamic relationship is expressed as:

[0054]

[0055] Here, t represents the time step.

[0056] These relationships are achieved through the same R-GCN structure and trainable parameters and extract.

[0057] In another implementation of the present invention, the method further includes: connecting the static relationship and the dynamic relationship, which is expressed as:

[0058]

[0059] This representation captures the evolving process of network conditions and entity interactions.

[0060] In another implementation of the present invention, the gated recurrent unit is used to model the temporal dynamics and dependencies, maintaining the state memory of past entity interactions, and the predicted future security state is expressed as:

[0061]

[0062] in, is the hidden state at time step t.

[0063] Exemplarily, GRUs are used to model temporal dynamics and dependencies. They maintain a state memory of past entity interactions, which is crucial for predicting future safety states. GRU captures the timing effects in the manufacturing process and enhances the model's ability to predict future safety states based on past interactions.

[0064] In another implementation of the present invention, in order to capture the interactive relationship between entities, a multi-head attention mechanism is further adopted, so that the model can simultaneously focus on different aspects of entity interaction and context:

[0065]

[0066] in, represents the query, key, and value in the attention mechanism, d k is the dimension of the key. The output of the multi-head attention mechanism is then passed into the linear layer to obtain the action probability or value required by PPO. Specifically, the linear layer shares the same weights between different entities, ensuring the consistency of feature fusion and the scalability of the network.

[0067] In another implementation of the present invention, the loss function of the policy network aims to maximize the expected return while avoiding significant deviation from the old policy, and keeping the policy update within a safe range through a shearing mechanism. The loss function of the proximal policy optimization algorithm is expressed as:

[0068]

[0069] in, is the new strategy πθ and the old strategy πθold in action and status The probability ratio of is the advantage function estimate at time t, ∈ PPOis a typically small hyperparameter that defines the clipping range.

[0070] For example, PPO is a policy gradient method, where the policy network is parameterized by θ and the value network is parameterized by φ. The policy network is responsible for directly defining the policy, and it outputs the distribution of actions that the agent can take in a specific state (i.e., the allocation of defense resources corresponding to each threat). The value network is responsible for estimating the given state Expected Return Evaluates the value function starting from a specific state and the current policy.

[0071] The update formula of the strategy parameter (\theta) is:

[0072]

[0073] Among them, αθ is the learning rate.

[0074] The value network is updated by the mean squared error between the predicted value and the actual return – .

[0075] By combining the above components, the STKG-PPO model can effectively dynamically allocate security resources and optimize security policies through the space-time knowledge graph, improve the system's responsiveness and defense effect, and adapt to the changes in security requirements in different industrial application scenarios. Through the STKG-PPO model, the system can deeply understand the current security situation under different operating conditions and provide a deeply embedded security deployment solution.

[0076] In another implementation of the present invention, meta-learning, or “learning how to learn”, focuses on the adaptability of the model in multiple tasks. Figure 6 As shown in the figure, for diversified business scenarios, the meta-reinforcement learning algorithm Reptile-CMDPs is used to improve the adaptability of the model between different tasks. The algorithm process is as follows:

[0077] 1. Task distribution definition: First, define the task distribution, taking into account the contextual information of each task to reflect its complexity and diversity. In the contextual Markov decision process (CMDP), the state transition probability matrix of each task τ is modeled as dependent on the specific context in the form of.

[0078] 2. Meta-learning goal setting: The meta-learning goal is to optimize the initial network parameter φ so that it can adapt quickly in a multi-task environment. The formula is as follows:

[0079]

[0080] Where D(φ,Wτ) 2 Represents the distance between the initial parameters and the optimal parameters for each task.

[0081] 3. Task training and updating: Perform multiple iterative training on each task τ. After each training, adjust the initial parameters using the following update rules:

[0082]

[0083] in, is a projection operation that considers the task-specific context, and ∈ is a hyperparameter that controls the update step size.

[0084] 4. Contextual information extraction: By building a state compression mechanism based on an autoencoder, the contextual information of each task is extracted to form a task-specific context vector The steady-state distribution is approximated by minimizing the following loss function:

[0085]

[0086] The steady-state distribution of the framework is approximated by an autoencoder to reduce the computational complexity, and the task complexity is estimated by computing the similarity of the extracted context to guide the adaptation process.

[0087] 5. Adaptive parameter adjustment: Based on the similarity of the context vectors, the task complexity is evaluated and the learning parameters are adaptively scaled using the following formula:

[0088]

[0089] In this way, more complex tasks will get larger learning steps, while simple tasks will get relatively small steps to ensure the convergence efficiency of the model on new tasks.

[0090] Through the above steps, the Reptile-CMDPs algorithm can achieve efficient adaptive learning in various manufacturing scenarios, ensuring that the model can quickly obtain generalized experience from multiple tasks and improve overall performance and safety.

[0091] exist Figure 7 and Figure 8 The comparison results of STKG-PPO algorithm and other reinforcement learning baseline algorithms are shown in the paper. The experiments show that STKG-PPO shows higher convergence speed and better policy stability in dynamic security resource management; under diversified tasks, Reptile-CMDPs algorithm significantly improves adaptability and efficiency compared with traditional meta-reinforcement learning algorithms.

[0092] In order to cope with diverse industrial scenarios, the traditional Markov decision process is extended to a CMDP that contains contextual information to reflect the difficulty and diversity of task training; the Reptile-CMDPs method can make intelligent decisions on potential threats in different industrial environments by achieving rapid adaptation to diverse security tasks. In each task iteration, the learning strategy is adjusted based on the context enhancement strategy to improve the response capability to emerging security threats.

[0093] Using the Monte Carlo simulation method, the stable state distribution is extracted from the state trajectory as task-specific contextual information to enhance the convergence efficiency of the model and guide the model's adaptive process. The Reptile-CMDPs meta-reinforcement learning algorithm is designed to meta-train the model so that it can learn generalized experience from multiple tasks.

[0094] The method of the present invention can be applied in industrial control systems in industries such as manufacturing, energy, and transportation, and can enhance the system's anti-attack capability by providing customized security services.

[0095] In another aspect of the present invention, an electronic device includes: a processor, a memory, a communication bus, and a communication interface.

[0096] in:

[0097] The processor, memory and communication interface communicate with each other through a communication bus.

[0098] Communication interface, used to communicate with other electronic devices or servers.

[0099] The processor is used to execute the program, and specifically can execute the steps of any one of the methods for providing industrial network security services for adaptive multi-type business scenarios in the above-mentioned embodiments.

[0100] Specifically, the program may include program codes including computer operation instructions.

[0101] The processor may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0102] The memory is used to store programs. The memory may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0103] The program can be specifically used to enable the processor to execute the steps of any method for providing industrial network security services for adaptive multi-type business scenarios described in the embodiment. The specific implementation of each step in the program can refer to the corresponding descriptions in the steps and units executed by any method for providing industrial network security services for adaptive multi-type business scenarios in the above steps, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the aforementioned method embodiment.

[0104] The exemplary embodiments of the present application also provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the methods of the various embodiments of the present application.

[0105] The above-described method according to an embodiment of the present invention may be implemented in hardware, firmware, or as software or computer code that may be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code that is originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded over a network and will be stored in a local recording medium, so that the method described herein may be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that a computer, processor, microprocessor controller, or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, processor, or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown herein.

[0106] Thus far, specific embodiments of the present invention have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired results. Additionally, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing may be advantageous.

[0107] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0108] In the description of the present invention, the terms "first" and "second" are only used to facilitate the description of different components or names, and cannot be understood as indicating or implying a sequential relationship, relative importance, or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features.

[0109] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0110] It should be noted that although the specific embodiments of the present invention are described in detail in conjunction with the accompanying drawings, it should not be understood as limiting the scope of protection of the present invention. Within the scope described in the claims, various modifications and variations that can be made by those skilled in the art without creative work still belong to the scope of protection of the present invention.

[0111] The examples of the embodiments of the present invention are intended to concisely illustrate the technical features of the embodiments of the present invention so that those skilled in the art can intuitively understand the technical features of the embodiments of the present invention, and are not intended to be improper limitations of the embodiments of the present invention.

[0112] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for providing industrial network security services that is adaptive to multiple types of business scenarios, characterized in that: include: Construct a knowledge graph based on the spatial and temporal relationships between the scene features of the production business; On the basis of the above-mentioned knowledge graph, the model training is carried out by combining the relational graph convolutional network, gated recurrent unit, multi-head attention mechanism and proximal strategy optimization algorithm to obtain the basic network security service provision model; Meta-training the basic network security service provision model according to a meta-reinforcement learning algorithm to obtain a standard network security service provision model; Industrial network security services are provided to different types of business scenarios through the standard network security service provision model.

2. The method according to claim 1, characterized in that The relationship graph convolution network is used to model static relationships and dynamic relationships in the knowledge graph; The static relationship is expressed as: in, represents the prior knowledge feature vector of entity node d on layer l, R represents different types of relationships, is a set of neighbor nodes connected by relationship r, is a trainable parameter specific to each relation type; The dynamic relationship is expressed as: Here, t represents the time step.

3. The method according to claim 2, characterized in that Also includes: Connecting the static relationship and the dynamic relationship is expressed as:

4. The method according to claim 3, characterized in that The gated recurrent unit is used to model the temporal dynamics and dependencies, maintaining the state memory of past entity interactions, and the predicted future security state is expressed as: in, is the hidden state at time step t.

5. The method according to claim 1, characterized in that The loss function of the proximal strategy optimization algorithm is expressed as: in, is the new strategy πθ and the old strategy πθold in action and status The probability ratio of is the advantage function estimate at time t, ∈ PPO is a typically small hyperparameter that defines the clipping range.

Images