An IIoT intrusion detection system and method based on neural architecture evolution federated learning

Abstract

The present invention discloses an industrial Internet of Things intrusion detection system and method based on neural architecture evolution federated learning. The server encodes the federated learning neural network architecture combination based on 5 neural network basic modules and sends it to the clients participating in the federated learning. Each client performs local training on the data of the local IIoT device. The server aggregates the intermediate parameters of each client model and performs weighted average fitness. Through population evolution operations, a federated learning model based on the optimal neural architecture is obtained, and it is deployed in the IIoT online intrusion detection system, thereby realizing efficient and accurate online intrusion detection of IIoT. The present invention can not only realize the optimized design and automatic generation of the federated learning model for IIoT intrusion detection, but also obtain a lightweight federated learning model that is easy to deploy online. While protecting the privacy and security of IIoT devices, it also improves the performance indicators such as precision, recall rate, and F1 score of IIoT intrusion detection.

Description

Technical Field

[0001] The present invention relates to intelligent intrusion detection technology in the field of information security of industrial Internet of Things systems, and in particular to an Industrial Internet of Things (IIoT) intrusion detection system and method based on neural architecture evolution federated learning. Background Art

[0002] In recent years, with the rapid development of 5G communication technology, smart sensors and processors and other embedded devices, the Internet of Things technology has been widely used in important fields such as industrial manufacturing, smart grids, natural gas transmission pipeline systems, and intelligent transportation systems. As a large-scale, geographically dispersed, complex and heterogeneous Internet of Things applied in industrial environments, the Industrial Internet of Things system integrates intelligent network and computing technologies, such as 5G communication technology, software-defined network, network function virtualization, cloud computing and artificial intelligence. These emerging technologies enable the Industrial Internet of Things system to operate more accurately and efficiently. However, due to the large-scale access of various IoT devices, the Industrial Internet of Things faces increasingly serious security threats. Therefore, the issue of protecting the Industrial Internet of Things system from cyber attacks has become increasingly important and indispensable in its design. Security protection measures such as industrial firewalls and intrusion detection systems have been widely used. However, as security threats become more and more complex, it is urgent to develop high-performance and efficient intrusion detection technology, which not only requires it to be able to detect various malicious attacks in a timely and accurate manner, but also requires the intrusion detection model to be as lightweight as possible so that it can be deployed in IoT devices with limited computing power in industrial environments.

[0003] The massive data generated by IIoT devices provides a good foundation for the application of deep learning in the field of IIoT. Therefore, deep learning has been widely used in IIoT in recent years. However, if industrial big data is collected and processed in the cloud of IIoT, it will not only lead to unbearable transmission overhead, but also violate data privacy regulations. Therefore, deep learning technology must transfer model training from the cloud to the edge. With the development of federated learning, federated learning not only ensures the efficient training of deep learning models, but also provides solutions for the data privacy security of these IIoT devices. Therefore, more and more scholars, from the perspective of deep learning, use convolutional neural networks, recurrent neural networks, etc. as the basis, and use the massive data generated by IIoT devices to conduct federated training, trying to build a high-performance IIoT system intrusion detection solution. However, most of the existing deep neural networks and federated learning models designed for IIoT system and equipment intrusion detection application scenarios are manually designed by designers and engineers, and rely heavily on experience and trial and error in a specific scenario. Not only does it take a lot of time and effort, but the final model architecture and parameter schemes designed are often difficult to balance in terms of computational cost and intrusion detection performance.

[0004] On the other hand, automatic deep learning with neural architecture search and hyperparameter optimization as the core has become one of the research hotspots in the field of artificial intelligence, but the main research results are still concentrated in the fields of computer vision and natural language processing, and there is little application research in the field of industrial Internet of Things intrusion detection. Therefore, in order to adapt to the low power consumption and low computing power of industrial Internet of Things devices, new industrial Internet of Things intrusion detection technology based on automatic federated learning is in urgent need of research and development. Summary of the invention

[0005] The purpose of the present invention is to provide an IIoT intrusion detection system and method based on neural architecture evolution federated learning in view of the deficiencies in the prior art.

[0006] The objective of the present invention is achieved through the following technical solutions: an IIoT intrusion detection system based on neural architecture evolution federated learning, which includes an offline optimization module of a federated learning model based on neural architecture evolution and an industrial Internet of Things online intrusion detection module based on the optimal federated learning model.

[0007] The offline optimization module of the federated learning model based on neural architecture evolution, that is, the server will perform individual encoding based on the combination of the federated learning neural network architecture of 5 basic neural network modules, randomly generate an initialization population, and send the code to each participating client; each client generates an initial local model, and each client performs initial local training for its own industrial Internet of Things local communication traffic historical data, and uploads its own model intermediate parameters to the server; the server aggregates the intermediate parameters, and then sends them to the local model of each client; each client then performs the next round of local training, repeating the process of uploading the intermediate parameters of each client model to the server, The server aggregates the intermediate parameters and sends them to each client to generate a local model. Each client then performs a new round of local training until each client reaches the specified maximum number of local training rounds. Then, each client performs local testing and uploads the evaluation indicators obtained to the server. The server performs fitness weighted average calculations on the evaluation indicators received from each client, and performs evolutionary operations such as crossover, mutation, and environmental selection on the individuals in the population. The above process is repeated to perform a specific round of neural network architecture search, and finally the neural network architecture with the best performance in historical data is obtained, thereby constructing a federated learning model based on the optimal neural network architecture.

[0008] The industrial Internet of Things online intrusion detection module based on the optimal federated learning model, that is, the server deploys the federated learning model based on the optimal neural network architecture online, sends the optimal federated learning model to each client, each client generates a local initial model, each client performs initial local training for its own industrial Internet of Things local communication traffic history data, and uploads its own model intermediate parameters to the server; after the server aggregates the model intermediate parameters, it sends the aggregated model intermediate parameters to the local model of each client, and then performs the next round of local training, repeating the process of uploading the intermediate parameters of each client model to the server, the server aggregates the intermediate parameters and sends them to each client to generate the local model, and each client performs a new round of local training, until each client reaches the specified maximum local training round, and each client generates a local final online test model; each client uses the local final online test model generated by itself for the real-time data of the industrial Internet of Things local communication process collected by itself to perform online intrusion monitoring, if abnormal data is detected, it will issue an abnormal alarm, and evaluate the performance of the online test model of each client, upload the performance index of each client to the server, and the server performs weighted average of the performance index of each client to obtain the final average performance index.

[0009] In the above two modules, each client performs data sniffing, data collection, data analysis and data normalization operations on the local industrial Internet of Things communication traffic historical data and real-time data in turn, that is, firstly, data sniffing is performed on the industrial Internet of Things device communication traffic data, and the communication traffic in the network is stored in the system in the form of messages by putting the network card in promiscuous mode; after receiving the communication message, the data collection module summarizes it into the form of network flow and generates corresponding features; the data analysis summarizes all the category features therein and generates a feature mapping table, in which the category features and the numerical features in the feature table correspond one to one; according to the feature table, all the category features in the data are converted into numerical features, thereby obtaining the source data set, marking it as X, and performing data normalization processing according to formula (1), and marking the data set obtained after normalization processing as X o , that is, the local historical data set required for offline optimization, and then X o The sequence numbers are randomized and assigned to the training set X at a ratio of 4:1. ox With the test set X oc middle;

[0010]

[0011] Among them, X oi Represents the dataset X o The feature of the i-th dimension, X i Represents the features of the i-th dimension of the source dataset X, X imax With X imin Respectively represent X oi The minimum and maximum values ​​of the data set X o The maximum value of the dimension;

[0012] An industrial Internet of Things intrusion detection method based on neural architecture evolution federated learning includes the following steps:

[0013] (1) Setting the initialization parameter values ​​of the industrial Internet of Things intrusion detection method based on neural architecture evolution federated learning, the parameter values ​​include the population size N, the number of iterations of optimization of the neural network model architecture and hyperparameters G, the number of rounds of offline training TP1 of each client neural network model, the number of rounds of federated training TP2 of each individual, the number of rounds of federated training TP3 of the optimal neural network model, and the crossover rate p c , mutation rate p m , the number of clients participating in federated learning is n;

[0014] (2) Five lightweight convolutional neural network basic modules are used as candidate component modules for optimizing the federated learning neural network architecture, and N individuals are randomly generated as the initial population S, where S = {s1, s2, …, s j ,…,s N}, where s1, s2, s j ,s N They represent the first individual, the second individual, the jth individual and the Nth individual in the population, 1≤j≤N, and each individual s j The encoding form is s j = {B1, B2, ... B k ,…,B 10}, 1≤k≤10, where B1, B2, B k and B 10 Respectively represent individual s j The first bit code, the second bit code, the kth bit code and the tenth bit code of each B k The encoding of is a randomly generated integer in the range of 1 to 5, which represents the type of the selected lightweight convolutional neural network basic module, and its definition is shown in formula (2):

[0015] B k =randint(1,5),1≤k≤10 (2)

[0016] Among them, Randint represents the random integer generation function, and Randint(1,5) represents randomly generating an integer from 1 to 5; the types of lightweight convolutional neural network basic modules selected by integers 1, 2, 3, 4, and 5 are basic module 1 (marked as block1), basic module 2 (marked as block2), basic module 3 (marked as block3), basic module 4 (marked as block4), and basic module 5 (marked as block5); each type of basic module is divided into normal block and dimension reduction block. The spatial dimension of the output of the normal block is consistent with the spatial dimension of the input, and the spatial dimension of the output of the dimension reduction block is one quarter of the input;

[0017] The normal block of basic module 1 (block1) does not perform any operation on the input, and the input is the output. When the normal block of module 1 is selected, the layer removal operation can be implemented, so that the number of layers of the model can be variable, that is, the actual number of layers of the model ranges from 2 to 10. The dimension reduction block of block1 consists of a conventional convolution block with a convolution kernel of 3×3. The dimension reduction effect is achieved by setting the stride of each convolution to 2. The Relu activation function is used to activate the output after convolution, so the spatial dimension of the output of the dimension reduction block is one quarter of the spatial dimension of the input;

[0018] Basic module 2 (block2) consists of a depthwise separable convolution block with a 3×3 convolution kernel and a convolution block with a 1×1 convolution kernel. The combination of a 3×3 depthwise separable convolution and a 1×1 convolution is equivalent to a 3×3 regular convolution, but the amount of calculation is much less than that of a regular convolution. Therefore, this combination can make the model architecture more lightweight. When the stride of each movement of the 3×3 depthwise separable convolution is set to 1, it means that block2 is a normal block, and the spatial dimension of the output is consistent with the spatial dimension of the input. When the stride is set to 2, it means that block2 is a dimensionality reduction block, and the spatial dimension of the output is one quarter of the spatial dimension of the input.

[0019] Basic module 3 (block3) consists of 1 1×1 convolution block, 1 3×3 depthwise separable convolution block and 1 1×1 convolution block. The function of the first 1×1 convolution block is to increase the dimension, which is used to increase the number of channels. The function of the second 1×1 convolution block is to reduce the dimension, which is used to reduce the number of channels. Therefore, the number of channels increases first and then decreases, so that the number of output channels can be consistent with the number of input channels. In addition, the second 1×1 convolution block uses a linear activation function to activate the output result of the 1×1 convolution. When the Stride of each movement step of the 3×3 depthwise separable convolution is set to 1, it means that block3 is a normal block, and the spatial dimensions of the output and input are consistent. When the Stride is set to 2, it means that block3 is a dimensionality reduction block, and the spatial dimension of the output is one quarter of the input.

[0020] Basic module 4 (block4) consists of 1 1×1 convolution block and 1 3×3 depthwise separable convolution, which is the opposite order of module 2. The combination of 1×1 convolution and 3×3 depthwise separable convolution is equivalent to 1 3×3 regular convolution, but the amount of calculation is much smaller than that of regular convolution. When the stride of each movement of the 3×3 depthwise separable convolution is set to 1, it indicates a normal block of block4, and the spatial dimension of the output is consistent with the spatial dimension of the input. When the stride is set to 2, it indicates a dimensionality reduction block of block4, and the spatial dimension of the output is one quarter of the input.

[0021] Basic module 5 (block5) is divided into a normal block and a dimensionality reduction block. In the normal block, the input is first divided into two parts according to the number of channels, that is, the input with a spatial dimension of C×W×W is randomly divided into 2 (C / 2)×W×W sub-vectors, C represents the number of input channels, W represents the data dimension of the input, one of which is not operated, and the other is sequentially operated by a 1×1 convolution block, a 3×3 depth-separable convolution block and a 1×1 convolution block. The output is then concatenated with the sub-vector that is not operated by channel. At this time, the spatial dimension of the output is consistent with the spatial dimension of the input, and then the channels are randomly rearranged to obtain the final output. In the dimension reduction block of block5, two operations are performed simultaneously on its input. Operation 1 consists of a 3×3 depth-wise separable convolution block and a 1×1 convolution block, and its structure is similar to block2. Operation 2 consists of a 1×1 convolution block, a 3×3 depth-wise separable convolution block, and a 1×1 convolution block, and its structure is similar to block3. The convolution movement stride of the 3×3 depth-wise separable convolution in both operations is set to 2 to achieve the dimension reduction effect. Finally, the output results of the two operations are spliced ​​by channel, and then the channels are randomly rearranged to obtain the final output. At this time, the spatial dimension of the output is one-fourth of the input.

[0022] An important module for constructing the neural network model architecture is the combination module, which is marked as a BLOCK module; the specific implementation process of the BLOCK module is: every 4 normal blocks of the basic modules are followed by a dimension reduction block of the basic module, and the output of the first normal block is spliced ​​with the output of the fourth normal block, so that the number of channels is doubled, and then used as the input of the dimension reduction block, and the dimension is halved after the dimension reduction block operation, so that the combination of 4 normal blocks and 1 dimension reduction block constitutes a combination module, which is marked as a BLOCK module;

[0023] The overall architecture of the neural network model participating in federated learning consists of a 3×3 regular convolution block, two of the above-mentioned BLOCK modules, an average pooling layer, and a fully connected layer. The 3×3 convolution block is used to increase the number of channels of the initial input to the size required for the BLOCK module operation.

[0024] (3) For each individual s in population S j (1≤j≤N) for fitness evaluation. The specific evaluation process is as follows: The federated learning server will evaluate the fitness of each individual s in the population S. j The code of (1≤j≤N) is sent to the participating n clients to generate a local initial model. Each client uses its own local historical data set X of the industrial Internet of Things. ox, respectively perform the initial local training of TP1 round, upload the intermediate parameters of each model to the server, the server aggregates the intermediate parameters, and sends them to their local models after aggregation, and then perform the next round of local training of TP1 round, repeat the process of clients uploading their own model intermediate parameters, the server aggregates the model intermediate parameters, sends the aggregated intermediate parameters to the local models of each client, and the client performs the next round of local training, until the maximum local training round TP2 specified by the client is reached; then, each client uses its own test set X oc Perform local testing and calculate the F1 score according to formula (3) as the fitness of the individual corresponding to the model, which is marked as f l , 1≤l≤n, upload the fitness obtained by each client to the server, and the server receives the fitness f uploaded by each client according to formula (4) l Perform weighted average operation to obtain individual s j The final fitness f(s) of (1≤j≤N) j );

[0025]

[0026]

[0027] Among them, TP represents the number of normal IIoT data samples correctly predicted as normal classification, TN represents the number of abnormal IIoT data samples correctly predicted as abnormal classification, FP represents the number of abnormal IIoT data samples incorrectly predicted as normal classification, and FN represents the number of normal IIoT data samples incorrectly predicted as abnormal classification.

[0028] The specific process of the server aggregating the model intermediate parameters uploaded by n clients is as follows: n clients participating in federated learning aggregate the model intermediate parameters w trained locally l (1≤l≤n) is uploaded to the server. After receiving the parameters uploaded by all clients, the server calculates the model intermediate parameters w of all clients according to formula (5). l (1≤l≤n) to obtain the aggregated parameter w(s j )(1≤l≤n), then w(s j ) is re-issued to each client;

[0029]

[0030] (4) Perform crossover and mutation operations on population S to generate offspring population S cThe specific implementation steps of the crossover operation are: randomly select 5 individuals from S to compare their fitness, select the two individuals with the highest fitness as parents, denoted as p1 and p2 respectively; generate a random number q1=Random(0,1), where Random(0,1) represents a uniform random number generated between 0 and 1; if q1<crossover rate p c , then generate a random integer P c =Randint(2,10) as the crossover site, P c Indicates that the crossover site is located in the Pth parent individual p1, p2 c Code, perform single-point crossover on p1 and p2, that is, the Pth c The coding fields of the first to the tenth coding are exchanged, while keeping the other coding fields of p1 and p2 unchanged, generating two offspring individuals c1 and c2; if q1≥p c , then c1=p1, c2=p2. The specific implementation steps of the mutation operation are: for each offspring individual c, generate a random number q2=Random(0,1), if q2<mutation rate p m , then generate a random integer P m =Randint(1,10) as the variation site, P m Indicates that the variant site is located in the Pth individual in the offspring c. m A code is randomly generated from 1 to 5, which is different from the Pth individual in the offspring c. m An integer that encodes its own value, and uses this integer to replace the Pth m The original encoding value is kept unchanged, thus generating a mutant individual c M ; If q2 ≥ p m , then c M = c, M Join S c Repeat the above crossover and mutation operations until S c The number of individuals in is N;

[0031] (5) Generate the next generation population S through environmental selection operation N The specific implementation steps of the environmental selection operation are: c Merge with the current population S, S = S ∪ S c ; For the merged population S, use the five-element competition selection as the environmental selection method, randomly select 5 individuals from S, evaluate the fitness of the individuals according to step (3), and select the individual with the highest fitness to enter the next generation population S N ; Repeat this operation until S N The population size is N;

[0032] (6) Unconditionally accept S = S N ;

[0033] (7) Repeat steps (4) to (6) until the set number of iterative optimization times G is met;

[0034] (8) Select the individual with the highest fitness in S and record it as S best ;

[0035] (9) The optimal fitness individual S finally obtained in the federated offline optimization training module best The encoding is sent to n clients to build a local initial model. Each client performs initial local training of TP1 round for its own local historical data set of industrial IoT, uploads its own model intermediate parameters to the server, aggregates the intermediate parameters, and sends them to its own local model after aggregation. Then, the next round of local training of TP1 round is carried out. The process of uploading the intermediate parameters of the client model, aggregating the intermediate parameters of the server, sending the intermediate parameters to the client to generate the local model, and the client performs a new round of local training of TP1 round is repeated until the maximum local training round TP3 specified by the client is reached; each client generates a local final online test model;

[0036] (10) The n clients participating in the federated test collect real-time data X of the local communication process of the industrial Internet of Things t , use the local final online test model generated by each client to perform online intrusion monitoring. If abnormal data is detected, an abnormal alarm is issued, and the precision (Pre), recall (Rec) and F1 score (F1) of each client online test model are calculated according to formula (6), formula (7) and formula (3), respectively, and are marked as Pre l 、Rec l and F 1l , 1≤l≤n; the Pre l ,Rec l and F 1l Upload to the server;

[0037]

[0038]

[0039] Among them, TP represents the number of normal IIoT data samples correctly predicted as normal classification, TN represents the number of abnormal IIoT data samples correctly predicted as abnormal classification, FP represents the number of abnormal IIoT data samples incorrectly predicted as normal classification, and FN represents the number of normal IIoT data samples incorrectly predicted as abnormal classification.

[0040] (11) The server performs weighted average of the performance indicators uploaded by each client according to formulas (8) to (10) to obtain the final average precision, average recall and average F1 score.

[0041]

[0042]

[0043]

[0044] The beneficial effects of the present invention are as follows: compared with the prior art, the present invention can not only realize the automatic generation and optimization design of the federated learning model for industrial Internet of Things intrusion detection, but also improve the intelligent design level and design efficiency of the industrial Internet of Things intrusion detection system; the use of neural architecture evolution technology can optimize and search offline for a lightweight federated learning model based on the optimal combination architecture of lightweight basic modules, which is more suitable for online deployment in performance-constrained industrial Internet of Things devices; while protecting the privacy and security of industrial Internet devices, it also improves the performance indicators of IIoT intrusion detection such as precision, recall rate, and F1 score. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of the principle of IIoT intrusion detection system and method based on neural architecture evolution federated learning;

[0046] Figure 2 The following is a structural diagram of the five types of lightweight convolutional neural network basic modules used. (a) shows the structural diagram of basic module 1 (block1). The left sub-image shows the normal block. N is the input of the normal block in block1, output1 N is the output of the normal block in block1, input1 N and output1 N The spatial dimensions are consistent, NULL means no operation is performed on the input. N =input1 N , so that the number of layers of the model can be variable, that is, the number of layers of the model composed of optional modules ranges from 2 to 10; the right sub-figure represents the dimension reduction block, input1 R is the input of the dimension reduction block in block1, output1 R is the output of the dimension reduction block in block1, output1 R The spatial dimension is input1 R One quarter of input1 RThe spatial dimension of is C×W×W, C represents the number of input channels, W represents the input data dimension, then output1 RThe spatial dimension is C×(W / 2)×(W / 2), where Conv 3×3 represents a conventional convolution with a convolution kernel of 3×3, Stride represents the step size of each convolution movement, and the dimension reduction effect is achieved by setting the step size of each convolution movement to Stride=2. Relu represents the use of the Relu activation function to activate the output after convolution; (b) The figure shows the composition structure diagram of the basic module 2 (block2), input2 is the input of block2, output2 is the output of block2, block2 consists of 1 Dwise Conv 3×3 convolution block and 1 Conv 1×1 convolution block, where Dwise Conv 3×3 represents the use of a depthwise separable convolution with a convolution kernel of 3×3, Bn (Batch Normalization) represents batch normalization of the output after convolution, Relu6 represents the use of the rectified linear unit (RectifiedLinearUnit) activation function with a maximum output limit of 6, Conv 1×1 means that a regular convolution with a convolution kernel of 1×1 is used, also known as pointwise convolution; s means the value range of Stride. When s is set to 1, it means that block2 is a normal block, and the spatial dimensions of input2 and output2 are consistent; when s is set to 2, it means that block2 is a dimensionality reduction block, and the spatial dimension of output2 is one quarter of input2; (c) Figure shows the composition structure of basic module 3 (block3), input3 is the input of block3, output3 is the output of block3, block3 consists of 1 Conv 1×1 convolution block, 1 Dwise Conv 3×3 convolution block and 1 Conv 1×1 convolution block, Linear means that the second Conv 1×1 convolution block uses a linear activation function to activate the output after convolution.When Stride is set to 1, it means that block3 is a normal block, and the spatial dimensions of input3 and output3 are consistent; when Stride is set to 2, it means that block3 is a dimensionality reduction block, and the spatial dimension of output3 is one quarter of input3; (d) The figure shows the composition structure diagram of basic module 4 (block4), input4 is the input of block4, output4 is the output of block4, block4 consists of 1 Conv 1×1 convolution block and 1 Dwise Conv3×3 convolution block, which is the opposite order of block2; when Stride is set to 1, it means that block4 is a normal block, and the spatial dimensions of input4 and output4 are consistent; when Stride is set to 2, it means that block4 is a dimensionality reduction block, and the spatial dimension of output4 is one quarter of input4; (e) The figure shows the composition structure diagram of basic module 5 (block5), and the left sub-figure represents a normal block, input5. N is the input of the normal block in block5, output5 N is the output of the normal block in block5, input5 N and output5 N The spatial dimensions of are consistent, and the right sub-graph represents the dimension reduction block, input5 R is the input of the dimension reduction block in block5, output5 R is the output of the dimension reduction block in block5, output5 R The spatial dimension is input5 R One quarter of the input 5 N Divide into two parts according to the number of channels, that is, input5 with spatial dimensions of C×W×W N The vector is randomly divided into two sub-vectors with spatial dimensions of (C / 2)×W×W. One of them is not operated, and the other is operated in sequence according to Conv 1×1, Dwise Conv 3×3 and Conv 1×1 as shown in the right branch of the left sub-figure. The spatial dimension after the operation remains unchanged, that is, it is still (C / 2)×W×W. Then, the two (C / 2)×W×W sub-modules are concatenated by channel, that is, they are concatenated into C×W×W by channel. Finally, the channel is randomly rearranged (ChannelShuffle), that is, the channels are randomly rearranged, and the final output output5 is obtained. N The right sub-graph is the dimension reduction block of block5. The structure of the left branch is similar to that of block2, and the structure of the right branch is similar to that of block3, but the strides of both branches are set to 2.R After these two branch operations, the output results of the two branch operations are concat-operated, that is, spliced ​​by channel, and then the Channel Shuffle operation is performed to obtain the final output output5 R .

[0047] Figure 3 It is a schematic diagram of the composition structure of the combined module BLOCK, which is composed of four normal blocks and one dimension reduction block in sequence, where C represents the number of input channels and W represents the input data dimension;

[0048] Figure 4 This is an example diagram of the overall structure of a neural network model participating in federated learning, which consists of a common 3×3 convolutional block (Conv) and two Figure 3 The BLOCK module shown in the figure consists of an average pooling layer (Avg Pooling) and a fully connected layer (FC) in sequence, where X represents the number of channels of the initial input; input represents the input, and output,2d means that the final output of each input is a two-dimensional data: the probability of normal samples and abnormal samples.

[0049] Figure 5 This is an example diagram of the conversion of individual codes [3,2,2,3,5,4,1,5,4,4] into corresponding neural network models;

[0050] Figure 6 This is an example diagram of the weighted average calculation of the fitness of each individual uploaded by each client by the server in federated learning;

[0051] Figure 7 This is an example diagram of crossover operations during the evolution of neural architectures;

[0052] Figure 8 This is an example diagram of the mutation operation during the evolution of the neural architecture;

[0053] Fig. 9 The performance comparison chart of the technology of the present invention and the existing deep federated learning technology DeepFed (Federated Deep Learning for Intrusion Detection in Industrial Cyber-Physical Systems) is shown in Figure 1, where (a) is a comparison chart of the average accuracy of the technology of the present invention and DeepFed. Fig. 9 (b) is a comparison chart of the average recall rate between the present invention and DeepFed. Fig. 9 (c) in FIG. 1 is a comparison chart of the average F1 scores of the technology of the present invention and DeepFed. DETAILED DESCRIPTION

[0054] The present invention will be further described below in conjunction with the accompanying drawings, and the purpose and effects of the present invention will become more obvious.

[0055] Taking the industrial Internet of Things in a certain region as an example, Figure 1 A principle schematic diagram of the IIoT intrusion detection system and method based on neural architecture evolution federated learning is given, which mainly includes an offline optimization module of the federated learning model based on neural architecture evolution and an IoT online intrusion detection module based on the optimal federated learning model.

[0056] The offline optimization module of the federated learning model based on neural architecture evolution, that is, the server will perform individual encoding based on the combination of the federated learning neural network architecture of 5 basic neural network modules, randomly generate an initialization population, and send the code to each participating client; each client generates an initial local model, and each client performs initial local training for its own industrial Internet of Things local communication traffic historical data, and uploads its own model intermediate parameters to the server; the server aggregates the intermediate parameters, and then sends them to the local model of each client; each client then performs the next round of local training, repeating the process of uploading the intermediate parameters of each client model to the server, The server aggregates the intermediate parameters and sends them to each client to generate a local model. Each client then performs a new round of local training until each client reaches the specified maximum number of local training rounds. Then, each client performs local testing and uploads the evaluation indicators obtained to the server. The server performs fitness weighted average calculations on the evaluation indicators received from each client, and performs evolutionary operations such as crossover, mutation, and environmental selection on the individuals in the population. The above process is repeated to perform a specific round of neural network architecture search, and finally the neural network architecture with the best performance in historical data is obtained, thereby constructing a federated learning model based on the optimal neural network architecture.

[0057] The Internet of Things online intrusion detection module based on the optimal federated learning model, that is, the server deploys the federated learning model based on the optimal neural network architecture online, sends the optimal federated learning model to each client, each client generates a local initial model, each client performs initial local training for its own Internet of Things local communication traffic history data, and uploads its own model intermediate parameters to the server; after the server aggregates the model intermediate parameters, it sends the aggregated model intermediate parameters to the local model of each client, and then performs the next round of local training, repeating the process of uploading the intermediate parameters of each client model to the server, the server aggregates the intermediate parameters and sends them to each client to generate the local model, and each client performs a new round of local training, until each client reaches the specified maximum local training round, and each client generates a local final online test model; each client uses the local final online test model generated by itself for the real-time data of the Internet of Things local communication process collected by itself to perform online intrusion monitoring, if abnormal data is detected, it will issue an abnormal alarm, and evaluate the performance of the online test model of each client, upload the performance index of each client to the server, and the server performs weighted average of the performance index of each client to obtain the final average performance index.

[0058] In the above two modules, each client performs data sniffing, data collection, data analysis and data normalization operations on the local IoT communication traffic historical data and real-time data in turn, that is, firstly, data sniffing is performed on the IoT device communication traffic data, and the communication traffic in the network is stored in the system in the form of messages by putting the network card in promiscuous mode; after receiving the communication message, the data collection module summarizes it into the form of network flow and generates corresponding features; the data analysis summarizes all the category features and generates a feature mapping table, in which the category features and numerical features in the feature table correspond one to one; according to the feature table, all the category features in the data are converted into numerical features, thereby obtaining the source data set, marking it as X, and performing data normalization processing according to formula (1), and marking the data set obtained after normalization processing as X o , that is, the local historical data set required for offline optimization, and then X o The sequence numbers are randomized and assigned to the training set X at a ratio of 4:1. ox With the test set X oc middle;

[0059]

[0060] Among them, X oj Represents the dataset X o The feature of the jth dimension, X j Represents the feature of the jth dimension of the source dataset X, X jmax With Xjmin Respectively represent X oj The minimum and maximum values ​​of the data set X o The maximum value of the dimension;

[0061] The IIoT intrusion detection system and method based on neural architecture evolution federated learning is specifically implemented as follows:

[0062] (1) Setting relevant parameter values ​​of the IIoT intrusion detection system based on neural architecture evolution federated learning, the parameter values ​​include population size N = 20, iterative optimization times of neural network model architecture and hyperparameter optimization G = 20, offline training rounds of each client neural network model TP1 = 5, federated training rounds of each individual TP2 = 10, optimal neural network model federated training rounds TP3 = 20, crossover rate p c =0.8, mutation rate p m =0.2, the number of clients participating in federated learning is n=5;

[0063] (2) Randomly generate N individuals as the initial population S, S = {s1, s2, …, s j ,…,s N}, where s1, s2, s j ,s N They represent the first individual, the second individual, the jth individual and the Nth individual in the population, 1≤j≤N, and each individual s j The encoding form is s j = {B1, B2, ... B k ,…,B 10}, 1≤k≤10, where B1, B2, B k and B 10 Respectively represent individual s j The first bit code, the second bit code, the kth bit code and the tenth bit code of each B k The encoding of is a randomly generated integer in the range of 1 to 5, which represents the type of the selected lightweight convolutional neural network basic module, and its definition is shown in formula (2):

[0064] B k =randint(1,5),1≤k≤10 (2)

[0065] Among them, Randint represents the random integer generation function, and Randint(1,5) represents randomly generating an integer from 1 to 5; the types of lightweight convolutional neural network basic modules selected by integers 1, 2, 3, 4, and 5 are basic module 1 (marked as block1), basic module 2 (marked as block2), basic module 3 (marked as block3), basic module 4 (marked as block4), and basic module 5 (marked as block5); each type of basic module is divided into normal block and dimension reduction block. The spatial dimension of the output of the normal block is consistent with the spatial dimension of the input, and the spatial dimension of the output of the dimension reduction block is one quarter of the input;

[0066] The composition structure diagram of the five types of lightweight convolutional neural network modules used is as follows: Figure 2 As shown, Figure 2 (a) shows the composition structure of basic module 1 (block1). The left sub-graph shows the normal block. N is the input of the normal block in block1, output1 N is the output of the normal block in block1, input1 N and output1 N The spatial dimensions are consistent, NULL means no operation is performed on the input. N =input1 N ; When the normal block of module 1 is selected, the layer removal operation can be implemented, so that the number of layers of the model can be variable, that is, the actual number of layers of the model ranges from 2 to 10; the right sub-image is a dimensionality reduction block, which consists of a conventional convolution block (Conv 3×3) with a convolution kernel of 3×3. The dimensionality reduction effect is achieved by setting the stride of each convolution to 2, where input1 R is the input of the dimension reduction block in block1, output1 R is the output of the dimension reduction block in block1, output1 R The spatial dimension is input1 R One quarter of input1 R The spatial dimension of is C×W×W, C represents the number of input channels, W represents the input data dimension, then output1 R The spatial dimension of is C×(W / 2)×(W / 2), and then the Relu activation function is used to activate the convolution output;

[0067] Figure 2(b) represents the structural diagram of module 2 (block2), input2 is the input of block2, and output2 is the output of block2. Block2 consists of a depthwise separable convolution block (Dwise Conv 3×3) with a convolution kernel of 3×3 and a convolution block (Conv 1×1) with a convolution kernel of 1×1. The combination of 3×3 depthwise separable convolution and 1×1 convolution is equivalent to a regular convolution with a convolution kernel of 3×3, but the amount of calculation is much smaller than that of regular convolution. Therefore, this combination can make the model architecture more lightweight. Where s represents the value range of Stride. When s is set to 1, it means that block2 is a normal block, and the spatial dimensions of input2 and output2 are consistent; when s is set to 2, it means that block2 is a dimensionality reduction block, and the spatial dimension of output2 is one-fourth of input2; then the rectified linear unit (Rectified Linear Unit) activation function Relu6 with a maximum output limit of 6 is used to activate the output after convolution;

[0068] Figure 2 (c) shows the structural diagram of basic module 3 (block3), input3 is the input of block3, output3 is the output of block3, block3 consists of 1 convolution block with a convolution kernel of 1×1, 1 depth-separable convolution block with a convolution kernel of 3×3, and 1 convolution block with a convolution kernel of 1×1. The function of the first 1×1 convolution block is to increase the number of channels, and the function of the second 1×1 convolution block is to reduce the number of channels, so that the number of channels increases first and then decreases, so that the number of output channels can be consistent with the number of input channels. In addition, the second 1×1 convolution block uses a linear activation function to activate the output result of the 1×1 convolution. When s is set to 1, it means that block3 is a normal block, and the spatial dimensions of input3 and output3 are consistent. When s is set to 2, it means that block3 is a dimensionality reduction block, and the spatial dimension of output3 is one-fourth of input3;

[0069] Figure 2(d) shows the structural diagram of basic module 4 (block4), where input4 is the input of block4 and output4 is the output of block4. Block4 consists of a convolution block with a 1×1 convolution kernel and a depthwise separable convolution block with a 3×3 convolution kernel, which is the opposite order of block2. The combination of 1×1 convolution and 3×3 depthwise separable convolution is equivalent to a 3×3 conventional convolution, and the amount of calculation is much smaller than that of conventional convolution. When s is set to 1, it means that block4 is a normal block, and the spatial dimensions of input4 and output4 are consistent. When s is set to 2, it means that block4 is a dimensionality reduction block, and the spatial dimension of output4 is one quarter of that of input4.

[0070] Figure 2 (e) shows the composition structure diagram of basic module 5 (block5). The left sub-graph shows the normal block, input5 N is the input of the normal block in block5, output5 N is the output of the normal block in block5, input5 N and output5 N The spatial dimensions of are consistent. The Channel Split operation represents the splitting of input 5 N Divide into two parts according to the number of channels, that is, input5 with spatial dimensions of C×W×W N Randomly divide it into two sub-vectors with spatial dimensions of (C / 2)×W×W, one of which is not operated, and the other is operated by a 1×1 convolution block, a 3×3 depth-separable convolution block and a 1×1 convolution block in sequence. The spatial dimension after the operation remains unchanged, that is, it is still (C / 2)×W×W. Then the two (C / 2)×W×W sub-vectors are concat-operated, that is, spliced ​​into C×W×W by channel, and finally the channels are randomly rearranged, that is, the Channel Shuffle operation is performed to obtain the final output output5 N ; The right sub-image represents the dimension reduction block, input5 R is the input of the dimension reduction block in block5, output5 R is the output of the dimension reduction block in block5, output5 R The spatial dimension is input5 R The left branch consists of 1 3×3 depth-separable convolution block and 1 1×1 convolution block, and its structure is similar to block2. The right branch consists of 1 1×1 convolution block, 1 3×3 depth-separable convolution block and 1 1×1 convolution block, and its structure is similar to block3. The convolution movement stride of both branches is set to 2. RAfter these two branch operations, the output results of the two branch operations are concatenated by channel, and then the channels are randomly rearranged (Channel Shuffle) to obtain the final output output5 R , at this time output5 R The spatial dimension is input5 R one-fourth;

[0071] Every 4 normal blocks of basic modules are followed by a dimension reduction block of basic module. At the same time, the output of the first normal block is spliced ​​with the output of the fourth normal block, so that the number of channels is doubled, and then used as the input of the dimension reduction block. After the dimension reduction block operation, the spatial dimension becomes one quarter of the input. In this way, the combination of 4 normal blocks and 1 dimension reduction block constitutes a combined module, and the combined module is marked as a BLOCK module, such as Figure 3 As shown;

[0072] An example of the overall structure of a neural network model participating in federated learning is as follows: Figure 4 As shown, it consists of a conventional convolution block (Conv) with a convolution kernel of 3×3, and two Figure 3 The BLOCK module shown in the figure, an average pooling layer (Avg Pooling) and a fully connected layer (FC) are composed in sequence, where X represents the initial number of channels, C represents the number of input channels, and W represents the input data dimension. The Conv block expands the spatial dimension of the initial input from X×W×W to C×W×W, which facilitates the calculation operation of the subsequent BLOCK module. Figure 5 An example of a neural network model structure representing a randomly generated individual encoding is given. The encoding information of the individual is: [3,2,2,3,5,4,1,5,4,4];

[0073] (3) For each individual s in population S j (1≤j≤N) for fitness evaluation. The specific evaluation process is as follows: The federated learning server will evaluate the fitness of each individual s in the population S. j The code of (1≤j≤N) is sent to the participating n clients to generate a local initial model. Each client uses its own local historical data set X ox , respectively perform the initial local training of TP1 round, upload the intermediate parameters of each model to the server, the server aggregates the intermediate parameters, and sends them to their local models after aggregation, and then perform the next round of local training of TP1 round, repeat the process of clients uploading their own model intermediate parameters, the server aggregates the model intermediate parameters, sends the aggregated intermediate parameters to the local models of each client, and the client performs the next round of local training, until the maximum local training round TP2 specified by the client is reached; then, each client uses its own test set Xoc Perform local testing and calculate the F1 score according to formula (3) as the fitness of the individual corresponding to the model, which is marked as f l , 1≤l≤n, upload the fitness obtained by each client to the server, and the server receives the fitness f uploaded by each client according to formula (4) l Perform weighted average operation to obtain individual s j The final fitness f(s) of (1≤j≤N) j ), Figure 6 An example of calculating the final fitness of individuals in population S is given;

[0074]

[0075]

[0076] Among them, TP represents the number of normal IoT data samples correctly predicted as normal classification, TN represents the number of abnormal IoT data samples correctly predicted as abnormal classification, FP represents the number of abnormal IoT data samples incorrectly predicted as normal classification, and FN represents the number of normal IoT data samples incorrectly predicted as abnormal classification.

[0077] The specific process of the server aggregating the model intermediate parameters uploaded by n clients is as follows: the client aggregates the model intermediate parameters w l (1≤l≤n) are uploaded to the server. After receiving the parameters uploaded by all clients, the server aggregates the parameters to obtain the aggregated parameters w(s j )(1≤l≤n), then w(s j ) is re-sent to each client, where the aggregation method is shown in formula (5):

[0078]

[0079] (4) Perform crossover and mutation operations on population S to generate offspring population S c The specific implementation steps of the crossover operation are: randomly select 5 individuals from S to compare their fitness, select the two individuals with the highest fitness as parents, denoted as p1 and p2 respectively; generate a random number q1=Random(0,1), where Random(0,1) represents a uniform random number generated between 0 and 1; if q1<crossover rate p c , then generate a random integer P c =Randint(2,10) as the crossover site, P c Indicates that the crossover site is located in the Pth parent individual p1, p2 cCode, perform single-point crossover on p1 and p2, that is, the Pth c The coding fields of the first to the tenth coding are exchanged, while keeping the other coding fields of p1 and p2 unchanged, generating two offspring individuals c1 and c2; if q1≥p c , then c1=p1, c2=p2; Figure 7 An example of a crossover operation is given, where the crossover point P c =3; The specific implementation steps of the mutation operation are: for each offspring individual c, generate a random number q2 = Random (0,1), if q2 < mutation rate p m , then generate a random integer P m =Randint(1,10) as the variation site, P m Indicates that the variant site is located in the Pth individual in the offspring c. m A code is randomly generated from 1 to 5, which is different from the Pth individual in the offspring c. m An integer that encodes its own value, and uses this integer to replace the Pth m The original encoding value is kept unchanged, thus generating a mutant individual c M ; If q2 ≥ p m , then c M =c; Figure 8 An example of mutation operation is given, the mutation site P m =3; c M Join S c , repeat the above crossover and mutation operations until S c The number of individuals in is N = 20;

[0080] (5) Generate the next generation population S through environmental selection operation N The specific implementation steps of the environmental selection operation are: c Merge with the current population S, S = S ∪ S c ; For the merged population S, use the five-element competition selection as the environmental selection method, randomly select 5 individuals from S, evaluate the fitness of the individuals according to step (3), and select the individual with the highest fitness to enter the next generation population S N ; Repeat this operation until S N The population size is N = 20;

[0081] (6) Unconditionally accept S = S N ;

[0082] (7) Repeat steps (4) to (6) until the set number of iterative optimization times G = 20 is met;

[0083] (8) Select the individual with the highest fitness in S and record it as Sbest =[5,5,4,3,5,2,5,2,1,1];

[0084] (9) The optimal fitness individual S finally obtained in the federated offline optimization training module best The encoding is sent to n clients to build a local initial model. Each client performs initial local training of TP1 rounds for its own local historical data set of the Internet of Things, uploads its own model intermediate parameters to the server, aggregates the intermediate parameters, and sends them to its own local model after aggregation. Then, the next round of local training of TP1 rounds is carried out. The process of uploading the intermediate parameters of the client model, aggregating the intermediate parameters of the server, sending the intermediate parameters to the client to generate the local model, and the client performing a new round of local training of TP1 rounds is repeated until the maximum local training round TP3 specified by the client is reached; each client generates a local final online test model; the optimal individual S in this case best The optimal architecture information represented is [5,5,4,3,5,2,5,2,1,1].

[0085] (10) The n clients participating in the federated test collect real-time data X of the local communication process of the industrial Internet of Things t , use the local final online test model generated by each client to perform online intrusion monitoring. If abnormal data is detected, an abnormal alarm is issued, and the precision (Pre), recall (Rec) and F1 score (F1) of each client online test model are calculated according to formula (6), formula (7) and formula (3), respectively, and are marked as Pre l 、Rec l and F 1l , 1≤l≤n; the Pre l ,Rec l and F 1l Upload to the server;

[0086]

[0087]

[0088] Among them, TP represents the number of normal IIoT data samples correctly predicted as normal classification, TN represents the number of abnormal IIoT data samples correctly predicted as abnormal classification, FP represents the number of abnormal IIoT data samples incorrectly predicted as normal classification, and FN represents the number of normal IIoT data samples incorrectly predicted as abnormal classification.

[0089] (11) The server performs weighted averaging on the performance indicators uploaded by each client according to formulas (8) to (10) to obtain the final average precision, average recall and average F1 score. In this embodiment, the performance indicators obtained by the technology of the present invention are: average precision = 0.9903, average recall = 0.9870, and F1 score = 0.9887.

[0090]

[0091]

[0092]

[0093] The performance comparison between the technology of the present invention and the existing deep federated learning technology DeepFed (Federated Deep Learning for Intrusion Detection in Industrial Cyber-Physical Systems) is as follows: Fig. 9 As shown, (a) is a comparison chart of the average precision of the technology of the present invention and DeepFed, Fig. 9 (b) is a comparison chart of the average recall rate between the present invention and DeepFed. Fig. 9 (c) is a comparison chart of the average F1 score between the technology of the present invention and DeepFed. It is not difficult to find that compared with the existing DeepFed technology that relies on artificial experience design, the intrusion detection performance indicators such as average precision, average recall, and average F1 score obtained by the technology of the present invention are the best. In addition, compared with the existing technology, the technology of the present invention realizes the automatic generation and optimization design of lightweight neural network models in the industrial Internet of Things intrusion detection system based on federated learning, which not only improves the intelligent design level and design efficiency of the industrial Internet of Things intrusion detection system, but also ensures the privacy security of industrial Internet of Things devices; at the same time, the neural architecture evolution technology adopted by the present invention can optimize the offline search for a lightweight federated learning model based on the optimal combination architecture of lightweight basic modules, which is more suitable for online deployment in performance-constrained industrial Internet of Things devices.

[0094] In summary, the beneficial effects of the present invention are as follows: compared with the prior art, the present invention can not only realize the automatic generation and optimization design of the federated learning model for industrial Internet of Things intrusion detection, but also improve the intelligent design level and design efficiency of the industrial Internet of Things intrusion detection system; the use of neural architecture evolution technology can optimize and search offline for a lightweight federated learning model based on the optimal combination architecture of lightweight basic modules, which is more suitable for online deployment in performance-constrained industrial Internet of Things devices; while protecting the privacy and security of industrial Internet devices, it also improves the performance indicators of IIoT intrusion detection such as precision, recall rate, and F1 score.

Claims

1. An industrial Internet of Things intrusion detection system based on neural architecture evolution federated learning, characterized in that: The system includes an offline optimization module of a federated learning model based on neural architecture evolution and an online intrusion detection module of the industrial Internet of Things based on the optimal federated learning model; The offline optimization module of the federated learning model based on neural architecture evolution, that is, the server will perform individual encoding based on the combination of the federated learning neural network architecture of 5 basic neural network modules, randomly generate an initialization population, and send the code to each participating client; each client generates an initial local model, and each client performs initial local training for its own industrial Internet of Things local communication traffic historical data, and uploads its own model intermediate parameters to the server; the server aggregates the intermediate parameters, and then sends them to the local model of each client; each client then performs the next round of local training, repeating the process of uploading the intermediate parameters of each client model to the server and the server. After the client aggregates the intermediate parameters, it sends them to each client to generate the local model. Each client then performs a new round of local training until each client reaches the specified maximum number of local training rounds. Then, each client performs local testing and uploads the evaluation indicators obtained by each client to the server. The server performs fitness weighted average calculation on the evaluation indicators received from each client, and performs evolution operations including crossover, mutation and environmental selection on the individuals in the population. The above process is repeated to perform a specific round of neural network architecture search, and finally obtains the neural network architecture that performs best in historical data, thereby constructing a federated learning model based on the optimal neural network architecture. The industrial Internet of Things online intrusion detection module based on the optimal federated learning model, that is, the server deploys the federated learning model based on the optimal neural network architecture online, sends the optimal federated learning model to each client, each client generates a local initial model, and each client performs initial local training for its own industrial Internet of Things local communication traffic history data, and uploads its own model intermediate parameters to the server; after the server aggregates the model intermediate parameters, it sends the aggregated model intermediate parameters to the local model of each client, and then performs the next round of local training, repeating the process of uploading the intermediate parameters of each client model to the server, the server aggregates the intermediate parameters and sends them to each client to generate the local model, and each client performs a new round of local training, until each client reaches the specified maximum local training round, and each client generates a local final online test model; each client uses the local final online test model generated by each client for the real-time data of the industrial Internet of Things local communication process collected by each client to perform online intrusion monitoring, if abnormal data is detected, an abnormal alarm is issued, and the performance of the online test model of each client is evaluated, and the performance index of each client is uploaded to the server, and the server performs weighted average of the performance index of each client to obtain the final average performance index; In the above two modules, each client performs data sniffing, data collection, data analysis and data normalization operations on the local industrial Internet of Things communication traffic historical data and real-time data in turn, that is, firstly, the communication traffic data of the industrial Internet of Things equipment is sniffed, and by putting the network card in promiscuous mode, the communication traffic in the network is stored in the system in units of messages; After receiving the communication message, the data acquisition module aggregates it into the form of network flow and generates corresponding features; Data parsing summarizes all the categorical features and generates a feature mapping table, in which the categorical features correspond to the numerical features one by one. According to the feature table, all the categorical features in the data are converted into numerical features to obtain the source data set, which is marked as X. The data is normalized according to formula (1), and the data set obtained after normalization is marked as X. o , that is, the local historical data set required for offline optimization, and then X o The sequence numbers are randomized and assigned to the training set X at a ratio of 4:

1. ox With the test set X oc middle; Among them, X oi Represents the dataset X o The feature of the i-th dimension, X i Represents the features of the i-th dimension of the source dataset X, X imax With X imin Respectively represent X oi The minimum and maximum values ​​of the data set X o The maximum value of the dimension.

2. An industrial Internet of Things intrusion detection method based on neural architecture evolution federated learning using the system described in claim 1, characterized in that: The following steps are involved: (1) Setting the initialization parameter values ​​of the industrial Internet of Things intrusion detection method based on neural architecture evolution federated learning, the parameter values ​​include the population size N, the number of iterations of optimization of the neural network model architecture and hyperparameters G, the number of rounds of offline training TP1 of each client neural network model, the number of rounds of federated training TP2 of each individual, the number of rounds of federated training TP3 of the optimal neural network model, and the crossover rate p c , mutation rate p m , the number of clients participating in federated learning is n; (2) Five lightweight convolutional neural network basic modules are used as candidate component modules for optimizing the federated learning neural network architecture, and N individuals are randomly generated as the initial population S, where S = {s1, s2, …, s j ,…,s N }, where s1, s2, s j ,s N They represent the first individual, the second individual, the jth individual and the Nth individual in the population, 1≤j≤N, and each individual s j The encoding form is s j = {B1, B2, ... B k ,…,B 10 }, 1≤k≤10, where B1, B2, B k and B 10 Respectively represent individual s j The first bit code, the second bit code, the kth bit code and the tenth bit code of each B k The encoding of is a randomly generated integer in the range of 1 to 5, which represents the type of the selected lightweight convolutional neural network basic module, and its definition is shown in formula (2): B k =randint(1,5),1≤k≤10 (2) Among them, Randint represents the random integer generation function, and Randint(1,5) represents randomly generating an integer from 1 to 5; the types of lightweight convolutional neural network basic modules selected by integers 1, 2, 3, 4, and 5 are basic module 1 (marked as block1), basic module 2 (marked as block2), basic module 3 (marked as block3), basic module 4 (marked as block4), and basic module 5 (marked as block5); each type of basic module is divided into normal block and dimension reduction block. The spatial dimension of the output of the normal block is consistent with the spatial dimension of the input, and the spatial dimension of the output of the dimension reduction block is one quarter of the input; The normal block of basic module 1 (block1) does not perform any operation on the input, and the input is the output. When the normal block of module 1 is selected, the layer removal operation can be implemented, so that the number of layers of the model can be variable, that is, the actual number of layers of the model ranges from 2 to 10; the dimension reduction block of block1 consists of a conventional convolution block with a convolution kernel of 3×3. The dimension reduction effect is achieved by setting the stride of each convolution to 2. The Relu activation function is used to activate the output after convolution, so the spatial dimension of the output of the dimension reduction block is one quarter of the spatial dimension of the input; Basic module 2 (block2) consists of a depthwise separable convolution block with a convolution kernel of 3×3 and a convolution block with a convolution kernel of 1×1. The combination of 3×3 depthwise separable convolution and 1×1 convolution is equivalent to a 3×3 regular convolution, but the amount of calculation is much less than that of regular convolution. Therefore, this combination can make the model architecture more lightweight. When the stride of each movement of the 3×3 depthwise separable convolution is set to 1, it means that block2 is a normal block, and the spatial dimension of the output is consistent with the spatial dimension of the input. When the stride is set to 2, it means that block2 is a dimensionality reduction block, and the spatial dimension of the output is one quarter of the spatial dimension of the input. Basic module 3 (block3) consists of 1 1×1 convolution block, 1 3×3 depth-separable convolution block and 1 1×1 convolution block; the first 1×1 convolution block is used to increase the number of channels, and the second 1×1 convolution block is used to reduce the number of channels, so that the number of channels increases first and then decreases, so that the number of output channels is consistent with the number of input channels. In addition, the second 1×1 convolution block uses a linear activation function to activate the output result of the 1×1 convolution; when the stride of each movement of the 3×3 depth-separable convolution is set to 1, it means that block3 is a normal block, and the spatial dimensions of the output and input are consistent; when the stride is set to 2, it means that block3 is a dimension reduction block, and the spatial dimension of the output is one quarter of the input; Basic module 4 (block4) consists of a 1×1 convolution block and a 3×3 depthwise separable convolution, which is the opposite order of module 2. The combination of 1×1 convolution and 3×3 depthwise separable convolution is equivalent to a 3×3 regular convolution, but the amount of calculation is much smaller than that of regular convolution. When the stride of each movement of the 3×3 depthwise separable convolution is set to 1, it indicates a normal block of block4, and the spatial dimension of the output is consistent with the spatial dimension of the input. When the stride is set to 2, it indicates a dimensionality reduction block of block4, and the spatial dimension of the output is one quarter of the input. Basic module 5 (block5) is divided into a normal block and a dimensionality reduction block. In the normal block, the input is first divided into two parts according to the number of channels, that is, the input with a spatial dimension of C×W×W is randomly divided into 2 (C / 2)×W×W sub-vectors, C represents the number of input channels, W represents the data dimension of the input, one of which is not operated, and the other is sequentially operated by a 1×1 convolution block, a 3×3 depth-separable convolution block, and a 1×1 convolution block. The output is then concatenated with the sub-vector that is not operated by channel. At this time, the spatial dimension of the output is consistent with the spatial dimension of the input, and then the channels are randomly rearranged to obtain the final output; in bl In the dimension reduction block of ock5, two operations are performed simultaneously on its input. Operation 1 consists of a 3×3 depth-separable convolution block and a 1×1 convolution block, and its structure is similar to block2. Operation 2 consists of a 1×1 convolution block, a 3×3 depth-separable convolution block, and a 1×1 convolution block, and its structure is similar to block3. The convolution movement stride of the 3×3 depth-separable convolution in both operations is set to 2 to achieve the dimension reduction effect. Finally, the output results of the two operations are spliced ​​by channel, and then the channels are randomly rearranged to obtain the final output. At this time, the spatial dimension of the output is one quarter of the input. An important module for constructing the neural network model architecture is the combination module, which is marked as a BLOCK module; the specific implementation process of the BLOCK module is: every 4 normal blocks of the basic modules are followed by a dimension reduction block of the basic module, and the output of the first normal block is spliced ​​with the output of the fourth normal block, so that the number of channels is doubled, and then used as the input of the dimension reduction block, and the dimension is halved after the dimension reduction block operation, so that the combination of 4 normal blocks and 1 dimension reduction block constitutes a combination module, which is marked as a BLOCK module; The overall architecture of the neural network model participating in federated learning consists of a 3×3 regular convolution block, two of the above-mentioned BLOCK modules, an average pooling layer, and a fully connected layer. The 3×3 convolution block is used to increase the number of channels of the initial input to the size required for the BLOCK module operation. (3) For each individual s in population S j (1≤j≤N) for fitness evaluation. The specific evaluation process is as follows: The federated learning server will evaluate the fitness of each individual s in the population S. j The code of (1≤j≤N) is sent to the participating n clients to generate a local initial model. Each client uses its own local historical data set X of the industrial Internet of Things. ox , respectively perform the initial local training of TP1 round, upload the intermediate parameters of each model to the server, the server aggregates the intermediate parameters, and sends them to their local models after aggregation, and then perform the next round of local training of TP1 round, repeat the process of clients uploading their own model intermediate parameters, the server aggregates the model intermediate parameters, sends the aggregated intermediate parameters to the local models of each client, and the client performs the next round of local training, until the maximum local training round TP2 specified by the client is reached; then, each client uses its own test set X oc Perform local testing and calculate the F1 score according to formula (3) as the fitness of the individual corresponding to the model, which is marked as f l , 1≤l≤n, upload the fitness obtained by each client to the server, and the server receives the fitness f uploaded by each client according to formula (4) l Perform weighted average operation to obtain individual s j The final fitness f(s) of (1≤j≤N) j ): Among them, TP represents the number of normal IIoT data samples correctly predicted as normal classification, TN represents the number of abnormal IIoT data samples correctly predicted as abnormal classification, FP represents the number of abnormal IIoT data samples incorrectly predicted as normal classification, and FN represents the number of normal IIoT data samples incorrectly predicted as abnormal classification; The specific process of the server aggregating the model intermediate parameters uploaded by n clients is as follows: n clients participating in federated learning aggregate the model intermediate parameters w trained locally l (1≤l≤n) is uploaded to the server. After receiving the parameters uploaded by all clients, the server calculates the model intermediate parameters w of all clients according to formula (5). l (1≤l≤n) to obtain the aggregated parameter w(s j )(1≤l≤n), then w(s j ) is re-issued to each client; (4) Perform crossover and mutation operations on population S to generate offspring population S c The specific implementation steps of the crossover operation are: randomly select 5 individuals from S to compare their fitness, select the two individuals with the highest fitness as parents, denoted as p1 and p2 respectively; generate a random number q1=Random(0,1), where Random(0,1) represents a uniform random number generated between 0 and 1; if q1<crossover rate p c , then generate a random integer P c =Randint(2,10) as the crossover site, P c Indicates that the crossover site is located in the Pth parent individual p1, p2 c Code, perform single-point crossover on p1 and p2, that is, the Pth c The coding fields of the first to the tenth coding are exchanged, while keeping the other coding fields of p1 and p2 unchanged, generating two offspring individuals c1 and c2; if q1≥p c , then c1=p1, c2=p2; the specific implementation steps of the mutation operation are: for each offspring individual c, generate a random number q2=Random(0,1), if q2<mutation rate p m , then generate a random integer P m =Randint(1,10) as the variation site, P m Indicates that the variant site is located in the Pth individual in the offspring c. m A code is randomly generated from 1 to 5, which is different from the Pth individual in the offspring c. m An integer that encodes its own value, and uses this integer to replace the Pth m The original value of the code is encoded, while keeping the values ​​of other codes unchanged, thus generating a mutant individual c M ; If q2 ≥ p m , then c M = c, M Join S c ; Repeat the above crossover and mutation operations until S c The number of individuals in is N; (5) Generate the next generation population S through environmental selection operation N The specific implementation steps of the environmental selection operation are: c Merge with the current population S, S = S ∪ S c ; For the merged population S, use the five-element competition selection as the environmental selection method, randomly select 5 individuals from S, evaluate the fitness of the individuals according to step (3), and select the individual with the highest fitness to enter the next generation population S N ; Repeat this operation until S N The population size is N; (6) Unconditionally accept S = S N ; (7) Repeat steps (4) to (6) until the set number of iterative optimization times G is met; (8) Select the individual with the highest fitness in S and record it as S best ; (9) The optimal fitness individual S finally obtained in the federated offline optimization training module best The encoding is sent to n clients to build a local initial model. Each client performs initial local training of TP1 round for its own local historical data set of industrial IoT, uploads its own model intermediate parameters to the server, aggregates the intermediate parameters, and sends them to its own local model after aggregation. Then, the next round of local training of TP1 round is carried out. The process of uploading the intermediate parameters of the client model, aggregating the intermediate parameters of the server, sending the intermediate parameters to the client to generate the local model, and the client performs a new round of local training of TP1 round is repeated until the maximum local training round TP3 specified by the client is reached; each client generates a local final online test model; (10) The n clients participating in the federated test collect real-time data X of the local communication process of the industrial Internet of Things t , use the local final online test model generated by each client to perform online intrusion monitoring. If abnormal data is detected, an abnormal alarm is issued, and the precision (Pre), recall (Rec) and F1 score (F1) of each client online test model are calculated according to formula (6), formula (7) and formula (3), respectively, and are marked as Pre l 、Rec l and F 1l , 1≤l≤n; the Pre l ,Rec l and F 1l Upload to the server; Among them, TP represents the number of normal IIoT data samples correctly predicted as normal classification, TN represents the number of abnormal IIoT data samples correctly predicted as abnormal classification, FP represents the number of abnormal IIoT data samples incorrectly predicted as normal classification, and FN represents the number of normal IIoT data samples incorrectly predicted as abnormal classification; (11) The server performs weighted average of the performance indicators uploaded by each client according to formulas (8) to (10) to obtain the final average precision, average recall and average F1 score:

Images