An Internet of Things network intrusion detection method, device, medium and product

Abstract

The present application discloses an Internet of Things network intrusion detection method, device, medium and product, which relate to the field of network intrusion detection technology. The method introduces digital twin technology to perceive and predict network traffic data, combines network traffic feature data to construct a training data set, and then uses the K clustering algorithm to mark the difficult-to-distinguish samples in the training data set. Finally, by controlling the reward value of the deep learning network model, the detection accuracy of the difficult-to-distinguish samples is improved, thereby achieving high-precision intrusion detection as a whole.

Description

Technical Field

[0001] The present application relates to the field of network intrusion detection technology, and in particular to an Internet of Things network intrusion detection method, device, medium and product based on deep reinforcement learning and K clustering. Background Art

[0002] The Internet of Things (IoT) is an "internet of connected things." It connects various objects (such as sensors and smart devices) via the internet, constantly sensing and processing changes in their state information, ultimately enabling a series of automated operations such as intelligent management and monitoring of objects. The continuous expansion of the IoT has led to increasingly prominent cybersecurity issues within the IoT. Ensuring IoT security, ensuring its rapid development, and providing safe, reliable, intelligent, and efficient services are pressing challenges. Intrusion detection, as an active defense measure, plays a vital role in improving IoT network and information security by proactively monitoring the current environment in real time, alerting users to malicious behavior, and taking appropriate measures. It has become a current research hotspot.

[0003] Researchers have conducted extensive research in the field of network intrusion detection for the Internet of Things (IoT) and have achieved considerable success. With the development of artificial intelligence (AI) technology, the efficiency of IDS has further improved. The range of AI technologies applicable to network intrusion detection is extensive, and various machine learning techniques, such as random forests, principal component analysis, and support vector machines, have been widely used to detect abnormal behavior patterns in networks. However, traditional machine learning suffers from high false positive rates, low detection rates, and detection difficulties caused by large data volumes.

[0004] Furthermore, the current IoT has a wide range of applications. This wide range of scenarios means that the IoT's network architecture and communication processes are becoming increasingly complex. Furthermore, with the advancement of network and artificial intelligence technologies, cyberattack methods are also constantly evolving. In this complex IoT network environment, facing the same constantly evolving cyberattack methods, high accuracy is one of the main challenges facing intrusion detection technology in IoT networks. Summary of the Invention

[0005] The purpose of this application is to provide an Internet of Things network intrusion detection method, equipment, medium and product, which can effectively improve the accuracy of Internet of Things network intrusion detection and enhance the intrusion detection performance.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides an IoT network intrusion detection method, comprising:

[0008] Constructing a training data set based on network traffic feature data and network traffic prediction values, wherein the network traffic feature data is data obtained by extracting features from the network communication data of the Internet of Things, and the network traffic prediction values ​​are data obtained by processing the network traffic feature data using digital twin technology;

[0009] Using a K clustering algorithm to process the training data set to obtain a difficult-to-classify sample set;

[0010] Selecting a current state from the training data set, and selecting a current action corresponding to the current state using a greedy strategy, wherein the current state represents the network traffic characteristic data and the network traffic prediction value at the current moment, and the current action represents whether the Internet of Things is invaded or normal at the current moment;

[0011] Calculating a current moment reward obtained after executing the current moment action, and obtaining a next moment state, wherein the current moment reward is determined based on a reward coefficient, wherein the reward coefficient is a coefficient determined based on a first judgment result, the first judgment result being a result indicating that the network traffic characteristic data and the network traffic prediction value at the current moment belong to the difficult sample set, or a result indicating that the network traffic characteristic data and the network traffic prediction value at the current moment do not belong to the difficult sample set;

[0012] Constructing a state transition quadruple according to the current state, the current action, the current reward, and the next state, and storing the state transition quadruple in an experience replay pool;

[0013] Selecting a number of the state transition quadruple from the experience replay pool as training samples to train a deep learning network model;

[0014] Based on the trained deep learning network model, predict whether the Internet of Things has been invaded.

[0015] Optionally, the network traffic characteristic data include: the number of network flows per second, the number of data packets transmitted per second, the total number of bytes of data packet headers transmitted per second, the average number of bytes of data packet headers transmitted per second, the standard deviation of the number of bytes of data packet headers transmitted per second, the total number of bytes of data packet messages transmitted per second, the average number of bytes of data packet messages transmitted per second, the standard deviation of the number of bytes of data packet messages transmitted per second, the number of data packets transmitted in the forward direction per second, the total number of bytes of data packet headers transmitted in the forward direction per second, the average number of bytes of data packet headers transmitted in the forward direction per second, the standard deviation of the number of bytes of data packet headers transmitted in the forward direction per second, the total number of bytes of data packet messages transmitted in the forward direction per second, the average number of bytes of data packet messages transmitted in the forward direction per second, the standard deviation of the number of bytes of data packet messages transmitted in the forward direction per second, the number of data packets transmitted in the reverse direction per second, the total number of bytes of data packet headers transmitted in the reverse direction per second, the average number of bytes of data packet headers transmitted in the reverse direction per second, the standard deviation of the number of data packet messages transmitted in the reverse direction per second Standard deviation of the number of transmitted packet header bytes, total number of reverse packet messages transmitted per second, average number of reverse packet messages transmitted per second, standard deviation of the number of reverse packet messages transmitted per second, number of TCP packets per second, total number of TCP packet header bytes per second, average number of TCP packet header bytes per second, standard deviation of the number of TCP packet header bytes per second, total number of TCP packet messages per second, average number of TCP packet messages per second, standard deviation of the number of TCP packet messages per second, number of UDP packets per second, total number of UDP packet header bytes per second, total number of UDP packet messages per second, average number of UDP packet messages per second, standard deviation of the number of UDP packet messages per second, average time interval between packets, standard time interval between packets, average time interval between forward packets, standard time interval between forward packets, and number of ACK flags.

[0016] Optionally, before executing the step of “constructing a training data set based on network traffic feature data and network traffic prediction values”, the IoT network intrusion detection method further includes:

[0017] The network traffic characteristic data and the network traffic prediction value are normalized.

[0018] Optionally, a K clustering algorithm is used to process the training data set to obtain a difficult-to-classify sample set, specifically including:

[0019] Randomly select a sample from the training data set as an initial cluster center;

[0020] Calculating a first distance between all samples in the training data set and the initial cluster center;

[0021] Calculating the probability of each sample being selected as a cluster center according to the first distance;

[0022] Selecting a number of new initial cluster centers according to the probability as first cluster centers;

[0023] Calculating a second distance between each of the samples and each of the first cluster centers;

[0024] Dividing the samples into corresponding clusters according to the second distance;

[0025] Calculate the average value of all sample points in each cluster, and use the average value as the new first cluster center, and return to the step of "calculating the second distance between each sample and each first cluster center";

[0026] When the preset iteration conditions are met, the clustering results are obtained;

[0027] Determining an attack sample set and a benign sample set according to the clustering result;

[0028] The difficult-to-separate sample set is determined according to the attack sample set and the benign sample set.

[0029] Optionally, the expression of the current moment reward is:

[0030]

[0031]

[0032] Among them, r t (s t ,a t ) is the reward at the current moment t; s t is the current state at time t; a t is the action at the current moment t; λ(t) is the reward coefficient at the current moment t; r is the basic reward; H I is the difficult sample set; υ is the reward coefficient for difficult samples.

[0033] Optionally, the deep learning network model is a DQN network.

[0034] Optionally, the loss function of the deep learning network model is expressed as:

[0035] L(θ)=E[r t +γmaxQ(s t+1 ,a t+1 ,θ′)-Q(s t ,a t ,θ)] 2 ;

[0036] Among them, r t is the reward at the current moment t; γ is the reward attenuation factor; Q(s t+1 ,a t+1,θ′) is the Q value calculated after the target network in the DQN network performs the action at time t+1; s t+1 is the state at time t+1; θ′ is the target network parameter; Q(s t ,a t ,θ) is the Q value calculated by the Q network in the DQN network after executing the action at the current time t; s t is the current state at time t; a t is the action at the current moment t; θ is the Q network parameter.

[0037] In a second aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement an IoT network intrusion detection method as described in any one of the first aspects above.

[0038] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an Internet of Things network intrusion detection method as described in any one of the first aspects above.

[0039] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements an Internet of Things network intrusion detection method described in any one of the first aspects above.

[0040] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0041] The present application provides an IoT network intrusion detection method, device, medium and product. By introducing digital twin technology to perceive and predict network traffic data, a training data set is constructed in combination with network traffic feature data. The hard-to-distinguish samples in the training data set are then marked using a K clustering algorithm. Finally, the detection accuracy of hard-to-distinguish samples is improved by controlling the reward value of a deep learning network model, thereby achieving high-precision intrusion detection as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 This is an application environment diagram of an Internet of Things network intrusion detection method in Example 1 of the present application;

[0044] Figure 2 A schematic diagram of a flow chart of an IoT network intrusion detection method provided in Example 1 of the present application;

[0045] Figure 3 Schematic diagram of the DQN-based intrusion detection process in Example 1 of the present application;

[0046] Figure 4 A schematic diagram of the structure of a computer device provided in Example 2 of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0049] Example 1

[0050] The embodiment of the present application provides an Internet of Things network intrusion detection method, which can be applied to Figure 1In the application environment shown, the terminal 102 communicates with the server 104 via a network. The data storage system can store data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the original network communication data of the Internet of Things to the server 104. After receiving the network communication data, the server 104 performs feature extraction on the network communication data to obtain network traffic feature data, and processes the network traffic feature data using digital twin technology to obtain network traffic prediction values. Secondly, a K clustering algorithm is used to process the training data set including the network traffic feature data and the network traffic prediction values ​​to obtain a difficult sample set. Then, a deep learning network model is constructed. Then, a current state is selected from the training data set, and a current action corresponding to the current state is selected using a greedy strategy. The current reward obtained after executing the current action is calculated, and the next state is obtained to obtain a state transition quadruple. Finally, the deep learning network model is trained using several state transition quadruples, and a detection result is output when the deep learning network model training is completed, i.e., whether the Internet of Things has been invaded. The server 104 can feed back the obtained detection results to the terminal 102. In addition, in some embodiments, the IoT network intrusion detection method can also be implemented independently by the server 104 or the terminal 102. For example, the terminal 102 can directly use the IoT network intrusion detection method to detect network communication data, or the server 104 can obtain network communication data from a data storage system and use the IoT network intrusion detection method to detect the network communication data.

[0051] Terminal 102 may include, but is not limited to, various desktop computers, laptops, smartphones, tablet computers, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, and smart car devices. Portable wearable devices may include smart watches, smart bracelets, and head-mounted devices. Server 104 may be implemented as a standalone server or a server cluster consisting of multiple servers, or may be a cloud server.

[0052] In an exemplary embodiment, Figure 2 As shown, this embodiment provides an Internet of Things network intrusion detection method, which is executed by a computer device, specifically a computer device such as a terminal or a server, or a terminal and a server. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate the method, including the following steps 1) to 3). In which:

[0053] 1) Build a training dataset:

[0054] Step 201: Extract features from network communication data and process them using digital twin technology to obtain network traffic feature data and network traffic prediction values.

[0055] In the network management center, the collected network original communication data is preprocessed using the network security data set traffic feature extraction tool (CICFlowMeter) to generate network traffic feature data based on the network communication data from all source IPs to destination IPs within a period of time.

[0056] The network traffic characteristic data is processed through digital twin technology to obtain the network traffic prediction value.

[0057] The network traffic feature data and the network traffic prediction value are combined to form 40 data feature types used in this embodiment as shown in Table 1.

[0058] Table 1 Features used in simulation verification

[0059]

[0060]

[0061] Step 202: The network management center performs normalization processing on the acquired network traffic characteristic data and network traffic prediction value. The mathematical representation process is as follows:

[0062] In the formula, x′ represents the result of normalization of the selected feature, x represents the attribute value of the selected feature, and X max Indicates the maximum value of the feature, X min Indicates the minimum value of this feature.

[0063] Step 203: construct a training data set based on the normalized network traffic feature data and the network traffic prediction value.

[0064] 2) Marking of difficult samples based on K clustering algorithm:

[0065] Step 204: Process the training data set using a K clustering algorithm to obtain a difficult-to-separate sample set. In this embodiment, the K-Means algorithm is preferably used.

[0066] The specific process of step 204 includes:

[0067] Step 204 - 1 : At the network management center, a sample is randomly selected from the obtained pre-processed training data set X as an initial cluster center c1 .

[0068] Step 204-2: Calculate the first distance d between all samples in X and the initial cluster center c1 j ,j∈X.

[0069] Step 204-3: According to the formula Calculate the probability p of each sample being selected as the cluster center j .

[0070] Step 204-4: Select K new initial cluster centers c based on probability i ,i∈K, as the first cluster center.

[0071] Step 204-5: Calculate the training data set X sample x j With each cluster center c i The second distance d between ji =||x j -c i || 2 .

[0072] Step 204-6: Determine sample x according to the second distance j Division of: λ j =argmin i＝1,...,K d ji , and divide the samples into corresponding clusters C accordingly λj =C λj ∪{x j}.

[0073] Step 204 - 7 : Calculate the average value of all sample points in each cluster, and use the average value as the new first cluster center, and return to step 204 - 4 .

[0074] Step 204-8: When the preset iteration conditions are met, the clustering results {C1, C2, ..., C k}.

[0075] Step 204-9: Get all clusters {C1, C2, ..., C k} in the attack sample set A k and benign sample set N k .

[0076] Step 204-10: Obtain a difficult sample set based on the attack sample set and the benign sample set:

[0077] H I ={A i}∪{N j},i=arg min k＝1,...,K |A k |,j=arg min k＝1,...,K |N k |.

[0078] 3) Intrusion detection based on deep learning network model:

[0079] Step 205: Select a current state from the training data set, and use a greedy strategy to select the current action corresponding to the current state, wherein the current state represents the network traffic feature data and the network traffic prediction value at the current moment, and the current action represents whether the Internet of Things is invaded at the current moment or whether the Internet of Things is normal at the current moment. In this embodiment, the ε-greedy strategy is preferred.

[0080] Step 206, calculate the current moment reward obtained after executing the current moment action, and obtain the next moment state, wherein the current moment reward is determined based on the reward coefficient, wherein the reward coefficient is a coefficient determined by a first judgment result, and the first judgment result is a result that characterizes that the network traffic feature data and the network traffic prediction value at the current moment belong to the difficult sample set, or the network traffic feature data and the network traffic prediction value at the current moment do not belong to the difficult sample set.

[0081] Step 207: construct a state transition quadruple based on the current state, the current action, the current reward, and the next state, and store the state transition quadruple in the experience replay pool.

[0082] Step 208: Select a number of the state transition quadruple groups from the experience replay pool as training samples to train the deep learning network model.

[0083] Step 209: predict whether the Internet of Things has been invaded based on the trained deep learning network model.

[0084] The deep learning network model in this embodiment can be a convolutional neural network, a generative adversarial network, an autoencoder, or other models.

[0085] Although the above detection method can improve the accuracy of intrusion detection on the Internet of Things network and enhance the intrusion detection performance. However, the inventors also found that due to the characteristics of the Internet of Things: First, various types of nodes are deployed in the Internet of Things, the number and scale of which are far greater than the Internet, and the ubiquitous terminal devices and application processes have generated massive data flows in the Internet of Things network. An excellent network intrusion detection technology must be able to effectively process the massive data in the Internet of Things. Secondly, due to the limited storage and computing resources of the Internet of Things network nodes, traditional intrusion detection technology is not suitable for the Internet of Things network. The more complex network behavior modeling process and the model training that requires more computing resources will cause greater losses to the nodes, affecting the normal operation of the nodes.

[0086] Therefore, on the basis of being able to improve the accuracy of IoT network intrusion detection, in order to further achieve efficient network behavior modeling and lightweight intrusion detection, this embodiment preferably uses the DQN network model as the deep learning network model.

[0087] In order to make those skilled in the art more clear about the intrusion detection process based on the DQN network model in this embodiment, Figure 3 Provide specific explanation.

[0088] (1): In the network management center, DQN is used to perform intrusion detection on the labeled data set (i.e., the training data set including the difficult-to-separate sample set after processing using the K clustering algorithm mentioned above), and the target network θ′ and the online Q network θ and the experience replay pool R are established.

[0089] (2): Get the current state s t ={X′(t),F(t)}, X′(t) represents the network traffic prediction value, and F(t) represents the network traffic characteristics.

[0090] (3): Adopt the ε-greedy strategy and set the exploration rate ε i =ε, according to the exploration ε i Probabilistically randomly select action a t , or select action a t =argmax a Q(s t ,a t ,θ); the environment is based on action a t Feedback reward value r t , and transfer to the next state s t+1 , save the state transfer tuple (s t ,a t ,r t ,s t+1 ) into R. The reward value calculation formula is: Where λ(t) represents the reward coefficient at time t, and r represents the base reward value. The mathematical expression of λ(t) is as follows: Where H I represents the difficult sample set, and υ represents the difficult sample reward coefficient.

[0091] (4): Randomly sample batch tuples from R, and according to the neural network loss function L(θ)=E[r t +γmaxQ(s t+1 ,a t+1 ,θ′)-Q(s t ,a t ,θ)] 2 Update the Q network parameters θ, where Q(s t+1 ,at+1 ,θ′) is the Q value calculated after the target network performs the action at time t+1, Q(s t ,a t ,θ)] is the Q value calculated after the Q network performs the action in the state at time t.

[0092] Every e times of learning, update the target network parameters θ′←θ and update the local exploration rate Where γ is the reward decay factor, To explore the descent step size.

[0093] (5): Set the next state as the current state and repeat steps (2) to (5) M times.

[0094] (6): Output the trained Q network.

[0095] This embodiment introduces digital twin technology to perceive and predict network traffic data, and on this basis uses K-means to mark the hard-to-distinguish samples in the network traffic data. Finally, by controlling the reward value of the Deep Q Network (DQN), the detection accuracy of hard-to-distinguish samples is improved, thus achieving lightweight and high-precision intrusion detection as a whole.

[0096] This embodiment uses five performance indicators, namely accuracy, precision, recall, false alarm rate and F value. Measures ) to evaluate the performance of the proposed intrusion detection method, which is defined as follows:

[0097]

[0098]

[0099]

[0100]

[0101]

[0102] Where TP means that the agent successfully detects the attack sample; TN means that the agent successfully detects the normal sample (i.e., benign sample); FP means that the agent incorrectly detects the attack sample; FN means that the agent incorrectly detects the normal sample.

[0103] This example introduces existing intrusion detection methods and compares them with the proposed method (in this example, the DQN network model) to verify the effectiveness of the intrusion detection method proposed in this example. The methods used for comparison include a stacked contractive auto-encoder and support vector machine (SCAE+SVM) method, a stacked non-symmetric deep auto-encoder (S-NDAE) method, and a vector convolutional deep learning (VCDL) method.

[0104] The SCAE+SVM method uses stacked contraction autoencoders for feature extraction, converting high-dimensional raw feature data into lower-dimensional data, and then uses support vector machines for classification. The S-NDAE method combines stacked asymmetric deep autoencoders with random forests to achieve efficient feature extraction and intrusion detection. The VCDL method uses convolutional neural networks for feature extraction and builds a VCDL model for intrusion detection.

[0105] At the same time, in order to verify the effectiveness of the proposed K-Means and DQN-based intrusion detection method combined with digital twins, this embodiment also uses the K-Means and DQN-based intrusion detection method not combined with digital twin data and the DQN-based method combined with digital twin data as comparisons.

[0106] The key parameters of the proposed method in model training based on the two datasets CSE-CIC-IDS2018 and CIC-DDOS2019 are shown in Tables 2 and 3.

[0107] Table 2 Training parameters based on the CSE-CIC-IDS2018 dataset

[0108]

[0109] Table 3 Training parameters based on the CIC-DDOS2019 dataset

[0110]

[0111]

[0112] The experimental results are shown in Tables 4 and 5.

[0113] Table 4 Evaluation of intrusion detection results based on CSE-CIC-IDS2018

[0114]

[0115] Table 5 Evaluation of intrusion detection results based on CIC-DDOS2019

[0116]

[0117] As shown in Table 4, the method proposed in this embodiment has the highest accuracy, precision, and F-value compared to existing methods. In terms of recall rate, it is slightly lower than the DQN-based intrusion detection method, which shows that the use of the K-Means-based hard-to-distinguish sample labeling method on the CSE-CIC-IDS2018 dataset can effectively improve the accuracy, precision, false alarm rate, and F1 value performance, but will slightly reduce the recall rate performance. Similarly, compared with SCE+SVM, the method proposed in this embodiment surpasses this method in accuracy, precision, false alarm rate, and F1 value, but this method still has an extremely low false alarm rate. The VCDL method and the S-NDAE method performed poorly on the CSE-CIC-IDS2018 dataset and could not achieve effective intrusion detection. Finally, based on the comparison results with the intrusion detection method based on K-Means and DQN without digital twins, it can be clearly seen that after combining the network traffic prediction value obtained by digital twins, the five evaluation performances are significantly improved. This shows that for the CSE-CIC-IDS2018 dataset, the traffic prediction result feature is significantly effective in improving intrusion detection performance.

[0118] As shown in Table 5, the method proposed in this embodiment achieves the highest performance in accuracy, recall, false alarm rate, and F-value, but is slightly lower than the three existing methods in terms of precision. Unlike their performance on the CSE-CIC-IDS2018 dataset, VCDL and S-NDAE perform well on the CIC-DDOS2019 dataset, especially in terms of detection accuracy, with all three existing methods exceeding 97%. However, the three existing methods perform poorly in terms of false alarm rate, exceeding 10%, while the false alarm rate of the method proposed in this embodiment is only 3.13%. By comparing with K-Means and DQN-based intrusion detection methods without digital twins, it can be seen that for the CIC-DDOS2019 dataset, the traffic prediction features proposed in this embodiment and the K-Means-based hard-to-distinguish sample labeling method can effectively improve intrusion detection performance.

[0119] From Tables 4 and 5, it can be seen that the method proposed in this embodiment has better intrusion detection accuracy than existing methods. Furthermore, the network traffic prediction features combined with digital twins and the K-Means-based hard-to-distinguish sample labeling method employed in this invention can effectively improve intrusion detection performance. Furthermore, the experimental results show that the detection accuracy without traffic prediction is lower than that without the hard-to-distinguish sample labeling method. This indirectly reflects that traffic prediction is one of the key features for achieving high-precision intrusion detection.

[0120] Therefore, the intrusion detection method proposed in this embodiment is as follows: the collected network raw communication data is subjected to feature extraction, and traffic prediction is performed through digital twin technology; then the feature data (i.e., including the above-mentioned network traffic feature data and network traffic prediction value) are merged for preprocessing to establish a training data set; the training data set is subjected to sample labeling by the proposed K-Means-based difficult sample labeling method; finally, the labeled data is subjected to intrusion detection using the proposed DQN-based intrusion detection method, thereby achieving high-precision and lightweight intrusion detection of the Internet of Things.

[0121] The present application also provides an application scenario that applies the aforementioned IoT network intrusion detection method. Specifically, the IoT network intrusion detection method provided in this embodiment can be applied in industrial IoT security protection scenarios. In industrial environments, IoT devices are used to monitor and control factory equipment, sensors, robots, etc. The intrusion detection system applies the aforementioned IoT network intrusion detection method of this embodiment to accurately detect whether there are malicious attacks or unauthorized access attempts to the industrial control system, thereby improving the security and reliability of the industrial production process.

[0122] Example 2

[0123] This embodiment provides a computer device, which can be a server or a terminal. Its internal structure diagram can be as follows: Figure 4As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store the required data and the final detection results in the Internet of Things network intrusion detection method. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements an Internet of Things network intrusion detection method in Example 1.

[0124] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0125] Example 3

[0126] This embodiment provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements an Internet of Things network intrusion detection method in Example 1.

[0127] Example 4

[0128] This embodiment provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the method for detecting intrusion into an Internet of Things network in embodiment 1 is implemented.

[0129] Example 5

[0130] This embodiment provides a computer program product, including a computer program, which, when executed by a processor, implements an Internet of Things network intrusion detection method in Example 1.

[0131] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0132] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0133] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0134] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for detecting intrusion in an Internet of Things network, characterized in that: The Internet of Things network intrusion detection method includes: Constructing a training data set based on network traffic feature data and network traffic prediction values, wherein the network traffic feature data is data obtained by extracting features from the network communication data of the Internet of Things, and the network traffic prediction values ​​are data obtained by processing the network traffic feature data using digital twin technology; Using a K clustering algorithm to process the training data set to obtain a difficult-to-classify sample set; Selecting a current state from the training data set, and selecting a current action corresponding to the current state using a greedy strategy, wherein the current state represents the network traffic characteristic data and the network traffic prediction value at the current moment, and the current action represents whether the Internet of Things is invaded at the current moment or whether the Internet of Things is normal at the current moment; Randomly select a current moment action according to the exploration rate, calculate the current moment reward obtained after executing the current moment action, and obtain the next moment state, wherein the current moment reward is determined according to the reward coefficient, wherein the reward coefficient is a coefficient determined by a first judgment result, and the first judgment result is a result indicating that the network traffic feature data and the network traffic prediction value at the current moment belong to the difficult sample set, or a result indicating that the network traffic feature data and the network traffic prediction value at the current moment do not belong to the difficult sample set; the expression of the current moment reward is: Among them, r t (s t ,a t ) is the reward at the current moment t; s t is the current state at time t; a t is the action at the current moment t; λ(t) is the reward coefficient at the current moment t; r is the basic reward; H I is the difficult sample set; υ is the reward coefficient for difficult samples; Constructing a state transition quadruple according to the current state, the current action, the current reward, and the next state, and storing the state transition quadruple in an experience replay pool; Selecting a number of the state transition quadruple from the experience replay pool as training samples to train a deep learning network model; Based on the trained deep learning network model, predict whether the Internet of Things has been invaded.

2. The method for detecting intrusion in an Internet of Things network according to claim 1, wherein: The network traffic characteristic data include: the number of network flows per second, the number of data packets transmitted per second, the total number of bytes of data packet headers transmitted per second, the average number of bytes of data packet headers transmitted per second, the standard deviation of the number of bytes of data packet headers transmitted per second, the total number of bytes of data packet messages transmitted per second, the average number of bytes of data packet messages transmitted per second, the standard deviation of the number of bytes of data packet messages transmitted per second, the number of data packets transmitted in the forward direction per second, the total number of bytes of data packet headers transmitted in the forward direction per second, the average number of bytes of data packet headers transmitted in the forward direction per second, the standard deviation of the number of bytes of data packet headers transmitted in the forward direction per second, the total number of bytes of data packet messages transmitted in the forward direction per second, the average number of bytes of data packet messages transmitted in the forward direction per second, the standard deviation of the number of bytes of data packet messages transmitted in the forward direction per second, the number of data packets transmitted in the reverse direction per second, the total number of bytes of data packet headers transmitted in the reverse direction per second, the average number of bytes of data packet headers transmitted in the reverse direction per second, the standard deviation of the number of data packet messages transmitted in the reverse direction per second Standard deviation of the number of packet header bytes, total number of reverse data packet messages per second, average number of reverse data packet messages per second, standard deviation of the number of reverse data packet messages per second, number of TCP packets per second, total number of TCP packet header bytes per second, average number of TCP packet header bytes per second, standard deviation of the number of TCP packet header bytes per second, total number of TCP packet messages per second, average number of TCP packet messages per second, standard deviation of the number of TCP packet messages per second, number of UDP packets per second, total number of UDP packet header bytes per second, total number of UDP packet messages per second, average number of UDP packet messages per second, standard deviation of the number of UDP packet messages per second, average time interval between data packets, standard time interval between data packets, average time interval between forward data packets, standard time interval between forward data packets, and number of ACK flags.

3. The method for detecting intrusion into an Internet of Things network according to claim 1, wherein: Before executing the step of "constructing a training data set based on network traffic feature data and network traffic prediction values", the IoT network intrusion detection method further includes: The network traffic characteristic data and the network traffic prediction value are normalized.

4. The method for detecting intrusion into an Internet of Things network according to claim 1, wherein: The training data set is processed using a K clustering algorithm to obtain a difficult-to-classify sample set, specifically including: Randomly select a sample from the training data set as an initial cluster center; Calculating a first distance between all samples in the training data set and the initial cluster center; Calculating the probability of each sample being selected as a cluster center according to the first distance; Selecting a number of new initial cluster centers according to the probability as first cluster centers; Calculating a second distance between each of the samples and each of the first cluster centers; Dividing the samples into corresponding clusters according to the second distance; Calculate the average value of all sample points in each cluster, and use the average value as the new first cluster center, and return to step "calculating the second distance between each sample and each first cluster center"; When the preset iteration conditions are met, the clustering results are obtained; Determining an attack sample set and a benign sample set according to the clustering result; The difficult-to-separate sample set is determined according to the attack sample set and the benign sample set.

5. The method for detecting intrusion into an Internet of Things network according to claim 1, wherein: The deep learning network model is a DQN network.

6. The method for detecting intrusion into an Internet of Things network according to claim 5, wherein: The loss function of the deep learning network model is expressed as: L(θ)=E[r t +γmaxQ(s t+1 ,a t+1 ,θ′)-Q(s t ,a t ,i)] 2 ; Among them, r t is the reward at the current moment t; γ is the reward attenuation factor; Q(s t+1 ,a t+1 ,θ′) is the Q value calculated after the target network in the DQN network performs the action at time t+1; s t+1 is the state at time t+1; θ′ is the target network parameter; Q(s t ,a t ,θ) is the Q value calculated by the Q network in the DQN network after executing the action at the current time t; s t is the current state at time t; a t is the action at the current moment t; θ is the Q network parameter.

7. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement an Internet of Things network intrusion detection method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, an Internet of Things network intrusion detection method according to any one of claims 1 to 6 is implemented.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, an Internet of Things network intrusion detection method according to any one of claims 1 to 6 is implemented.

Images