Internet of things network intrusion detection method, equipment, medium and product
The integration of digital twin technology, K-means clustering, and deep reinforcement learning improves IoT network intrusion detection accuracy and efficiency by processing large data volumes and reducing resource demands.
Patent Information
- Application Number
- CN202411282290.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-13
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-09-13
AI Technical Summary
Current network intrusion detection technologies in the Internet of Things face challenges such as high false alarm rates, low detection accuracy, and difficulty in handling complex network environments with evolving attack methods, especially due to large data volumes and resource constraints.
A method combining digital twin technology, K-means clustering, and deep reinforcement learning (DQN) to construct a training dataset, identify hard-to-separate samples, and improve detection accuracy by using a reward-based deep learning model.
Enhances intrusion detection accuracy and efficiency by effectively processing large data volumes and reducing resource demands, achieving high-precision and lightweight intrusion detection in IoT networks.
Smart Images

Figure CN119094216A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of network intrusion detection technology, and in particular to an Internet of Things network intrusion detection method, device, medium and product based on deep reinforcement learning and K clustering. Background Art
[0002] The Internet of Things is "the Internet of Things that connects things to things". The Internet of Things connects various objects (such as sensors, smart devices, etc.) through the Internet, constantly senses and processes changes in the state information of objects, and ultimately realizes a series of automated operations such as intelligent management and monitoring of objects. The continuous expansion of the scale of the Internet of Things has made the network security issues of the Internet of Things increasingly prominent. How to ensure the security of the Internet of Things, ensure the rapid development of the Internet of Things, and further provide people with safe, reliable, intelligent and efficient services is a problem that needs to be solved urgently. As an active defense method, intrusion detection plays a vital role in improving the network and information security of the Internet of Things by actively detecting the current environment in real time, alarming malicious behaviors in the current environment and taking further corresponding measures. It has become a current research hotspot.
[0003] In the current field of network intrusion detection in the Internet of Things, researchers have conducted a lot of research work and achieved certain results. With the development of artificial intelligence technology, the efficiency of IDS has been further improved. The scope of artificial intelligence technology applicable to network intrusion detection is very wide. Various types of machine learning technologies, such as random forest, principal component analysis method, support vector machine method, etc., have been widely used to detect abnormal behavior patterns in the network. However, traditional machine learning has problems such as high false alarm rate, low detection rate, and difficulty in detection due to large data volume.
[0004] In addition, the current application fields of the Internet of Things are very broad. The wide range of application scenarios means that the network system structure and network communication process of the Internet of Things will become more and more complex, and with the development of network technology and artificial intelligence technology, network attack methods are also constantly improving. In the complex Internet of Things network environment, facing the same constantly iterative and updated network attack methods, high accuracy is one of the main challenges of intrusion detection technology in the current Internet of Things network. Summary of the invention
[0005] The purpose of this application is to provide an Internet of Things network intrusion detection method, equipment, medium and product, which can effectively improve the accuracy of Internet of Things network intrusion detection and enhance the intrusion detection performance.
[0006] To achieve the above objectives, this application provides the following solutions:
[0007] In a first aspect, the present application provides an IoT network intrusion detection method, comprising:
[0008] Constructing a training data set according to network traffic characteristic data and network traffic prediction values, wherein the network traffic characteristic data is data obtained by extracting features from network communication data of the Internet of Things, and the network traffic prediction value is data obtained by processing the network traffic characteristic data using digital twin technology;
[0009] Using a K clustering algorithm to process the training data set to obtain a difficult-to-separate sample set;
[0010] Select a current state from the training data set, and use a greedy strategy to select a current action corresponding to the current state, wherein the current state represents the network traffic feature data and the network traffic prediction value at the current moment, and the current action represents that the Internet of Things is invaded at the current moment or that the Internet of Things is normal at the current moment;
[0011] Calculate the current moment reward obtained after executing the current moment action, and obtain the next moment state, wherein the current moment reward is determined according to the reward coefficient, wherein the reward coefficient is a coefficient determined by a first judgment result, and the first judgment result is a result that indicates that the network traffic feature data and the network traffic prediction value at the current moment belong to the difficult sample set, or that the network traffic feature data and the network traffic prediction value at the current moment do not belong to the difficult sample set;
[0012] Constructing a state transition quadruple according to the current state, the current action, the current reward and the next state, and storing the state transition quadruple in an experience replay pool;
[0013] Selecting a number of the state transition quadruplets from the experience replay pool as training samples to train a deep learning network model;
[0014] Based on the trained deep learning network model, predict whether the Internet of Things has been invaded.
[0015] Optionally, the network traffic characteristic data includes: the number of network flows per second, the number of data packets transmitted per second, the total number of bytes of data packet headers transmitted per second, the average number of bytes of data packet headers transmitted per second, the standard deviation of the number of bytes of data packet headers transmitted per second, the total number of bytes of data packet messages transmitted per second, the average number of bytes of data packet messages transmitted per second, the standard deviation of the number of bytes of data packet messages transmitted per second, the number of data packets transmitted forward per second, the total number of bytes of data packet headers transmitted forward per second, the average number of bytes of data packet headers transmitted forward per second, the standard deviation of the number of data packet header bytes transmitted forward per second, the total number of bytes of data packet messages transmitted forward per second, the average number of bytes of data packet messages transmitted forward per second, the standard deviation of the number of data packet messages transmitted forward per second, the number of data packets transmitted in the reverse direction per second, the total number of bytes of data packet headers transmitted in the reverse direction per second, the average number of bytes of data packet headers transmitted in the reverse direction per second, the standard deviation of the number of data packet header bytes transmitted in the reverse direction per second Standard deviation of the number of bytes in the transmitted data packet header, total number of bytes in the reverse data packet message per second, average number of bytes in the reverse data packet message per second, standard deviation of the number of bytes in the reverse data packet message per second, number of TCP data packets per second, total number of bytes in the TCP data packet header per second, average number of bytes in the TCP data packet header per second, standard deviation of the number of bytes in the TCP data packet header per second, total number of bytes in the TCP data packet message per second, average number of bytes in the TCP data packet message per second, standard deviation of the number of bytes in the TCP data packet message per second, number of UDP data packets per second, total number of bytes in the UDP data packet header per second, total number of bytes in the UDP data packet message per second, average number of bytes in the UDP data packet message per second, standard deviation of the number of bytes in the UDP data packet message per second, average time interval of data packets, standard time interval of data packets, average time interval of forward data packets, standard time interval of forward data packets and number of ACK tags.
[0016] Optionally, before executing the step of “constructing a training data set according to network traffic characteristic data and network traffic prediction values”, the Internet of Things network intrusion detection method further includes:
[0017] The network traffic characteristic data and the network traffic prediction value are normalized.
[0018] Optionally, the training data set is processed using a K clustering algorithm to obtain a difficult-to-separate sample set, specifically including:
[0019] Randomly select a sample from the training data set as an initial cluster center;
[0020] Calculating a first distance between all samples in the training data set and the initial cluster center;
[0021] Calculating the probability of each sample being selected as a cluster center according to the first distance;
[0022] Selecting a plurality of new initial cluster centers according to the probability as first cluster centers;
[0023] Calculating a second distance between each of the samples and each of the first cluster centers;
[0024] Dividing the samples into corresponding clusters according to the second distance;
[0025] Calculate the average value of all sample points in each cluster, and use the average value as the new first cluster center, and return to the step of "calculating the second distance between each of the samples and each of the first cluster centers";
[0026] When the preset iteration conditions are met, the clustering results are obtained;
[0027] Determining an attack sample set and a benign sample set according to the clustering result;
[0028] The difficult sample set is determined according to the attack sample set and the benign sample set.
[0029] Optionally, the expression of the current moment reward is:
[0030]
[0031]
[0032] Among them, r t (s t ,a t ) is the reward at the current moment t; s t is the current state at time t; a t is the action at the current time t; λ(t) is the reward coefficient at the current time t; r is the basic reward; H I is the difficult sample set; υ is the difficult sample reward coefficient.
[0033] Optionally, the deep learning network model is a DQN network.
[0034] Optionally, the loss function of the deep learning network model is expressed as:
[0035] L(θ)=E[r t +γmaxQ(s t+1 ,a t+1 ,θ′)-Q(s t ,a t ,θ)] 2 ;
[0036] Among them, r t is the reward at the current time t; γ is the reward attenuation factor; Q(s t+1 ,a t+1,θ′) is the Q value calculated after the target network in the DQN network performs the action at the state of time t+1; s t+1 is the state at time t+1; θ′ is the target network parameter; Q(s t ,a t ,θ) is the Q value calculated by the Q network in the DQN network after executing the action at the current time t; s t is the current state at time t; a t is the action at the current moment t; θ is the Q network parameter.
[0037] In a second aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement an Internet of Things network intrusion detection method described in any one of the first aspects above.
[0038] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an Internet of Things network intrusion detection method as described in any one of the first aspects above.
[0039] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements an Internet of Things network intrusion detection method described in any one of the first aspects above.
[0040] According to the specific embodiments provided in this application, this application discloses the following technical effects:
[0041] The present application provides an Internet of Things network intrusion detection method, device, medium and product. By introducing digital twin technology to perceive and predict network traffic data, a training data set is constructed in combination with network traffic feature data. The hard-to-distinguish samples in the training data set are then marked using a K clustering algorithm. Finally, the detection accuracy of hard-to-distinguish samples is improved by controlling the reward value of a deep learning network model, thereby achieving high-precision intrusion detection as a whole. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0043] Figure 1 This is an application environment diagram of an Internet of Things network intrusion detection method in Example 1 of the present application;
[0044] Figure 2 A flowchart of an Internet of Things network intrusion detection method provided in Example 1 of the present application;
[0045] Figure 3 Schematic diagram of the intrusion detection process based on DQN in Example 1 of the present application;
[0046] Figure 4 A schematic diagram of the structure of a computer device provided in Example 2 of the present application. DETAILED DESCRIPTION
[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0048] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.
[0049] Example 1
[0050] The present invention provides an Internet of Things network intrusion detection method, which can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set up separately, integrated on the server 104, or placed on the cloud or other servers. The terminal 102 can send the original network communication data of the Internet of Things to the server 104. After the server 104 receives the network communication data, it extracts the characteristics of the network communication data to obtain network traffic characteristic data, and uses the digital twin technology to process the network traffic characteristic data to obtain the network traffic prediction value; secondly, the K clustering algorithm is used to process the training data set including the network traffic characteristic data and the network traffic prediction value to obtain a difficult sample set; then a deep learning network model is constructed; then a current state is selected from the training data set, and the current action corresponding to the current state is selected using a greedy strategy, and the current reward obtained after executing the current action is calculated, and the next state is obtained, so as to obtain a state transition quadruple; finally, a deep learning network model is trained using several state transition quadruple, and a detection result is output when the deep learning network model training is completed, that is, whether the Internet of Things is invaded. The server 104 may feed back the obtained detection result to the terminal 102. In addition, in some embodiments, the Internet of Things network intrusion detection method may also be implemented by the server 104 or the terminal 102 alone, such as the terminal 102 may directly use the Internet of Things network intrusion detection method to detect the network communication data, or the server 104 may obtain the network communication data from the data storage system and use the Internet of Things network intrusion detection method to detect the network communication data.
[0051] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.
[0052] In an exemplary embodiment, Figure 2 As shown, this embodiment provides an IoT network intrusion detection method, which is executed by a computer device, and can be executed by a computer device such as a terminal or a server alone, or by a terminal and a server together. In the embodiment of the present application, the method is applied to Figure 1 The server 104 in the example is used as an example to illustrate, including the following steps 1) to 3). Among them:
[0053] 1) Build a training dataset:
[0054] Step 201: extract features from network communication data and process them using digital twin technology to obtain network traffic feature data and network traffic prediction values.
[0055] In the network management center, the network security data set traffic feature extraction tool (CICFlowMeter) is used to pre-process the collected network original communication data to generate network traffic feature data based on the network communication data from all source IPs to the destination IPs within a period of time.
[0056] The network traffic characteristic data is processed through digital twin technology to obtain the network traffic prediction value.
[0057] The network traffic feature data and the network traffic prediction value are combined to form 40 data feature types used in this embodiment as shown in Table 1.
[0058] Table 1 Features used in simulation verification
[0059]
[0060]
[0061] Step 202: The obtained network traffic characteristic data and network traffic prediction value are normalized in the network management center. The mathematical representation process is as follows:
[0062] In the formula, x′ represents the result of normalization of the selected feature, x represents the attribute value of the selected feature, and X max represents the maximum value of the feature, X min Indicates the minimum value of this feature.
[0063] Step 203: construct a training data set based on the normalized network traffic feature data and the network traffic prediction value.
[0064] 2) Labeling of difficult samples based on K clustering algorithm:
[0065] Step 204: Use a K clustering algorithm to process the training data set to obtain a difficult-to-separate sample set. In this embodiment, the K-Means algorithm is preferably used.
[0066] The specific process of step 204 includes:
[0067] Step 204 - 1 : A sample is randomly selected from the obtained pre-processed training data set X at the network management center as an initial cluster center c1 .
[0068] Step 204-2: Calculate the first distance d between all samples in X and the initial cluster center c1 j ,j∈X.
[0069] Step 204-3: According to the formula Calculate the probability p of each sample being selected as the cluster center j .
[0070] Step 204-4: Select K new initial cluster centers c according to probability i ,i∈K, as the first cluster center.
[0071] Step 204-5: Calculate the training data set X sample x j With each cluster center c i The second distance d between ji =||x j -c i || 2 .
[0072] Step 204-6: Determine sample x according to the second distance j Division of: λ j =argmin i=1,...,K d ji , and divide the samples into corresponding clusters C accordingly λj =C λj ∪{x j}.
[0073] Step 204 - 7 : Calculate the average value of all sample points in each cluster, and use the average value as the new first cluster center, and return to step 204 - 4 .
[0074] Step 204-8: When the preset iteration conditions are met, the clustering results {C1, C2, ..., C k}.
[0075] Step 204-9: Get all clusters {C1, C2, ..., C k} in the attack sample set A k and benign sample set N k .
[0076] Step 204-10: Obtain a difficult sample set based on the attack sample set and the benign sample set:
[0077] H I ={A i}∪{N j},i=arg min k=1,...,K |A k |,j=arg min k=1,...,K |N k |.
[0078] 3) Intrusion detection based on deep learning network model:
[0079] Step 205, select a current state from the training data set, and use a greedy strategy to select the current action corresponding to the current state, wherein the current state represents the network traffic feature data and the network traffic prediction value at the current moment, and the current action represents that the Internet of Things is invaded at the current moment or the Internet of Things is normal at the current moment. This embodiment preferably uses an ε-greedy strategy.
[0080] Step 206, calculate the current moment reward obtained after executing the current moment action, and obtain the next moment state, wherein the current moment reward is determined according to the reward coefficient, wherein the reward coefficient is a coefficient determined by a first judgment result, and the first judgment result is a result that characterizes that the network traffic feature data and the network traffic prediction value at the current moment belong to the difficult sample set, or the network traffic feature data and the network traffic prediction value at the current moment do not belong to the difficult sample set.
[0081] Step 207, constructing a state transition quadruple according to the current state, the current action, the current reward and the next state, and storing the state transition quadruple in the experience replay pool.
[0082] Step 208: Select a number of the state transition quadruple groups from the experience replay pool as training samples to train the deep learning network model.
[0083] Step 209: predict whether the Internet of Things is invaded based on the trained deep learning network model.
[0084] The deep learning network model in this embodiment can be a convolutional neural network, a generative adversarial network, an autoencoder or other models.
[0085] Although the above detection method can improve the accuracy of intrusion detection on the Internet of Things network and enhance the intrusion detection performance. However, the inventors also found that due to the characteristics of the Internet of Things: First, various types of nodes are deployed in the Internet of Things, and their number and scale are far larger than the Internet, and the ubiquitous terminal devices and application processes have generated massive data flows in the Internet of Things network. An excellent network intrusion detection technology must be able to effectively process the massive data in the Internet of Things. Secondly, due to the limited storage and computing resources of the Internet of Things network nodes, traditional intrusion detection technology is not suitable for the Internet of Things network. The more complex network behavior modeling process and the model training that requires more computing resources will cause greater losses to the nodes and affect the normal operation of the nodes.
[0086] Therefore, on the basis of being able to improve the accuracy of IoT network intrusion detection, in order to further achieve efficient network behavior modeling and lightweight intrusion detection, this embodiment preferably uses the DQN network model as the deep learning network model.
[0087] In order to make those skilled in the art more clear about the intrusion detection process based on the DQN network model in this embodiment, Figure 3 Provide specific explanation.
[0088] (1): In the network management center, DQN is used to perform intrusion detection on the labeled data set (i.e., the training data set including the difficult sample set after being processed by the K clustering algorithm mentioned above), and the target network θ′ and the online Q network θ and the experience replay pool R are established.
[0089] (2): Get the current state s t ={X′(t),F(t)}, X′(t) represents the network traffic prediction value, and F(t) represents the network traffic characteristics.
[0090] (3): Adopt the ε-greedy strategy and set the exploration rate ε i =ε, according to the exploration of ε i Probability of randomly selecting action a t , or select action a t = argmax a Q(s t ,a t ,θ); the environment is based on action a t Feedback reward value r t , and transfer to the next state s t+1 , save the state transfer tuple (s t ,a t ,r t ,s t+1 ) into R. The reward value calculation formula is: Where λ(t) represents the reward value coefficient at time t, and r represents the basic reward value. The mathematical expression of λ(t) is as follows: Where H I represents the difficult sample set, and υ represents the difficult sample reward coefficient.
[0091] (4): Randomly sample batch tuples from R, according to the neural network loss function L(θ) = E[r t +γmaxQ(s t+1 ,a t+1 ,θ′)-Q(s t ,a t ,θ)] 2 Update the Q network parameters θ, where Q(s t+1 ,at+1 ,θ′) is the Q value calculated after the target network performs the action at time t+1, Q(s t ,a t ,θ)] is the Q value calculated after the Q network performs an action in the state at time t.
[0092] Every e learning times, update the target network parameters θ′←θ and update the local exploration rate Where γ is the reward decay factor, To explore the decreasing step size.
[0093] (5): Set the next state as the current state and repeat steps (2) to (5) M times.
[0094] (6): Output the trained Q network.
[0095] This embodiment introduces digital twin technology to perceive and predict network traffic data, and on this basis uses K-means to mark the hard-to-distinguish samples in the network traffic data. Finally, by controlling the reward value of the Deep Q Network (DQN) to improve the detection accuracy of hard-to-distinguish samples, lightweight and high-precision intrusion detection is achieved as a whole.
[0096] This embodiment uses five performance indicators, namely accuracy, precision, recall, false alarm rate, and F value. Measures ) evaluates the performance of the proposed intrusion detection method, which is defined as follows:
[0097]
[0098]
[0099]
[0100]
[0101]
[0102] In the formula, TP means that the agent successfully detects the attack sample; TN means that the agent successfully detects the normal sample (i.e., benign sample); FP means that the agent incorrectly detects the attack sample; FN means that the agent incorrectly detects the normal sample.
[0103] This embodiment introduces the existing intrusion detection method and compares it with the proposed method (in this embodiment, the DQN network model) to verify the effectiveness of the intrusion detection method proposed in this embodiment. The methods used for comparison are the method combining stacked contractive auto-encoder and support vector machine (SCAE+SVM), the stacked non-symmetric deep auto-encoder (S-NDAE) method and the vector convolutional deep learning (VCDL) method.
[0104] Among them, the SCAE+SVM method uses superimposed contraction autoencoders for feature extraction, converts high-dimensional original feature data into lower-dimensional data, and then uses the support vector machine method to achieve classification. The S-NDAE method combines superimposed asymmetric deep autoencoders with the random forest method to achieve efficient feature extraction and intrusion detection. The VCDL method realizes feature extraction based on convolutional neural networks and realizes intrusion detection by building a VCDL model.
[0105] At the same time, in order to verify the effectiveness of the proposed intrusion detection method based on K-Means and DQN combined with digital twins, this embodiment also uses the intrusion detection method based on K-Means and DQN without combining digital twin data and the DQN-based method combined with digital twin data as comparisons.
[0106] The key parameters of the proposed method in model training based on the two datasets CSE-CIC-IDS2018 and CIC-DDOS2019 are shown in Tables 2 and 3.
[0107] Table 2 Training parameters based on CSE-CIC-IDS2018 dataset
[0108]
[0109] Table 3 Training parameters based on the CIC-DDOS2019 dataset
[0110]
[0111]
[0112] The experimental results are shown in Tables 4 and 5.
[0113] Table 4 Evaluation of intrusion detection results based on CSE-CIC-IDS2018
[0114]
[0115] Table 5 Evaluation of intrusion detection results based on CIC-DDOS2019
[0116]
[0117] As shown in Table 4, the method proposed in this embodiment has the highest accuracy, precision and F value compared with the existing methods. The recall rate is slightly lower than the DQN-based intrusion detection method, which shows that on the CSE-CIC-IDS2018 dataset, the use of the K-Means-based difficult sample labeling method can effectively improve the accuracy, precision, false alarm rate and F1 value, but will slightly reduce the recall rate. Similarly, compared with SCE+SVM, the method proposed in this embodiment surpasses this method in accuracy, precision, false alarm rate and F1 value, but the method still has a very low false alarm rate. The VCDL method and the S-NDAE method performed poorly on the CSE-CIC-IDS2018 dataset and could not achieve effective intrusion detection. Finally, according to the comparison results with the intrusion detection methods based on K-Means and DQN without digital twins, it can be clearly seen that after combining the network traffic prediction values obtained by digital twins, the five evaluation performances are significantly improved. This shows that for the CSE-CIC-IDS2018 dataset, the traffic prediction result feature is effective in improving the intrusion detection performance.
[0118] As shown in Table 5, the method proposed in this embodiment has the highest performance in accuracy, recall rate, false alarm rate and F value, but is slightly lower than the three existing methods in terms of precision. Different from the performance on the CSE-CIC-IDS2018 dataset, VCDL and S-NDAE have better performance on the CIC-DDOS2019 dataset, especially in terms of detection accuracy. The three existing methods all exceed 97%, but the three existing methods perform poorly in terms of false alarm rate, all exceeding 10%, while the false alarm rate of the method proposed in this embodiment is only 3.13%. By comparing with the intrusion detection methods based on K-Means and DQN without digital twins, it can be seen that for the CIC-DDOS2019 dataset, the traffic prediction features proposed in this embodiment and the hard-to-distinguish sample marking method based on K-Means can effectively improve the intrusion detection performance.
[0119] It can be seen from Table 4 and Table 5 that the method proposed in this embodiment has better intrusion detection accuracy than the existing method, and the network traffic prediction feature combined with digital twins and the hard-to-distinguish sample marking method based on K-Means adopted by the present invention can effectively improve the performance of intrusion detection. At the same time, by observing the experimental results, it can be seen that the detection accuracy without combining traffic prediction is lower than the detection accuracy without using the hard-to-distinguish sample marking method, which reflects from the side that the traffic prediction feature is one of the important features for achieving high-precision intrusion detection.
[0120] Therefore, the intrusion detection method proposed in this embodiment is as follows: extracting features from the collected network raw communication data and predicting traffic through digital twin technology; then merging the feature data (i.e., including the above-mentioned network traffic feature data and network traffic prediction values) for preprocessing to establish a training data set; marking the training data set using the proposed K-Means-based difficult sample marking method; finally, using the proposed DQN-based intrusion detection method to perform intrusion detection on the marked data, thereby achieving high-precision and lightweight intrusion detection of the Internet of Things.
[0121] The present application also provides an application scenario, which applies the above-mentioned IoT network intrusion detection method. Specifically: The IoT network intrusion detection method provided in this embodiment can be applied in the industrial IoT security protection scenario. In an industrial environment, IoT devices are used to monitor and control factory equipment, sensors, robots, etc. The intrusion detection system applies the above-mentioned IoT network intrusion detection method of this embodiment to accurately detect whether there is a malicious attack or unauthorized access attempt to the industrial control system, thereby improving the safety and reliability of the industrial production process.
[0122] Example 2
[0123] This embodiment provides a computer device, which may be a server or a terminal. Its internal structure diagram may be as follows: Figure 4As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the required data and the final detection results in the Internet of Things network intrusion detection method. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an Internet of Things network intrusion detection method in Example 1 is implemented.
[0124] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0125] Example 3
[0126] This embodiment provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, an Internet of Things network intrusion detection method in Embodiment 1 is implemented.
[0127] Example 4
[0128] This embodiment provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements an Internet of Things network intrusion detection method in Embodiment 1.
[0129] Example 5
[0130] This embodiment provides a computer program product, including a computer program, which, when executed by a processor, implements an Internet of Things network intrusion detection method in Embodiment 1.
[0131] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0132] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0133] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.
[0134] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0135] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.
Claims
1. A method for detecting intrusion in an Internet of Things network, characterized in that: The Internet of Things network intrusion detection method comprises: Constructing a training data set according to network traffic characteristic data and network traffic prediction values, wherein the network traffic characteristic data is data obtained by extracting features from network communication data of the Internet of Things, and the network traffic prediction value is data obtained by processing the network traffic characteristic data using digital twin technology; Using a K clustering algorithm to process the training data set to obtain a difficult-to-separate sample set; Select a current state from the training data set, and use a greedy strategy to select a current action corresponding to the current state, wherein the current state represents the network traffic feature data and the network traffic prediction value at the current moment, and the current action represents that the Internet of Things is invaded at the current moment or that the Internet of Things is normal at the current moment; Calculate the current moment reward obtained after executing the current moment action, and obtain the next moment state, wherein the current moment reward is determined according to the reward coefficient, wherein the reward coefficient is a coefficient determined by a first judgment result, and the first judgment result is a result that indicates that the network traffic feature data and the network traffic prediction value at the current moment belong to the difficult sample set, or that the network traffic feature data and the network traffic prediction value at the current moment do not belong to the difficult sample set; Constructing a state transition quadruple according to the current state, the current action, the current reward and the next state, and storing the state transition quadruple in an experience replay pool; Selecting a number of the state transition quadruplets from the experience replay pool as training samples to train a deep learning network model; Based on the trained deep learning network model, predict whether the Internet of Things has been invaded.
2. The method for detecting intrusion in an Internet of Things network according to claim 1, characterized in that: The network traffic characteristic data include: the number of network flows per second, the number of data packets transmitted per second, the total number of bytes of data packet headers transmitted per second, the average number of bytes of data packet headers transmitted per second, the standard deviation of the number of bytes of data packet headers transmitted per second, the total number of bytes of data packet messages transmitted per second, the average number of bytes of data packet messages transmitted per second, the standard deviation of the number of bytes of data packet messages transmitted per second, the number of data packets transmitted forward per second, the total number of bytes of data packet headers transmitted forward per second, the average number of bytes of data packet headers transmitted forward per second, the standard deviation of the number of bytes of data packet headers transmitted forward per second, the total number of bytes of data packet messages transmitted forward per second, the average number of bytes of data packet messages transmitted forward per second, the standard deviation of the number of bytes of data packet messages transmitted forward per second, the number of data packets transmitted reversely per second, the total number of bytes of data packet headers transmitted reversely per second, the average number of bytes of data packet headers transmitted reversely per second, the standard deviation of the number of data packet header bytes transmitted reversely per second The standard deviation of the number of bytes in the packet header, the total number of bytes of data packets transmitted in the reverse direction per second, the average number of bytes of data packets transmitted in the reverse direction per second, the standard deviation of the number of bytes of data packets transmitted in the reverse direction per second, the number of TCP data packets per second, the total number of bytes of TCP data packet headers per second, the average number of bytes of TCP data packet headers per second, the standard deviation of the number of bytes of TCP data packet headers per second, the total number of bytes of TCP data packet messages per second, the average number of bytes of TCP data packet messages per second, the standard deviation of the number of bytes of TCP data packet messages per second, the number of UDP data packets per second, the total number of bytes of UDP data packet headers per second, the total number of bytes of UDP data packet messages per second, the average number of bytes of UDP data packet messages per second, the standard deviation of the number of bytes of UDP data packet messages per second, the average time interval of data packets, the standard time interval of data packets, the average time interval of forward data packets, the standard time interval of forward data packets and the number of ACK marks.
3. The method for detecting intrusion in an Internet of Things network according to claim 1, characterized in that: Before executing the step of "constructing a training data set according to network traffic characteristic data and network traffic prediction values", the Internet of Things network intrusion detection method further includes: The network traffic characteristic data and the network traffic prediction value are normalized.
4. The method for detecting intrusion in an Internet of Things network according to claim 1, characterized in that: The training data set is processed using a K clustering algorithm to obtain a difficult-to-separate sample set, specifically including: Randomly select a sample from the training data set as an initial cluster center; Calculating a first distance between all samples in the training data set and the initial cluster center; Calculating the probability of each sample being selected as a cluster center according to the first distance; Selecting a number of new initial cluster centers according to the probability as first cluster centers; Calculating a second distance between each of the samples and each of the first cluster centers; Dividing the samples into corresponding clusters according to the second distance; Calculate the average value of all sample points in each cluster, and use the average value as the new first cluster center, and return to the step of "calculating the second distance between each of the samples and each of the first cluster centers"; When the preset iteration conditions are met, the clustering results are obtained; Determining an attack sample set and a benign sample set according to the clustering result; The difficult sample set is determined according to the attack sample set and the benign sample set.
5. The method for detecting intrusion in an Internet of Things network according to claim 1, characterized in that: The expression of the current moment reward is: Among them, r t (s t ,a t ) is the reward at the current moment t; s t is the current state at time t; a t is the action at the current time t; λ(t) is the reward coefficient at the current time t; r is the basic reward; H I is the difficult sample set; υ is the difficult sample reward coefficient.
6. The method for detecting intrusion in an Internet of Things network according to claim 1, characterized in that: The deep learning network model is a DQN network.
7. The method for detecting intrusion in an Internet of Things network according to claim 6, characterized in that: The expression of the loss function of the deep learning network model is: L(θ)=E[r t +γmaxQ(s t+1 ,a t+1 ,θ′)-Q(s t ,a t ,i)] 2 ; Among them, r t is the reward at the current time t; γ is the reward attenuation factor; Q(s t+1 ,a t+1 ,θ′) is the Q value calculated after the target network in the DQN network performs the action at the state of time t+1; s t+1 is the state at time t+1; θ′ is the target network parameter; Q(s t ,a t ,θ) is the Q value calculated by the Q network in the DQN network after executing the action at the current time t; s t is the current state at time t; a t is the action at the current moment t; θ is the Q network parameter.
8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement an Internet of Things network intrusion detection method as described in any one of claims 1 to 7.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, an Internet of Things network intrusion detection method described in any one of claims 1 to 7 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, an Internet of Things network intrusion detection method described in any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Training method and device of network intrusion detection model
CN111741002A
Network intrusion detection method based on unsupervised and supervised fusion reinforcement learning
CN116633639A
Power information system network intrusion detection method
CN117336034A
Method and system for image registration using an intelligent artificial agent
EP3246875A2
Cited By
Industrial Internet of Things intrusion detection system based on conditional variation auto-encoder
CN117201058A
