Abnormal flow data detection method, system and device and medium
By using an abnormal traffic detection model in the Internet of Things environment, the problem of traffic data in the existing technology that relying on manual tags is difficult to adapt to changes is solved, and efficient detection and identification of abnormal traffic data in the Internet of Things environment is achieved.
Patent Information
- Application Number
- CN202411515273.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-29
- Publication Date
- 2025-02-14
AI Technical Summary
When detecting abnormal traffic data in an IoT environment, it is difficult to adapt to changing and evolving traffic data by relying on manual tags.
An abnormal traffic detection model is adopted, which includes a coding layer, a time domain convolutional network layer, an attention mechanism layer and a decoding layer. The traffic data is characterized by self-supervised deep learning and the abnormal traffic data is distinguished through pre-subsidized data.
The identification of the abnormal traffic data of new categories and the effective detection of traffic data of known categories are realized, and the computing power and memory are fixed or slowly increasing with the increase of the number of categories.
Smart Images
Figure CN119449399A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of Internet of Things, and in particular to a method, system, device and medium for detecting abnormal flow data. Background Art
[0002] Abnormal IoT traffic generally refers to abnormal behaviors or traffic patterns that occur in IoT devices and networks. These behaviors or patterns usually indicate potential security threats or device failures. Detection and analysis of these abnormal traffic data are critical to maintaining the security of the IoT environment.
[0003] In the process of detecting abnormal traffic data in the Internet of Things, the supervised deep learning method can be used to detect abnormal traffic data from the data set. However, this method relies on manual labels and is difficult to adapt to the constantly changing and evolving traffic data in the Internet of Things environment. Summary of the invention
[0004] Aiming at the problem in the prior art that manual labels are difficult to adapt to the constantly changing flow data, the present invention proposes a method, system, device and medium for detecting abnormal flow data.
[0005] In a first aspect, an embodiment of the present invention provides a method for detecting abnormal traffic data, the detection method comprising:
[0006] Get real-time traffic data;
[0007] Inputting the real-time traffic data into the abnormal traffic detection model to obtain target reconstructed data of the real-time traffic data, the abnormal traffic detection model comprising an encoding layer, at least one time-domain convolutional network layer, at least one attention mechanism layer and a decoding layer arranged in sequence, each time-domain convolutional network layer of the at least one time-domain convolutional network layer and each attention mechanism layer of the at least one attention mechanism layer are alternately stacked;
[0008] Whether the real-time traffic data is abnormal traffic data is determined based on the comparison result between the target reconstructed data and the threshold data.
[0009] Preferably, the detection method further comprises:
[0010] Obtain normal traffic data and abnormal traffic data;
[0011] Inputting the normal traffic data and the abnormal traffic data into the abnormal traffic detection model, and obtaining first reconstructed data corresponding to the normal traffic data and second reconstructed data corresponding to the abnormal traffic data respectively;
[0012] The threshold data is determined based on the comparison result of the first reconstructed data and the second reconstructed data.
[0013] Preferably, the detection method further comprises a step of training an abnormal traffic detection model;
[0014] The training steps of the abnormal traffic detection model include:
[0015] Acquire normal traffic data and a basic model, where the basic model includes an encoding layer, at least one time-domain convolutional network layer, at least one attention mechanism layer, and a decoding layer arranged in sequence, and each time-domain convolutional network layer of the at least one time-domain convolutional network layer and each attention mechanism layer of the at least one attention mechanism layer are alternately stacked;
[0016] Input normal traffic data into the basic model for iteration until the loss function of the basic model iterates to the minimum value;
[0017] The trained basic model is determined as the abnormal traffic detection model.
[0018] Preferably, the encoding layer is used to encode the traffic data into a latent vector;
[0019] The temporal convolutional network layer is used to capture the long-term dependencies of the latent vectors;
[0020] The attention mechanism layer is used to enhance the features of the latent vector;
[0021] The decoding layer is used to decode the latent vector into reconstructed data.
[0022] Preferably, the loss function includes at least one of a reconstruction loss function and a KL divergence loss function.
[0023] Preferably, the training steps of the abnormal traffic detection model also include:
[0024] During the iteration of the basic model, when the iteration rounds of the basic model meet the tolerance rounds, the trained basic model is determined as the abnormal traffic detection model;
[0025] and / or, during the iteration of the basic model, when the accuracy of the basic model meets the accuracy threshold, the trained basic model is determined as the abnormal traffic detection model;
[0026] And / or, during the iteration of the basic model, the parameters of the basic model are updated by the optimizer.
[0027] Preferably, the encoding layer further includes a normal mixture model, which is used to obtain a latent vector of a multi-modal distribution of real-time traffic data;
[0028] And / or, the abnormal traffic detection model further includes a normalized flow layer after the encoding layer, and the normalized flow layer is used to perform multi-layer flow transformation on the potential vector obtained by the encoding layer.
[0029] In a second aspect, an embodiment of the present invention provides a system for detecting abnormal traffic data, the detection system comprising:
[0030] Get real-time traffic data;
[0031] Inputting the real-time traffic data into the abnormal traffic detection model to obtain target reconstructed data of the real-time traffic data, the abnormal traffic detection model comprising an encoding layer, at least one time-domain convolutional network layer, at least one attention mechanism layer and a decoding layer arranged in sequence, each time-domain convolutional network layer of the at least one time-domain convolutional network layer and each attention mechanism layer of the at least one attention mechanism layer are alternately stacked;
[0032] Whether the real-time traffic data is abnormal traffic data is determined based on the comparison result between the target reconstructed data and the threshold data.
[0033] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement a method for detecting abnormal traffic data as described in any one of the first aspects.
[0034] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement a method for detecting abnormal traffic data as described in any one of the first aspects.
[0035] The present invention provides a method, system, device and medium for detecting abnormal flow data, which have the following technical effects:
[0036] The abnormal traffic data detection model provided in this embodiment adopts the structure of encoding layer, time domain convolution network layer, attention mechanism layer and decoding layer. This structure can effectively characterize the traffic data through self-supervised deep learning, and then distinguish the abnormal traffic data through advance data. When new categories of abnormal traffic data appear at different times, the abnormal detection model can identify them. At the same time, it has a good detection effect in the traffic data of known categories at any time. In addition, the computing power and memory of the abnormal traffic data detection model should be fixed or grow slowly with the increase of the number of categories.
[0037] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:
[0039] Figure 1 A first flow chart of a method for detecting abnormal flow data provided by an exemplary embodiment of the present invention;
[0040] Figure 2 A structural diagram of a basic model provided for an exemplary embodiment of the present invention;
[0041] Figure 3 A second flow chart of a method for detecting abnormal flow data provided by an exemplary embodiment of the present invention;
[0042] Figure 4 A comparison diagram provided for an exemplary embodiment of the present invention;
[0043] Figure 5 A structural diagram of a detection system for abnormal flow data provided by an exemplary embodiment of the present invention;
[0044] Figure 6 A structural diagram of an electronic device provided by an exemplary embodiment of the present invention. DETAILED DESCRIPTION
[0045] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present invention can be combined with each other without conflict.
[0046] The following detailed description is an exemplary description, which is intended to provide further detailed description of the present invention. Unless otherwise specified, all technical terms used in the present invention have the same meaning as those generally understood by those skilled in the art to which the present invention belongs. The terms used in the present invention are only for describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present invention.
[0047] In real life, the cost of obtaining labels for IoT traffic data is high, and traffic data of known categories cannot cover all uncountable categories. The representation learned by the self-supervised method can help the model better generalize to unseen traffic data, which is especially important for the constantly changing and evolving traffic data in the IoT environment. In real IoT scenarios, most attacks are unlabeled traffic data of unknown categories.
[0048] Based on this, an exemplary embodiment of the present invention provides a method for detecting abnormal traffic data. Figure 1 , detection methods include:
[0049] S11. Obtain real-time traffic data.
[0050] Among them, real-time traffic data includes time series traffic data.
[0051] S12, input the real-time traffic data into the abnormal traffic detection model to obtain the target reconstruction data of the real-time traffic data,
[0052] Among them, the abnormal traffic detection model includes an encoding layer, at least one time-domain convolutional network layer, at least one attention mechanism layer and a decoding layer arranged in sequence, and each time-domain convolutional network layer of the at least one time-domain convolutional network layer is alternately stacked with each attention mechanism layer in the at least one attention mechanism layer.
[0053] The architecture of the abnormal traffic detection model used in this embodiment can be specifically referred to Figure 2 The following will be Figure 2 and Figure 3 Taking as an example, the training process of the abnormal traffic detection model is explained:
[0054] S121. Obtain normal traffic data and a basic model.
[0055] exist Figure 2 In the example, normal traffic data includes time series traffic data, which is obtained by X1...X n To characterize.
[0056] The basic model is based on the VAE (Variational Autoencoder)-TCN (Time-domain Convolutional Network) architecture, which specifically includes an encoding layer, at least one Time-domain Convolutional Network layer, at least one Attention mechanism layer and a decoding layer arranged in sequence. Each Time-domain Convolutional Network layer of at least one Time-domain Convolutional Network layer is alternately stacked with each Attention mechanism layer in at least one Attention mechanism layer.
[0057] Among them, the encoding layer is used to encode traffic data into latent vectors; the time domain convolutional network layer is used to capture the long-term dependencies of latent vectors; the attention mechanism layer is used to enhance the features of latent vectors; and the decoding layer is used to decode latent vectors into reconstructed data.
[0058] Specifically, the encoding layer is located at the front end of the model and is responsible for receiving normal traffic data. The encoding layer maps the original high-dimensional time series data to the latent space to encode it into a low-dimensional latent vector. This process is usually implemented through a convolutional layer (for feature extraction) and outputs the mean and variance of the latent vector, which define the latent distribution. In addition, the encoding layer can capture the basic patterns in the sequence, especially in the compression of high-dimensional data, it can extract useful features and provide high-quality input for the downstream time domain convolutional network layer and attention mechanism layer.
[0059] The time domain convolutional network layer is located after the encoding layer and is used to further process the encoded features. The time domain convolutional network layer models long-term dependencies through causal convolution and dilated convolution. The dilated convolution in the time domain convolutional network layer helps the model retain the perception of long-distance information when processing long time series, so that complex time patterns can be captured. In addition, the time domain convolutional network layer replaces the traditional recursive neural network structure, which can achieve more efficient parallelization while maintaining the temporal nature of time series data. In this embodiment, the time domain convolutional network layer preferably uses dilated convolution.
[0060] The attention mechanism layer is located after the time domain convolutional network layer, and is used to further enhance the model's capture of key features of the time series. The attention mechanism layer calculates the weight of each time step so that the model can adaptively focus on important features at different time points. This mechanism can help the model identify important positions in the sequence and improve its sensitivity to specific patterns or anomalies. In addition, the attention mechanism can enhance the interpretability of the model, making it more robust in sequence generation and anomaly detection tasks, and amplify some significant features, thereby enhancing the expressive power of the abnormal traffic detection model. The attention mechanism layer in this embodiment can assign different weights to different time steps when processing latent vectors to focus on the most important parts of the latent vectors.
[0061] The decoding layer is at the end of the model and is used to decode the latent vector into the reconstructed data Y1...Y of the original time series. n . The decoding layer decodes the sampling points from the latent space into a time series and outputs generated or reconstructed data. Its structure is symmetrical with the encoding layer and is usually composed of convolutional layers or deconvolutional layers to ensure that the output reconstructed data can maintain the same dimension and time series characteristics as the input. In addition, the main goal of the decoder is to convert the representation of the latent space into an easily interpretable time series pattern, so that the model can fit different features and time patterns in generation and anomaly detection tasks.
[0062] In one embodiment, each time-domain convolutional network layer and each attention mechanism layer can be stacked alternately at least once, which helps the model learn complex patterns and dependencies in sequence data. Each layer further extracts features and context information based on the previous layer.
[0063] S122. Input the normal traffic data into the basic model for iteration until the loss function of the basic model iterates to a minimum value.
[0064] The loss function includes at least one of a reconstruction loss function and a KL divergence loss function.
[0065] Specifically, the loss function consists of the reconstruction loss function and the KL divergence loss function. The overall loss function can be expressed as:
[0066]
[0067] Among them, β is a weight term used to adjust the balance between reconstruction loss and KL divergence loss.
[0068] The reconstruction loss function is used to measure the difference between the reconstructed sequence generated by the decoder and the input sequence, usually using mean squared error (MSE) or cross entropy loss.
[0069] The formula is:
[0070]
[0071] Among them, x i is the original input data, is the reconstructed data, and N is the sequence length.
[0072] The KL divergence loss function is used to match the latent distribution with the standard normal distribution (usually set as the prior distribution), forcing the distribution of the latent variables to be close to the normal distribution.
[0073] The formula is:
[0074]
[0075] Here, q(z|x) is the latent distribution generated by the encoder and p(z) is the prior distribution, which is usually a standard normal distribution.
[0076] In one embodiment, during the iteration of the basic model, the parameters of the basic model are updated by an optimizer.
[0077] In one embodiment, during the iteration of the basic model, the basic model may also be iterated by an early stopping method. The following are two examples of the early stopping method:
[0078] When the iteration rounds of the basic model meet the tolerance rounds, the trained basic model is determined as the abnormal traffic detection model. Specifically, through the tolerance rounds, the model is allowed to continue training for several rounds when the performance of the validation set stagnates or slightly deteriorates. If there is no improvement within the tolerance rounds, the training is terminated. In this embodiment, the value setting of the tolerance rounds needs to balance premature stopping and overtraining.
[0079] Alternatively, when the accuracy of the basic model meets the accuracy threshold, the trained basic model is determined as the abnormal traffic detection model.
[0080] S123: Determine the trained basic model as an abnormal traffic detection model.
[0081] In one embodiment, the encoding layer further includes a normal mixture model, which is used to obtain a latent vector of a multi-modal distribution of real-time traffic data.
[0082] In standard encoding layers, latent variables are usually assumed to be a single normal distribution. However, for complex multimodal data, a single normal distribution may not be sufficient to capture the latent structure. Normal mixture models can represent the latent space as a mixture of multiple Gaussian distributions, which is suitable for data containing multiple modes or clusters.
[0083] A normal mixture model is added to the encoder output of the encoding layer so that the latent space representation approximates the latent features through multiple Gaussian distributions. This approach can optimize the normal mixture model parameters by maximizing the log-likelihood, thereby better learning the multimodal distribution of the latent variables.
[0084] Specifically, during the training process, the parameters (mean and variance) of the normal mixture model will be optimized along with the training of the encoding layer, so that the encoding layer can model complex time series more effectively.
[0085] In one embodiment, the abnormal traffic detection model further includes a normalized flow layer after the encoding layer, and the normalized flow layer is used to perform multi-layer flow transformation on the potential vector obtained by the encoding layer.
[0086] The Normalization Flow layer transforms a simple distribution (such as the normal distribution) into a more complex distribution through a series of reversible transformations, making the latent space representation more flexible. For the base model, the Normalization Flow layer can make the latent space more adaptable to the complex distribution of the original data by performing a series of reversible transformations on the latent variables. In the base model, the standard normal latent variables are transformed through the Normalization Flow layer so that the latent space representation can capture non-Gaussian or multimodal distributions. In this case, the Normalization Flow layer is usually used as a post-processing layer of the base model encoder output to enhance the diversity of the latent variables.
[0087] Specifically, the normalized fluidization layer uses a multi-layer reversible network (such as RealNVP or MAF) to expand the latent space to complex distributions, which can be used in conjunction with convolutional layers in the base model to preserve temporal dependencies while increasing the expressive power of latent variables.
[0088] In this embodiment, after passing through the normal mixture model or the normalized flow layer, the basic model can more effectively handle complex and multimodal data distributions and improve the performance of anomaly detection and generation. The normal mixture model provides a more expressive latent space through multimodal latent distribution, while the normalized flow layer provides a more adaptable latent structure through flexible transformation.
[0089] S13. Determine whether the real-time traffic data is abnormal traffic data based on the comparison result between the target reconstructed data and the threshold data.
[0090] In one embodiment, the threshold data is generally determined based on the verification of the trained abnormal traffic data detection model, as follows:
[0091] Acquire normal traffic data and abnormal traffic data; input the normal traffic data and the abnormal traffic data into an abnormal traffic detection model to obtain first reconstructed data corresponding to the normal traffic data and second reconstructed data corresponding to the abnormal traffic data; determine threshold data based on the comparison result of the first reconstructed data and the second reconstructed data.
[0092] Among them, normal traffic data and abnormal traffic data can come from CICIDS2018 (an IoT traffic dataset) or from (CICIoT2023).
[0093] By verifying the abnormal traffic data detection model with the detection results of normal traffic data and abnormal traffic data, it is possible to clearly identify the threshold data of the reconstructed data obtained by the abnormal traffic data in the abnormal traffic data detection model and the reconstructed data obtained by the normal traffic data in the abnormal traffic data detection model. Based on the threshold data, the abnormal traffic data can be better distinguished.
[0094] In one embodiment, before step S12, the method for detecting abnormal traffic data further includes:
[0095] The real-time traffic data is preprocessed. The preprocessing methods include but are not limited to noise removal, feature extraction, etc.
[0096] In this embodiment, see Figure 4 , a comparison is performed between the existing abnormal traffic data detection method and the abnormal traffic data detection model obtained in this embodiment.
[0097] Existing methods for detecting abnormal traffic data include Isolation Forest, Long Short-Term Memory Network (LSTM), etc., using accuracy, precision, recall, F1 score and receiver operating characteristic curve (ROC-AUC) as evaluation indicators:
[0098] method Accuracy Accuracy Recall F1 score ROC-AUC isolated forest 85％ 80％ 75％ 78％ 0.82 LSTM 88％ 82％ 80％ 81％ 0.85 This embodiment 92％ 90％ 88％ 89％ 0.91
[0099] It can be seen that the method provided in this embodiment is superior to most existing methods in evaluation.
[0100] Specifically, since the abnormal traffic data detection model provided in this embodiment adopts the structure of encoding layer, time domain convolution network layer, attention mechanism layer and decoding layer, this structure can effectively characterize the traffic data through self-supervised deep learning, and then distinguish the abnormal traffic data through advance data. When new categories of abnormal traffic data appear at different times, the abnormal detection model can identify them. At the same time, it has a good detection effect in the traffic data of known categories at any time. In addition, the computing power and memory of the abnormal traffic data detection model should be fixed or grow slowly with the increase of the number of categories.
[0101] An exemplary embodiment of the present invention provides a detection system for abnormal flow data, see Figure 5 , the detection system includes:
[0102] The first acquisition module 51 is used to acquire real-time traffic data.
[0103] A first reconstruction module 52 is used to input the real-time traffic data into the abnormal traffic detection model to obtain target reconstruction data of the real-time traffic data, wherein the abnormal traffic detection model includes an encoding layer, at least one time-domain convolutional network layer, at least one attention mechanism layer and a decoding layer arranged in sequence, and each time-domain convolutional network layer of the at least one time-domain convolutional network layer and each attention mechanism layer of the at least one attention mechanism layer are alternately stacked;
[0104] The comparison module 53 is used to determine whether the real-time traffic data is abnormal traffic data based on the comparison result between the target reconstructed data and the threshold data.
[0105] In one embodiment, the detection system further comprises:
[0106] The second acquisition module is used to acquire normal flow data and abnormal flow data;
[0107] A second reconstruction module, inputting the normal flow data and the abnormal flow data into the abnormal flow detection model, and obtaining first reconstruction data corresponding to the normal flow data and second reconstruction data corresponding to the abnormal flow data respectively;
[0108] The determination module is used to determine the threshold data based on the comparison result of the first reconstructed data and the second reconstructed data.
[0109] In one embodiment, the detection system further includes a training module for an abnormal traffic detection model;
[0110] The training module of the abnormal traffic detection model is used to:
[0111] Acquire normal traffic data and a basic model, where the basic model includes an encoding layer, at least one time-domain convolutional network layer, at least one attention mechanism layer, and a decoding layer arranged in sequence, and each time-domain convolutional network layer of the at least one time-domain convolutional network layer and each attention mechanism layer of the at least one attention mechanism layer are alternately stacked;
[0112] Input normal traffic data into the basic model for iteration until the loss function of the basic model iterates to the minimum value;
[0113] The trained basic model is determined as the abnormal traffic detection model.
[0114] In one embodiment, an encoding layer is used to encode traffic data into a latent vector;
[0115] The temporal convolutional network layer is used to capture the long-term dependencies of the latent vectors;
[0116] The attention mechanism layer is used to enhance the features of the latent vector;
[0117] The decoding layer is used to decode the latent vector into reconstructed data.
[0118] In one embodiment, the loss function includes at least one of a reconstruction loss function and a KL divergence loss function.
[0119] In one embodiment, the training module of the abnormal traffic detection model is also used to:
[0120] During the iteration of the basic model, when the iteration rounds of the basic model meet the tolerance rounds, the trained basic model is determined as the abnormal traffic detection model;
[0121] and / or, during the iteration of the basic model, when the accuracy of the basic model meets the accuracy threshold, the trained basic model is determined as the abnormal traffic detection model;
[0122] And / or, during the iteration of the basic model, the parameters of the basic model are updated by the optimizer.
[0123] In one embodiment, the encoding layer further includes a normal mixture model, which is used to obtain a latent vector of a multi-modal distribution of real-time traffic data;
[0124] And / or, the abnormal traffic detection model further includes a normalized flow layer after the encoding layer, and the normalized flow layer is used to perform multi-layer flow transformation on the potential vector obtained by the encoding layer.
[0125] An exemplary embodiment of the present invention further provides an electronic device, see Figure 6The electronic device 600 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 610 (the processor 610 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic battery FPGA), a memory 630 for storing data, and one or more storage media 620 (such as one or more mass storage devices) for storing application programs 623 or data 622. Among them, the memory 630 and the storage medium 620 can be short-term storage or permanent storage. The storage medium 620 stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the above method.
[0126] Furthermore, the CPU 610 may be configured to communicate with the storage medium 620 to execute a series of instruction operations in the storage medium 620 on the electronic device 600. The electronic device 600 may also include one or more power supplies 660, one or more wired or wireless network interfaces 60, one or more input and output interfaces 630, and / or one or more operating systems 621, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.
[0127] The input / output interface 630 may be used to receive or send data via a network. The specific example of the network may include a wireless network provided by a communication provider of the electronic device 600. In one example, the input / output interface 630 includes a network adapter (Network Interface Controller, NIC), which may be connected to other network devices via a base station so as to communicate with the Internet. In one example, the input / output interface 630 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0128] It can be understood by those skilled in the art that Figure 6 The structure shown is for illustration only and does not limit the structure of the above electronic device. Figure 6 More or fewer components as shown, or with Figure 6 Different configurations are shown.
[0129] An embodiment of the present invention also provides a computer-readable storage medium, which can be set in a server to store at least one instruction, at least one program, code set or instruction set related to implementing a method in a method embodiment. The at least one instruction, the at least one program, the code set or instruction set is loaded and executed by the processor to implement the above method.
[0130] Optionally, in this embodiment, the storage medium may be located in a network server among multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.
[0131] It is known from common technical knowledge that the present invention can be implemented by other embodiments that do not deviate from its spirit or essential features. Therefore, the embodiments of the above invention are only illustrative in all respects and are not exclusive. All changes within the scope of the present invention or within the scope equivalent to the present invention are included in the present invention.
[0132] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0133] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0134] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.
Claims
1. A method for detecting abnormal traffic data, characterized in that: The detection method comprises: Get real-time traffic data; Inputting the real-time traffic data into an abnormal traffic detection model to obtain target reconstructed data of the real-time traffic data, wherein the abnormal traffic detection model comprises an encoding layer, at least one time-domain convolutional network layer, at least one attention mechanism layer and a decoding layer arranged in sequence, and each time-domain convolutional network layer of the at least one time-domain convolutional network layer is alternately stacked with each attention mechanism layer of the at least one attention mechanism layer; Whether the real-time traffic data is abnormal traffic data is determined based on a comparison result between the target reconstructed data and the threshold data.
2. The detection method according to claim 1, characterized in that The detection method further comprises: Obtain normal traffic data and abnormal traffic data; Inputting the normal flow data and the abnormal flow data into an abnormal flow detection model to obtain first reconstructed data corresponding to the normal flow data and second reconstructed data corresponding to the abnormal flow data; Threshold data is determined based on a comparison result of the first reconstructed data and the second reconstructed data.
3. The detection method according to claim 1, characterized in that The detection method also includes a training step of an abnormal flow detection model; The training steps of the abnormal traffic detection model include: Acquire normal traffic data and a basic model, wherein the basic model includes an encoding layer, at least one time-domain convolutional network layer, at least one attention mechanism layer, and a decoding layer arranged in sequence, and each time-domain convolutional network layer of the at least one time-domain convolutional network layer is alternately stacked with each attention mechanism layer in the at least one attention mechanism layer; Inputting the normal traffic data into the basic model for iteration until the loss function of the basic model iterates to a minimum value; The trained basic model is determined as the abnormal traffic detection model.
4. The detection method according to claim 3, characterized in that The encoding layer is used to encode the traffic data into a latent vector; The time domain convolutional network layer is used to capture the long-term dependency of the potential vector; The attention mechanism layer is used to enhance the features of the latent vector; The decoding layer is used to decode the latent vector into reconstructed data.
5. The detection method according to claim 3, characterized in that: The loss function includes at least one of a reconstruction loss function and a KL divergence loss function.
6. The detection method according to claim 3, characterized in that: The training step of the abnormal traffic detection model also includes: During the iteration of the basic model, when the iteration round of the basic model meets the tolerance round, the trained basic model is determined as the abnormal traffic detection model; And / or, during the iteration of the basic model, when the accuracy of the basic model meets the accuracy threshold, the trained basic model is determined as the abnormal traffic detection model; And / or, during the iteration of the basic model, the parameters of the basic model are updated by an optimizer.
7. The detection method according to claim 1, characterized in that The encoding layer further includes a normal mixture model, which is used to obtain a latent vector of a multi-modal distribution of the real-time traffic data; And / or, the abnormal traffic detection model further includes a normalized flow layer after the encoding layer, and the normalized flow layer is used to perform multi-layer flow transformation on the potential vector obtained by the encoding layer.
8. A detection system for abnormal flow data, characterized in that: The detection system comprises: Get real-time traffic data; Inputting the real-time traffic data into an abnormal traffic detection model to obtain target reconstructed data of the real-time traffic data, wherein the abnormal traffic detection model comprises an encoding layer, at least one time-domain convolutional network layer, at least one attention mechanism layer and a decoding layer arranged in sequence, and each time-domain convolutional network layer of the at least one time-domain convolutional network layer is alternately stacked with each attention mechanism layer of the at least one attention mechanism layer; Whether the real-time traffic data is abnormal traffic data is determined based on a comparison result between the target reconstructed data and the threshold data.
9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the abnormal traffic data detection method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the method for detecting abnormal traffic data as described in any one of claims 1-7.