Abnormal network traffic detection method based on multimodal data fusion

Abstract

The present invention belongs to the technical field of abnormal network traffic detection, and specifically provides an abnormal network traffic detection method based on multimodal data fusion, including obtaining different types of network traffic data sets, including image data sets, text data sets and audio data sets; preprocessing of data sets, including data cleaning, data enhancement, and data conversion; encoding different data sets; using the encoding result to obtain a loss function; using the loss function to train a model, and detecting abnormal network traffic through the model. The present invention uses multimodal data features to achieve semantically similar alignment through graph distribution, introduces cross-modal comparative learning, and uses consistency information to learn semantic features, thereby improving the accuracy and efficiency of abnormal network traffic detection, and providing strong technical support for network security management.

Description

Technical Field

[0001] The present invention belongs to the technical field of abnormal network traffic detection, and specifically provides an abnormal network traffic detection method based on multimodal data fusion. Background Art

[0002] With the popularization of the Internet and the continuous advancement of network technology, network traffic has shown explosive growth. Network traffic contains a large amount of user behavior data, such as web browsing, file transfer, email exchanges, social media interactions, etc. These data not only reflect the normal network activities of users, but may also hide various abnormal behaviors. These abnormal behaviors not only pose a threat to network security, but also infringe on the legitimate rights and interests of users and damage the healthy development of the network environment. In order to effectively identify and prevent these abnormal network traffic, real-time monitoring and analysis of network traffic has become an important task in the field of network security. By detecting network traffic, the patterns and characteristics of abnormal behaviors can be discovered, so as to timely warn and respond to potential security threats.

[0003] However, abnormal network traffic detection faces many challenges. First, the scale of network traffic is huge and the data types are diverse, including text, pictures, audio, etc., which requires the analysis method to have efficient data processing capabilities and flexible adaptability. Secondly, abnormal network traffic is often hidden and complex, and attackers may use various means to evade detection, such as encrypted communication and the use of botnets. Therefore, studying efficient, accurate and secure abnormal network traffic detection methods is of great significance to improving network security protection capabilities. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes an abnormal network traffic detection method based on multimodal data fusion to improve the accuracy and efficiency of abnormal network traffic detection.

[0005] The present invention is implemented by providing a method for detecting abnormal network traffic based on multimodal data fusion, comprising the following steps:

[0006] Step 1: Obtain different types of network traffic datasets, including image datasets, text datasets, and audio datasets;

[0007] Step 2: Preprocessing of the data set, including data cleaning, data enhancement, and data conversion;

[0008] Step 3: Encode different data sets. For image data sets, use the Vision Transformer network to encode and output feature vectors ; For the text dataset, BEiT-3 network is used for encoding and the output feature vector is ; For the audio dataset, use the Meta-Transformer network for encoding and output the feature vector , the feature vector and the eigenvector Concatenate and get the feature vector ;

[0009] Step 4: Using feature vectors , eigenvector And the alignment matrix P to obtain the loss function;

[0010] Step 5: Use the loss function to train the model and detect abnormal network traffic through the model.

[0011] Preferably, in step 1, the data set includes a data set obtained from an open source data set platform, an academic research data set platform, and a data set collected and created by oneself.

[0012] Further preferably, in step 2, data cleaning specifically includes:

[0013] For image datasets: remove noise, adjust brightness and contrast;

[0014] For text datasets: remove HTML tags, URL links, special symbols, and correct spelling or grammatical errors;

[0015] For audio datasets: remove background noise and perform sound enhancement.

[0016] Further preferably, in step 2, data enhancement specifically includes:

[0017] For image datasets: increase the diversity of image data by rotating, flipping, and scaling to enhance the data;

[0018] For text datasets: Increase the diversity of text data to enhance the data by using synonym replacement and sentence rewriting methods;

[0019] For audio datasets: Increase the diversity of audio data by adding noise and changing the pitch to enhance the data.

[0020] Further preferably, in step 2, the data conversion converts the original data set into a format suitable for input, specifically comprising:

[0021] For image datasets: perform normalization;

[0022] For text datasets: convert to word vectors or tokenize;

[0023] For audio datasets: perform sampling rate conversion or frame segmentation.

[0024] Further preferably, in step 3, the step of encoding the image data set using the Vision Transformer network is:

[0025] The image dataset that passes through the Embedding layer is input into the Transformer encoder. In the Transformer encoder, it first passes through the Norm layer, then enters the Multi-Head Attention layer for addition operation. After addition, it enters the Norm layer again, and then passes through the MLP module. The operations in the Transformer encoder are repeatedly stacked multiple times to output the feature vector. .

[0026] Further preferably, in step 3, the step of encoding the text data set using the BEiT-3 network is as follows: by inputting the text data set, entering the Multiway Transformer module, selecting the Language Expert module to perform mask data modeling on the input text data set, and thus repeatedly stacking the operations in the Multiway Transformer module for multiple times, and outputting the feature vector .

[0027] Further preferably, in step 3, the step of encoding the audio data set using the Meta-Transformer network is:

[0028] first, Represents the spectral data samples of the audio dataset, using the logarithmic Mel filter bank for duration 1 second audio waveform is preprocessed; then, The frequency span used is The Hamming window is used to split the original wave into interval; through Convolution divides the entire spectrum into {N}_{s}=12\left [ {(100t-16) / 10} \right ] Patches are then flattened into a labeled sequence. The whole process is as shown in formula (1):

[0029] (1)

[0030] in, represents the set of real numbers, is the patch size, is the embedding dimension, and Represents the time dimension and frequency dimension, and finally outputs the feature vector .

[0031] Further preferably, in step 4, using the feature vector , eigenvector The process of obtaining the loss function by aligning the matrix P is:

[0032] Step 401: First, pass the feature vector and the eigenvector Get the loss function :

[0033] (2)

[0034] in, represents the sample index of the aligned data in different datasets, represents a subset of features derived from the known correspondence part of the sample, represents mutual information, represents information entropy, parameter as an entropy regularization term;

[0035] Step 402: Feature Vector , eigenvector Use fully connected layer modules respectively , Get potential features , :

[0036] (3)

[0037] (4)

[0038] Step 403: Align the matrix P to assist in learning semantics and latent features, and obtain two loss functions , :

[0039] (5)

[0040] (6)

[0041] in, represents the trace operation of the matrix, F represents a fully connected layer, which calculates the alignment matrix P based on the Euclidean distance between latent features. and represents a known correspondence, which is assigned 1 in this alignment matrix. is a matrix of all 1s, and I is the identity matrix.

[0042] Further preferably, in step 5, the training process is:

[0043] S501: In the pre-training phase, use the loss function To train the decoder, map different dataset samples to the corresponding latent space;

[0044] S502: Then, the alignment is learned in a sequential iterative manner, that is, the alignment data is aligned on the similarity graph using a loss function Train to obtain the alignment matrix P, feature , It is cyclically referenced and learned to obtain the loss function training model;

[0045] S503: Detect abnormal network traffic by training a model through a loss function.

[0046] Compared with the prior art, the advantages of the present invention are:

[0047] This paper provides an abnormal network traffic detection method based on multimodal data fusion, which uses multimodal data features to achieve semantically similar alignment through graph distribution, introduces cross-modal comparative learning, and uses consistency information to learn semantic features, thereby improving the accuracy and efficiency of abnormal network traffic detection and providing strong technical support for network security management. DETAILED DESCRIPTION

[0048] The present invention is further explained below in conjunction with specific implementation schemes, but is not intended to limit the protection scope of the present invention.

[0049] The present invention provides an abnormal network traffic detection method based on multimodal data fusion, comprising the following steps:

[0050] Step 1: Obtain different types of network traffic datasets, including image datasets, text datasets, and audio datasets;

[0051] The datasets include those obtained from open source dataset platforms, academic research dataset platforms, and datasets collected and created by ourselves.

[0052] Open source dataset platforms include Kaggle, GitHub, HyperAI, etc. These platforms provide a variety of datasets, including but not limited to LAV-DF multimodal audio and visual dataset, OpenForensics face forgery detection dataset, Scene-15, Caltech101, Reuters-dim10, etc.

[0053] The datasets collected by the academic research dataset platform may be, for example, relevant datasets released by many universities and research institutions.

[0054] You can also collect data yourself from social media, news sites, public forums, etc., or create simulated data sets.

[0055] Step 2: Preprocessing of the data set, including data cleaning, data enhancement, and data conversion;

[0056] Data cleaning includes removing duplicate data, processing missing values, outliers, etc., specifically including:

[0057] For image datasets: remove noise, adjust brightness and contrast;

[0058] For text datasets: remove HTML tags, URL links, special symbols, and correct spelling or grammatical errors;

[0059] For audio datasets: remove background noise and perform sound enhancement.

[0060] Data enhancement specifically includes:

[0061] For image datasets: increase the diversity of image data by rotating, flipping, and scaling to enhance the data;

[0062] For text datasets: Increase the diversity of text data to enhance the data by using synonym replacement and sentence rewriting methods;

[0063] For audio datasets: Increase the diversity of audio data by adding noise and changing the pitch to enhance the data.

[0064] Data conversion is to convert the original data set into a format suitable for model input, including:

[0065] For image datasets: perform normalization;

[0066] For text datasets: convert to word vectors or tokenize;

[0067] For audio datasets: perform sampling rate conversion or frame segmentation.

[0068] Step 3: Encode different data sets. For image data sets, use the Vision Transformer network to encode and output feature vectors ; For the text dataset, use the BEiT-3 network for encoding and output the feature vector ; For the audio dataset, use the Meta-Transformer network for encoding and output the feature vector , the feature vector and the eigenvector Concatenate and get the feature vector ;

[0069] The steps for encoding the image dataset using the Vision Transformer network are:

[0070] The image dataset that passes through the Embedding layer is input into the Transformer encoder. In the Transformer encoder, it first passes through the Norm layer, then enters the Multi-Head Attention layer for addition operation. After addition, it enters the Norm layer again, and then passes through the MLP module. The operations in the Transformer encoder are repeatedly stacked multiple times to output the feature vector. .

[0071] The steps for encoding text datasets using the BEiT-3 network are as follows: input the text dataset, enter the Multiway Transformer module, select the Language Expert module to perform mask data modeling on the input text dataset, and repeat the operations in the Multiway Transformer module multiple times to output the feature vector. .

[0072] The steps for encoding the audio dataset using the Meta-Transformer network are:

[0073] first, Represents the spectral data samples of the audio dataset, using the log Mel filter bank for duration 1 second audio waveform is preprocessed; then, The frequency span used is The Hamming window is used to split the original wave into interval; through Convolution divides the entire spectrum into {N}_{s}=12\left [ {(100t-16)10} \right ] Patches are then flattened into a labeled sequence. The whole process is as shown in formula (1):

[0074] (1)

[0075] in, represents the set of real numbers, is the patch size, is the embedding dimension, and Represents the time dimension and frequency dimension, and finally outputs the feature vector .

[0076] Step 4: Using feature vectors , eigenvector And the alignment matrix P to obtain the loss function;

[0077] Using feature vectors , eigenvector The process of obtaining the loss function by aligning the matrix P is:

[0078] Step 401: First, pass the feature vector and the eigenvector Get the loss function :

[0079] (2)

[0080] in, represents the sample index of the aligned data in different datasets, represents a subset of features derived from the known correspondence part of the sample, represents mutual information, represents information entropy, parameter as an entropy regularization term;

[0081] Step 402: Feature Vector , eigenvector Use fully connected layer modules respectively , Get potential features , :

[0082] (3)

[0083] (4)

[0084] Step 403: Align the matrix P to assist in learning semantics and latent features, and obtain two loss functions , :

[0085] (5)

[0086] (6)

[0087] in, represents the trace operation of the matrix, F represents a fully connected layer, which calculates the alignment matrix P based on the Euclidean distance between latent features. and represents a known correspondence, which is assigned 1 in this alignment matrix. is a matrix of all 1s, and I is the identity matrix.

[0088] Step 5: Use the loss function to train the model and detect abnormal network traffic through the model.

[0089] The process of training the model is:

[0090] S501: In the pre-training phase, use the loss function To train the decoder, map different dataset samples to the corresponding latent space;

[0091] S502: Then, the alignment is learned in a sequential iterative manner, that is, the alignment data is aligned on the similarity graph using a loss function Train to obtain the alignment matrix P, feature , It is cyclically referenced and learned to obtain the loss function training model;

[0092] S503: Detect abnormal network traffic by training a model through a loss function.

Claims

1. An abnormal network traffic detection method based on multimodal data fusion, characterized in that: The following steps are involved: Step 1: Obtain different types of network traffic datasets, including image datasets, text datasets, and audio datasets; Step 2: Preprocessing of the data set, including data cleaning, data enhancement, and data conversion; Step 3: Encode different data sets. For image data sets, use the Vision Transformer network to encode and output feature vectors ; For the text dataset, BEiT-3 network is used for encoding and the output feature vector is ; For the audio dataset, use the Meta-Transformer network for encoding and output the feature vector ,Will and Concatenate and get the feature vector ; Step 4: Using feature vectors , eigenvector And the alignment matrix P to obtain the loss function; Step 5: Use the loss function to train the model and detect abnormal network traffic through the model; In step 3, the steps of encoding the audio data set using the Meta-Transformer network are as follows: first, Represents the spectral data samples of the audio dataset, using the logarithmic Mel filter bank for duration 1 second audio waveform is preprocessed; then, The frequency span used is The Hamming window is used to split the original wave into interval; through Convolution divides the entire spectrum into Patches are then flattened into a labeled sequence. The whole process is as shown in formula (1): （1） in, represents the set of real numbers, is the patch size, is the embedding dimension, and Represents the time dimension and frequency dimension, and finally outputs the feature vector ; In step 4, the feature vector , eigenvector The process of obtaining the loss function by aligning the matrix P is: Step 401: First, pass the feature vector and the eigenvector Get the loss function : （2） in, represents the sample index of the aligned data in different datasets, represents a subset of features derived from the known correspondence part of the sample, represents mutual information, represents information entropy, parameter as an entropy regularization term; Step 402: Feature Vector , eigenvector Use fully connected layer modules respectively , Get potential features , : （3） （4） Step 403: Align the matrix P to assist in learning semantics and latent features, and obtain two loss functions , : （5） （6） in, represents the trace operation of the matrix, F represents a fully connected layer, which calculates the alignment matrix P based on the Euclidean distance between latent features. and represents a known correspondence, which is assigned 1 in this alignment matrix. is a matrix of all 1s, and I is the identity matrix.

2. The abnormal network traffic detection method based on multimodal data fusion according to claim 1 is characterized in that: In step 1, the data sets include data sets obtained from open source data set platforms, academic research data set platforms, and data sets collected and created by oneself.

3. The abnormal network traffic detection method based on multimodal data fusion according to claim 1 is characterized in that: In step 2, data cleaning specifically includes: For image datasets: remove noise, adjust brightness and contrast; For text datasets: remove HTML tags, URL links, special symbols, and correct spelling or grammatical errors; For audio datasets: remove background noise and perform sound enhancement.

4. The abnormal network traffic detection method based on multimodal data fusion according to claim 1 is characterized in that: In step 2, data enhancement specifically includes: For image datasets: increase the diversity of image data by rotating, flipping, and scaling to enhance the data; For text datasets: Increase the diversity of text data to enhance the data by using synonym replacement and sentence rewriting methods; For audio datasets: Increase the diversity of audio data by adding noise and changing the pitch to enhance the data.

5. The abnormal network traffic detection method based on multimodal data fusion according to claim 1 is characterized in that: In step 2, data conversion converts the original data set into a format suitable for input, specifically including: For image datasets: perform normalization; For text datasets: convert to word vectors or tokenize; For audio datasets: perform sampling rate conversion or frame segmentation.

6. The abnormal network traffic detection method based on multimodal data fusion according to claim 1 is characterized in that: In step 3, the steps of encoding the image dataset using the Vision Transformer network are as follows: The image dataset that passes through the Embedding layer is input into the Transformer encoder. In the Transformer encoder, it first passes through the Norm layer, then enters the Multi-Head Attention layer for addition operation. After addition, it enters the Norm layer again, and then passes through the MLP module. The operations in the Transformer encoder are repeatedly stacked multiple times to output the feature vector. .

7. The abnormal network traffic detection method based on multimodal data fusion according to claim 1 is characterized in that: In step 3, the steps of encoding the text data set using the BEiT-3 network are as follows: by inputting the text data set, entering the Multiway Transformer module, selecting the Language Expert module to perform mask data modeling on the input text data set, and thus repeatedly stacking the operations in the Multiway Transformer module multiple times to output a feature vector .

8. The abnormal network traffic detection method based on multimodal data fusion according to claim 1 is characterized in that: In step 5, the training process is: S501: In the pre-training phase, use the loss function To train the decoder, map different dataset samples to the corresponding latent space; S502: Then, the alignment is learned in a sequential iterative manner, that is, the alignment data is aligned on the similarity graph using a loss function Train to obtain the alignment matrix P, feature , It is cyclically referenced and learned to obtain the loss function training model; S503: Detect abnormal network traffic by training a model through a loss function.

Images