Adaptive multi-mode network flow feature fusion method and system

Abstract

The invention belongs to the technical field of artificial intelligence and network security, and discloses a self-adaptive multi-modal network traffic feature fusion method and system, and the method comprises the steps: obtaining statistical features, a packet length sequence and a load sequence of a network traffic sample, and extracting deep feature vectors of three types of modals; self-adaptive weights of all modes are generated through a gating mechanism, and weighted fusion is carried out on the deep feature vectors to obtain initial fusion features; classifying the initial fusion features, calculating an attribution value of each modal to a classification result by adopting a layer-by-layer correlation propagation algorithm, and optimizing a process of generating an adaptive weight based on consistency loss between the adaptive weight and the modal attribution value so as to obtain a final fusion feature vector consistent with the model decision logic; and determining a traffic category based on the final fusion feature vector. According to the method, an interpretation consistency supervision mechanism is introduced, so that the consistency of the fusion weight and the model decision basis is ensured, and the interpretability of the model is remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and network security, in particular to a self-adaptive multi-modal network traffic feature fusion method and system based on explanation consistency supervision. BACKGROUND

[0002] With the rapid development of mobile Internet, the number of mobile applications has increased dramatically, and the network traffic generated by them has become increasingly complex and diverse. In order to achieve effective network management, resource allocation and security monitoring, it is essential to accurately classify mobile application network traffic.

[0003] Existing traffic classification methods mainly fall into two categories: one is single-modal modeling method, which usually focuses on a certain type of feature flow, such as only based on traffic statistical features or only based on raw byte streams. Although this method can work in specific scenarios, it cannot fully describe complex traffic behavior due to the neglect of complementary information between different modal data, resulting in significant limitations in classification performance when facing complex scenarios such as encrypted traffic and disguised traffic. The other is a shallow multi-modal fusion method, which attempts to integrate multi-modal data through simple feature splicing or weighted averaging. This method alleviates the shortcomings of single-modal methods to some extent, but they lack deep modeling of explicit dependency between modalities, and essentially still regard each modality as an independent input, resulting in the loss of cross-modal association information that can reflect the deep behavior of traffic, and the fusion effect is not good.

[0004] Further analysis shows that existing methods generally have the problems of shallow fusion strategy and lack of explanation. Although some advanced methods attempt to use attention mechanisms for modality weighting, the weight learning process is often a "black box" that is disconnected from the final classification decision logic. This means that the weight assigned by the model may not reflect the true contribution of the modality in the actual decision-making process, resulting in insufficient generalization of the model in different network environments, and it is also difficult to provide reliable analysis for network security analysts. At the same time, due to the lack of effective modeling of modality cooperation, existing methods often show low classification accuracy and poor robustness when dealing with complex network scenarios.

[0005] Therefore, the core technical difficulty of traffic classification lies in how to effectively model the deep dependency between multi-modal data, how to dynamically assign appropriate fusion weights to different modalities, and how to ensure the consistency of the fusion result and the final classification decision logic. Developing a new multi-modal network traffic feature fusion method that can achieve high precision, high interpretability and strong adaptability has become a technical demand that needs to be solved in the current network security field. SUMMARY

[0006] This invention aims to address the problems of large differences in multimodal features, insufficient modeling of intermodal dependencies, and lack of interpretability in fusion weights in existing network traffic classification technologies. It provides an adaptive multimodal network traffic feature fusion method and system based on interpretation consistency supervision.

[0007] In a first aspect, the present invention provides an adaptive multimodal network traffic feature fusion method, comprising the following steps: Obtain multimodal feature data of network traffic samples to be classified, wherein the multimodal feature data includes statistical feature modal data, packet length sequence modal data, and payload sequence modal data; For each type of modal data in the multimodal feature data, a preset modality-specific feature extraction network is used to process the data to generate their respective deep feature vectors. For the three types of deep feature vectors corresponding to the statistical feature modal data, packet length sequence modal data, and payload sequence modal data, an adaptive fusion process for interpretation consistency is performed, the process including: Adaptive weights corresponding to each modality are generated by using a cross-modal attention mechanism and introducing gated dynamic weighting. Based on the adaptive weights, the three types of deep feature vectors are weighted and fused to obtain an initial fused feature vector; The classification result of the network traffic sample is obtained based on the initial fusion feature vector; The classification results are analyzed by a layer-by-layer correlation propagation algorithm to obtain the modality attribution value of each modality for the classification results; Calculate the consistency loss between the adaptive weights and the modal attribution values; Based on the consistency loss, the calculation process for generating the adaptive weights is optimized to obtain a final fused feature vector consistent with the model's decision logic. The category of the network traffic sample is determined based on the final fused feature vector.

[0008] As an optional implementation of the first aspect of this application, the step of obtaining multimodal feature data of network traffic samples to be classified specifically includes: extracting statistical feature fields from the original network traffic data to form a statistical feature matrix, and performing Z-score standardization on the statistical feature matrix to obtain the statistical feature modal data; extracting packet length sequences from the original network traffic data, performing length normalization on the packet length sequences, and mapping the packet length values ​​to a vocabulary index to obtain the packet length sequence modal data; merging forward and backward payload data from the original network traffic data, performing length normalization on the merged payload data, and obtaining a byte index sequence as the payload sequence modal data.

[0009] As an optional implementation of the first aspect of this application, the step of processing through a preset modality-specific feature extraction network specifically includes: inputting the statistical feature modality data into a multilayer perceptron network, extracting high-order semantics through nonlinear transformation, and generating a deep feature vector corresponding to the statistical feature modality data; inputting the packet-length sequence modality data into an embedding layer for dense vector mapping, and then inputting the mapped dense vector into a long short-term memory network to capture temporal dependencies and generate a deep feature vector corresponding to the packet-length sequence modality data; inputting the payload sequence modality data into an embedding layer for dense vector mapping, and then inputting the mapped dense vector into a one-dimensional convolutional network and a global max pooling layer to extract local pattern features and generate a deep feature vector corresponding to the payload sequence modality data.

[0010] As an optional implementation of the first aspect of this application, before generating the adaptive weights corresponding to each modality, the method further includes: concatenating the three types of deep feature vectors to form a multimodal representation matrix; processing the multimodal representation matrix through a cross-modal attention mechanism to calculate the attention weights between each modality, and performing a weighted summation of the three types of deep feature vectors to obtain a fused context representation; wherein the adaptive weights are generated based on the context representation.

[0011] As an optional implementation of the first aspect of this application, the step of generating adaptive weights corresponding to each modality specifically involves: applying a Sigmoid activation function to the context representation to calculate an adaptive weight value between 0 and 1 for each modality.

[0012] As an optional implementation of the first aspect of this application, the step of calculating the consistency loss between the adaptive weights and the modal attribution values ​​specifically includes: normalizing each modal attribution value output by the layer-by-layer correlation propagation algorithm to obtain normalized modal attribution values; and calculating the mean square error between the adaptive weights and the normalized modal attribution values ​​as the consistency loss.

[0013] As an optional implementation of the first aspect of this application, the step of determining the category of the network traffic sample based on the final fused feature vector specifically includes: inputting the final fused feature vector into a network structure composed of a fully connected layer, a batch normalization layer, and a Dropout layer for processing to perform information compression and enhance generalization ability, thereby obtaining a classification feature vector; inputting the classification feature vector into a classifier composed of a fully connected layer and a Softmax layer for processing, outputting the probability distribution of the network traffic sample for multiple preset categories, and taking the category with the highest probability value as the final classification result.

[0014] Secondly, embodiments of this application provide an adaptive multimodal network traffic feature fusion system, comprising: The data acquisition module is used to acquire multimodal feature data of network traffic samples to be classified. The multimodal feature data includes statistical feature modal data, packet length sequence modal data, and payload sequence modal data. The feature extraction module is used to process each type of modality data in the multimodal feature data through a preset modality-specific feature extraction network to generate their respective deep feature vectors. An interpretation consistency adaptive fusion module is used to perform interpretation consistency adaptive fusion processing on the three types of deep feature vectors corresponding to the statistical feature modal data, packet length sequence modal data, and payload sequence modal data, so as to obtain a final fused feature vector consistent with the model decision logic. The interpretation consistency adaptive fusion module includes: The weight generation unit is used to generate adaptive weights corresponding to each modality through a cross-modal attention mechanism and the introduction of gated dynamic weighting. The feature fusion unit is used to perform weighted fusion of the three types of deep feature vectors based on the adaptive weights to obtain an initial fused feature vector. A classification unit is used to obtain the classification result of the network traffic sample based on the initial fusion feature vector; The attribution analysis unit is used to perform attribution analysis on the classification results using a hierarchical correlation propagation algorithm to obtain the modality attribution value of each modality for the classification results; A loss calculation unit is used to calculate the consistency loss between the adaptive weights and the modal attribution values; An optimization unit is configured to optimize the calculation process for generating the adaptive weights based on the consistency loss, so as to output the final fused feature vector; The category determination module is used to determine the category of the network traffic sample based on the final fused feature vector.

[0015] Thirdly, embodiments of this application provide an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the method described in the first aspect.

[0016] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0017] Compared with the prior art, the beneficial effects of this invention are: 1. Improved classification performance: By designing dedicated deep network structures for different modalities and combining them with cross-modal attention mechanisms, information from each modality can be extracted and interacted more fully, effectively improving the accuracy of identifying complex network traffic.

[0018] 2. Enhanced model interpretability: An innovative Layer-by-Layer Relevance Propagation (LRP) consistency supervision mechanism is introduced, ensuring that the modal weights during the fusion process remain consistent with the attribution analysis results of the model's final decisions. This gives the fusion weights a clear physical meaning, providing network administrators and security analysts with a reliable basis for decision-making.

[0019] 3. Improved model robustness and generalization ability: The adaptive gating weighting mechanism can dynamically adjust the contribution of each modality according to the characteristics of different traffic samples, making the model more adaptable to changes in the network environment and traffic pattern drift. Attached Figure Description

[0020] Figure 1 This is a flowchart of an adaptive multimodal network traffic feature fusion method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of an adaptive multimodal network traffic feature fusion system according to an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0023] Example 1 Please see Figure 1 This is a flowchart illustrating an adaptive multimodal network traffic feature fusion method provided in an embodiment of the present invention. The method may include the following steps: S1: Obtain multimodal feature data of the network traffic samples to be classified. The multimodal feature data includes statistical feature modal data, packet length sequence modal data, and payload sequence modal data.

[0024] This step transforms the network traffic features to be classified into a standardized multimodal input format to support subsequent deep feature extraction and model training. Assume the network traffic dataset to be classified contains... A sample, representing For, each sample It includes multiple features such as statistical characteristics, packet length sequences, and payload information. First, irrelevant fields are removed from the original data to construct a statistical feature matrix. ,in This is a statistical feature dimension. Protocol anomalies and empty packets are removed to ensure data quality. Subsequently, the data is split into three modalities: Statistical characteristic modes: obtained after data cleaning Standardized using Z-score: in and These are the mean and standard deviation of the statistical features of the training set, respectively; Packet-length sequence modality: Representing the packet-length sequence of each sample as follows The sequence length is uniformly set to The package length is limited to the vocabulary size. Inside: Load sequence mode: Forward and backward loads are combined into a fixed-length byte index sequence. in The length is uniform Fill any missing parts with zeros.

[0025] For example, the statistical characteristics of a network traffic sample include 20 numerical fields (such as average packet length, maximum packet length, flow duration, etc.). After cleaning and standardization, a 20-dimensional vector is obtained, such as [0.12, -0.45, 1.03, ...]. Its corresponding packet length sequence might be [60, 1500, 64, 128, ...], uniformly truncated or padded to a length of 100. Its payload byte sequence might be [72, 101, 108, 108, 111, 0, 0, ...], with a fixed length of 256. This yields three different modalities of input data for subsequent network use.

[0026] S2: For each type of modality data in the multimodal feature data, a preset modality-specific feature extraction network is used to process the data to generate their respective deep feature vectors.

[0027] After the input is standardized and formatted, independent feature extraction networks are designed for the three modalities STATS, PKT, and PYL to obtain deep semantic representations.

[0028] For STATS modes, the input is a normalized statistical eigenvector. ,in This represents the statistical feature dimension. Higher-order semantics are extracted using a multilayer perceptron (MLP). in Represents the ReLU activation function. The weight matrix is ​​output. , , These represent the model bias parameters.

[0029] For the PKT mode, the input is a sequence of packet length amplitude values. in This represents the maximum sequence length. First, an embedding layer is used to map discrete magnitude indices to dense vectors: in Indicates the length of the embedding layer vector; Then, LSTM is used to capture timing dependencies: For the PYL modality, the input is the concatenated forward and reverse payload byte sequences. Similarly, it is mapped to a dense representation through the embedding layer: Then use one-dimensional convolution (Conv1D) and global max pooling ( Extracting local pattern features: Ultimately, the three modalities yielded 128-dimensional deep feature representations. These features will be used in subsequent feature fusion.

[0030] For example, a sample with a statistical feature vector of [0.5, -1.2, 0.8, ...] is input into an MLP, which outputs a 128-dimensional vector, such as [0.23, 0.11, -0.56, ...]. The corresponding packet length sequence [60, 500, 1500, 64] is processed by Embedding + LSTM to obtain another 128-dimensional vector, such as [0.05, -0.14, 0.67, ...]. The payload sequence [72, 101, 108, 108, 111] (corresponding to ASCII "Hello") is processed by convolution and pooling, which also outputs a 128-dimensional feature vector, such as [0.34, -0.29, 0.47, ...]. Thus, a sample yields three sets of deep semantic features of equal length, facilitating subsequent fusion.

[0031] S3: For the three types of deep feature vectors corresponding to the statistical feature modality data, packet length sequence modality data, and payload sequence modality data, perform an interpretation consistency adaptive fusion process. The process includes: generating adaptive weights corresponding to each modality through a cross-modal attention mechanism; weighting and fusing the three types of deep feature vectors based on the adaptive weights to obtain an initial fused feature vector; obtaining the classification result of the network traffic sample based on the initial fused feature vector; performing attribution analysis on the classification result using a layer-by-layer correlation propagation algorithm to obtain the modality attribution value of each modality for the classification result; calculating the consistency loss between the adaptive weights and the modality attribution value; and optimizing the calculation process for generating the adaptive weights based on the consistency loss to obtain a final fused feature vector consistent with the model decision logic.

[0032] After completing the independent feature extraction of the three modalities, this invention proposes an interpretability-consistent adaptive fusion mechanism based on cross-modal attention and gating dynamic weighting to achieve intermodal dependency modeling, adaptive weight learning, and interpretability constraints on the fusion results.

[0033] ① Cross-modal attention mechanism Let the deep features of the three modes be as follows: First, the modal features are concatenated to form a multimodal representation matrix: Then, the explicit dependencies between modalities are calculated using a cross-modal attention mechanism: in For trainable weight matrix, express (modal) )right (modal) Attention weights, This represents the dimension of the query vector. The fused context representation is obtained through weighted summation. in Represents the weight matrix; This process can explicitly model the dependencies between different modalities, thereby improving modal interaction and collaboration capabilities.

[0034] For example, a sample might have three feature vectors: [0.23, 0.11, -0.56, ...] (statistical features), [0.05, -0.14, 0.67, ...] (packet length), and [0.34, -0.29, 0.47, ...] (payload). The attention mechanism calculates the correlation between them. For instance, the model might consider that "packet length has a strong influence on payload features," and thus assign a higher weight to the payload features. ② Gated dynamic weighted fusion After obtaining the cross-modal context representation, a gated dynamic weighting method is introduced to adaptively model the importance of different modalities. The specific calculation is as follows: in The Sigmoid activation function is used. For the first Adaptive weights for each modality Represents the weight matrix. This represents the bias parameter. The final initial fused feature vector is: This mechanism can adaptively adjust the contribution of each modality under different traffic patterns, thereby improving the flexibility and robustness of the fusion results.

[0035] For example, in a traffic classification task, the statistical feature contribution of a traffic sample may be 0.2, the packet length feature contribution may be 0.5, and the payload feature contribution may be 0.3. The gating dynamic weighting will automatically allocate weights so that the final fused features are more in line with the actual task requirements.

[0036] ③ LRP Consistency Monitoring Mechanism To ensure consistency between the gating dynamic weighting and the model's decision-making logic, this invention introduces Layer-wise Relevance Propagation (LRP) for consistency supervision. Attribution values ​​are calculated from the final classification results obtained from the initial fused feature vectors to determine the actual contribution of each modality in the decision-making process. and with dynamic weighted gating weights Alignment, the optimization goal is: in Indicates consistency loss. This represents the normalized modal attribution value. This constraint ensures that the weight allocation of the gating dynamic weighting is consistent with the model's true decision-making basis, thereby enhancing the interpretability of the fusion mechanism.

[0037] After multimodal independent feature extraction and cross-modal alignment optimization, the three-modal features of each group of samples are fused and mapped to a unified low-dimensional representation space: in This represents the final fused feature vector. The fusion operation includes fully connected layers, batch normalization, and Dropout to compress redundant information and enhance generalization ability.

[0038] For example, if the model identifies a traffic sample as traffic for a specific application, the LRP analysis results show that the packet length modality contributes 45%, statistical features 30%, and payload 25%. In this case, the gated dynamic weighting will also be optimized through supervised learning to make the learned weights approximate this distribution, thereby ensuring interpretability.

[0039] S4: Determine the category of the network traffic sample based on the final fused feature vector.

[0040] Unified feature vector after fusion The input is fed into the classifier module. The classifier consists of a fully connected layer and a Softmax layer, and the output is a pair. Probability distribution of each category: in Represents the weight matrix. Indicates the bias parameter; This step transforms the representational capabilities of fused features into the final classification result, achieving an end-to-end closed-loop recognition process from multimodal input to accurate type prediction.

[0041] For example, the final fused feature is a 64-dimensional vector [0.12, -0.33, 0.45, ...]. After being input into a fully connected layer + Softmax classifier, the output result may be [0.05, 0.90, 0.05], indicating that the probability of the traffic being predicted as second-type application traffic is 90%.

[0042] Example 2 Please see Figure 2The diagram shown is a schematic representation of an adaptive multimodal network traffic feature fusion system proposed in the second embodiment of this application. The system includes the following key modules: The data acquisition module 100 is used to acquire multimodal feature data of network traffic samples to be classified. The multimodal feature data includes statistical feature mode data, packet length sequence mode data, and payload sequence mode data. The feature extraction module 200 is used to process each type of modality data in the multimodal feature data through a preset modality-specific feature extraction network to generate their respective deep feature vectors. An interpretation consistency adaptive fusion module 300 is used to perform interpretation consistency adaptive fusion processing on the three types of deep feature vectors corresponding to the statistical feature modal data, packet length sequence modal data, and payload sequence modal data, so as to obtain a final fused feature vector consistent with the model decision logic. The interpretation consistency adaptive fusion module includes: The weight generation unit 310 is used to generate adaptive weights corresponding to each modality through a cross-modal attention mechanism and the introduction of gated dynamic weighting. The feature fusion unit 320 is used to perform weighted fusion of the three types of deep feature vectors based on the adaptive weights to obtain an initial fused feature vector. Classification unit 330 is used to obtain the classification result of the network traffic sample based on the initial fusion feature vector; The attribution analysis unit 340 is used to perform attribution analysis on the classification results using a hierarchical correlation propagation algorithm to obtain the modality attribution value of each modality for the classification results; The loss calculation unit 350 is used to calculate the consistency loss between the adaptive weights and the modal attribution values; The optimization unit 360 is used to optimize the calculation process for generating the adaptive weights based on the consistency loss, so as to output the final fused feature vector; The category determination module 400 is used to determine the category of the network traffic sample based on the final fused feature vector.

[0043] An adaptive multimodal network traffic feature fusion system in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network attached storage (NAS), personal computers (PCs), etc. This application embodiment does not specifically limit the specific implementation.

[0044] The adaptive multimodal network traffic feature fusion system in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0045] The adaptive multimodal network traffic feature fusion system provided in this application embodiment can achieve... Figure 1 The various processes implemented by the adaptive multimodal network traffic feature fusion method in the method embodiment are not described in detail here to avoid repetition.

[0046] Optionally, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described adaptive multimodal network traffic feature fusion method embodiment and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0047] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiment of an adaptive multimodal network traffic feature fusion method and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0048] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0049] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0050] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0051] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An adaptive multimodal network traffic feature fusion method, characterized in that, Includes the following steps: Obtain multimodal feature data of network traffic samples to be classified, wherein the multimodal feature data includes statistical feature modal data, packet length sequence modal data, and payload sequence modal data; For each type of modal data in the multimodal feature data, a preset modality-specific feature extraction network is used to process the data to generate their respective deep feature vectors. For the three types of deep feature vectors corresponding to the statistical feature modal data, packet length sequence modal data, and payload sequence modal data, an adaptive fusion process for interpretation consistency is performed, the process including: Adaptive weights corresponding to each modality are generated by using a cross-modal attention mechanism and introducing gated dynamic weighting. Based on the adaptive weights, the three types of deep feature vectors are weighted and fused to obtain an initial fused feature vector; The classification result of the network traffic sample is obtained based on the initial fusion feature vector; The classification results are analyzed by a layer-by-layer correlation propagation algorithm to obtain the modality attribution value of each modality for the classification results; Calculate the consistency loss between the adaptive weights and the modal attribution values; Based on the consistency loss, the calculation process for generating the adaptive weights is optimized to obtain a final fused feature vector consistent with the model's decision logic. The category of the network traffic sample is determined based on the final fused feature vector.

2. The method according to claim 1, characterized in that, The steps for obtaining multimodal feature data of network traffic samples to be classified specifically include: Statistical feature fields are extracted from the raw network traffic data to form a statistical feature matrix, and the statistical feature matrix is ​​Z-score standardized to obtain the statistical feature modal data. The packet length sequence is extracted from the original network traffic data, the packet length sequence is normalized, and the packet length value is mapped to a vocabulary index to obtain the packet length sequence modal data. The forward and backward payload data are merged from the original network traffic data, and the merged payload data is length normalized to obtain a byte index sequence as the payload sequence modal data.

3. The method according to claim 1, characterized in that, The step of processing through a preset modality-specific feature extraction network specifically includes: The statistical feature modal data is input into a multilayer perceptron network, and high-order semantics are extracted through nonlinear transformation to generate deep feature vectors corresponding to the statistical feature modal data. The packet-length sequence modal data is input into an embedding layer for dense vector mapping, and then the mapped dense vector is input into a long short-term memory network to capture temporal dependencies and generate deep feature vectors corresponding to the packet-length sequence modal data. The payload sequence modal data is input into an embedding layer for dense vector mapping. The mapped dense vector is then input into a one-dimensional convolutional network and a global max pooling layer to extract local pattern features and generate deep feature vectors corresponding to the payload sequence modal data.

4. The method according to claim 1, characterized in that, Before generating the adaptive weights corresponding to each modality, the process also includes: The three types of deep feature vectors are concatenated to form a multimodal representation matrix; The multimodal representation matrix is ​​processed through a cross-modal attention mechanism to calculate the attention weights between each modality, and the three types of deep feature vectors are weighted and summed to obtain the fused context representation. The adaptive weights are generated based on the context representation.

5. The method according to claim 4, characterized in that, The step of generating adaptive weights corresponding to each modality is as follows: The context representation is calculated by applying a Sigmoid activation function to generate an adaptive weight value between 0 and 1 for each mode.

6. The method according to claim 1, characterized in that, The step of calculating the consistency loss between the adaptive weights and the modal attribution values ​​specifically includes: The modal attribution values ​​output by the layer-by-layer correlation propagation algorithm are normalized to obtain normalized modal attribution values; The mean square error between the adaptive weights and the normalized modal attribution values ​​is calculated as the consistency loss.

7. The method according to claim 1, characterized in that, The step of determining the category of the network traffic sample based on the final fused feature vector specifically includes: The final fused feature vector is input into a network structure consisting of a fully connected layer, a batch normalization layer, and a Dropout layer for processing to compress information and enhance generalization ability, thereby obtaining a classification feature vector. The classification feature vector is input into a classifier consisting of a fully connected layer and a softmax layer for processing. The classifier outputs the probability distribution of the network traffic sample for multiple preset categories and takes the category with the highest probability value as the final classification result.

8. An adaptive multimodal network traffic feature fusion system, characterized in that, include: The data acquisition module is used to acquire multimodal feature data of network traffic samples to be classified. The multimodal feature data includes statistical feature modal data, packet length sequence modal data, and payload sequence modal data. The feature extraction module is used to process each type of modality data in the multimodal feature data through a preset modality-specific feature extraction network to generate their respective deep feature vectors. An interpretation consistency adaptive fusion module is used to perform interpretation consistency adaptive fusion processing on the three types of deep feature vectors corresponding to the statistical feature modal data, packet length sequence modal data, and payload sequence modal data, so as to obtain a final fused feature vector consistent with the model decision logic. The interpretation consistency adaptive fusion module includes: The weight generation unit is used to generate adaptive weights corresponding to each modality through a cross-modal attention mechanism and the introduction of gated dynamic weighting. The feature fusion unit is used to perform weighted fusion of the three types of deep feature vectors based on the adaptive weights to obtain an initial fused feature vector. A classification unit is used to obtain the classification result of the network traffic sample based on the initial fusion feature vector; The attribution analysis unit is used to perform attribution analysis on the classification results using a hierarchical correlation propagation algorithm to obtain the modality attribution value of each modality for the classification results; A loss calculation unit is used to calculate the consistency loss between the adaptive weights and the modal attribution values; An optimization unit is configured to optimize the calculation process for generating the adaptive weights based on the consistency loss, so as to output the final fused feature vector; The category determination module is used to determine the category of the network traffic sample based on the final fused feature vector.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein when the program or instructions are executed by the processor, they implement the steps of an adaptive multimodal network traffic feature fusion method as described in any one of claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions, which, when executed by a processor, implement the steps of an adaptive multimodal network traffic feature fusion method as described in any one of claims 1-7.

Images