Network anomaly detection method based on multi-modal federal active learning

Abstract

The invention discloses a multi-modal federation active learning-based network anomaly detection method, which comprises the following steps of: fusing data of a plurality of modals through multi-modal feature aggregation to generate uniform feature representation; an active learning mechanism is introduced into the federated learning framework, and distributed nodes are allowed to autonomously select learning strategies according to characteristics and distribution conditions of own data; the node isomerism in the distributed environment is dealt with through an adaptive learning optimization strategy; the communication efficiency is optimized and the global synchronization frequency is reduced through a progressive federated synchronization mechanism; according to the method, multi-modal data fusion and federated active learning are combined, so that the precision and efficiency of network anomaly detection are improved on the premise of protecting data privacy.

Description

Technical Field

[0001] The present invention relates to the field of network anomaly detection, and in particular to a network anomaly detection method based on multimodal federated active learning. Background Art

[0002] With the vigorous development of technologies such as the Internet, cloud computing, and the Internet of Things, the number of global network traffic and connected devices has exploded, and the complexity and diversity of network attacks have continued to escalate. As a key technology to ensure network security, network anomaly detection aims to discover abnormal behaviors in network systems, such as malicious attacks, data leakage, and abnormal operations. However, the distributed nature of modern network environments and data privacy requirements have brought new challenges to network anomaly detection. First, in cloud computing and Internet of Things applications, data is usually distributed in different physical devices or nodes and cannot be collected and processed centrally. This distributed environment requires anomaly detection methods to adapt to decentralized network architectures and perform distributed data analysis and model training. Second, many organizations and individual users are highly concerned about data privacy and security. Traditional centralized model training methods usually require a large amount of data to be uploaded to a central server for processing, which is prone to data leakage and privacy violations. Therefore, how to perform efficient network anomaly detection without centralizing data has become an urgent need. Finally, in actual network security scenarios, multiple data modalities such as network traffic, log information, system calls, and sensor data can be used to detect network abnormal behaviors. Data of different modalities contain different information dimensions. Relying only on single-modal data may miss important abnormal features and fail to fully reflect complex network attack behaviors. Therefore, how to integrate multimodal data and enhance the accuracy and comprehensiveness of network anomaly detection has become a key issue.

[0003] At present, there are mainly the following network anomaly detection methods: Rule-based anomaly detection methods rely on predefined rules or thresholds to judge abnormal behavior. They usually use fixed attack signature libraries, feature rule sets, etc. to identify known threats. This type of method has a high detection rate for known attacks, but has poor detection capabilities for unknown threats and is difficult to adapt to complex and dynamically changing network environments. In order to address the limitations of rule-based methods, machine learning technology has been gradually applied to network anomaly detection in recent years. By learning patterns of normal and abnormal behaviors from historical data, machine learning models can detect unknown threats that are not clearly defined. However, most machine learning methods rely on centralized data collection and model training, which is not feasible in modern distributed network environments, especially when the data involves privacy-sensitive content, centralized processing will cause privacy leakage problems. In addition, most existing machine learning methods only use single-modality data for detection, such as only using network traffic data or log information, resulting in limited detection coverage. Summary of the invention

[0004] To this end, it is necessary to provide a network anomaly detection method based on multimodal federated active learning. By combining multimodal data fusion and federated active learning, the accuracy and efficiency of network anomaly detection can be improved while protecting data privacy.

[0005] To achieve the above object, the inventor provides a network anomaly detection method based on multimodal federated active learning, comprising the steps of: S1, through multimodal feature aggregation, the data of several modalities are fused to generate a unified feature representation; S2, introduces active learning mechanism into the federated learning framework, allowing distributed nodes to autonomously select learning strategies based on the characteristics and distribution of their own data; S3, copes with node heterogeneity in distributed environments through adaptive learning optimization strategies; S4, optimizes communication efficiency and reduces the frequency of global synchronization through a progressive federated synchronization mechanism.

[0006] As a preferred embodiment of the present invention, the several modalities include: text modality, visual modality and voice modality, the text modality includes log files, the visual modality includes data stream data, and the voice modality includes voice communication data.

[0007] As a preferred embodiment of the present invention, step S1 further includes: S101, several modalities including text modality, extract text features from the data source of the text modality, divide the text features into context levels, and represent the context of each level through a bidirectional encoder representation model to form a hierarchical context embedding representation of the text modality , and use the attention mechanism to adaptively select informative text features between different levels , the expression is: ; in, represents a time-dependent adaptive weight that is dynamically adjusted as the time step t changes. represents the self-attention mechanism, Represents the dynamic weighting coefficient related to the attention weight; S102, several modalities including visual modality, extracting data stream data from the data source of the visual modality, and fusing the data stream data through a multi-view feature learning model, the expression is: ; in, represents the visual feature representation after multi-view fusion, represents the adaptive weight coefficient of the i-th view, Represents the geometric transformation matrix, N represents the total number of viewing angles, and is used to align the features of the i-th camera viewing angle to the geometric space consistent with the j-th camera viewing angle. represents the feature representation extracted from the i-th camera view, represents the feature representation extracted from the j-th camera view, It is used to balance the geometric consistency loss and the overall loss in the feature fusion process. represents the geometric consistency constraint loss function; S103, several modules include speech modality, extracting speech communication data from the data source of speech modality, and fusing the speech communication data through differentiable signal decomposition and neural modality alignment mechanism, the expression is: ; in, represents the speech feature representation, Represents differentiable signal decomposition, which decomposes the input speech into several components , represents the weights of different components, represents the frequency alignment operation, represents the alignment weight, DTW represents the time domain alignment operation, T represents the time alignment parameter, represents the contribution of each semantic component, and M represents the number of semantic components; S104, an adaptive fusion mechanism based on modal differentiation collaborative modeling, actively utilizes modal differences and adjusts the fusion of modal features through modal collaborative conflict resolution mechanism. The expression is: ; in, and represents the characteristics of any mode, Indicates different modal indexes, and Represents the collaborative learning enhancement weight, which controls the influence of other modules on the current k-modal enhancement. Represents feature collaborative learning, represents the weight of controlling conflicting features, Indicates conflict characteristics, Represents the fused multimodal features.

[0008] As a preferred embodiment of the present invention, the text features include system log features, and the system log features are divided into context levels according to time, location and operation type.

[0009] As a preferred embodiment of the present invention, step S2 further includes: S201, in the distributed federated learning model, each node independently selects a learning strategy to perform multi-objective optimization of the node. The multi-objective optimization function of the node The expression is: ; in, represents the learning strategy of node b, Indicates The influence weight of each factor is represents the fused multimodal features, Representation and Learning Strategies Related The measurement between factors; In S202, each node generates a pseudo label through self-supervision signals and continuously learns and updates through unlabeled data. The expression is: ; in, Indicates that node b is The parameters after round training, represents the current training parameters, Represents the learning rate, which is used to control the update of parameters. represents the gradient of the loss function, represents an unlabeled dataset, represents the loss function, represents the error on the unlabeled data x, represents the pseudo-label generated by self-supervision; S203, dynamically adjust the communication frequency through the dynamic evaluation mechanism of the variation and the adaptive communication control strategy, the expression is: ; in, Representation Node The communication trigger probability in round t is used to decide whether to synchronize with the global model. Indicates the parameter change, is the set change threshold, To adjust the parameters, Represents an exponential function with a natural number e as base.

[0010] As a preferred embodiment of the present invention, step S3 further includes: S301, based on the adaptive learning optimization strategy, dynamically adjust the task allocation strategy through the Q-learning learning algorithm, the expression is: ; in, Representation Node The task allocation strategy, Representation Node computing resources, storage resources, and network bandwidth, Indicates the current node task Execute action task allocation The long-term cumulative reward function is represents the discount factor, which is used to weigh the immediate reward and long-term reward, represents the mathematical expectation; S302, according to the resource status of each node, dynamically prune the non-critical layers in the model to make it lightweight, the expression is: ; in, represents the initial complexity of the distributed federated learning model, Representation Node computing resources, storage resources, and network bandwidth, Indicates the weight parameter that determines the impact of each resource on pruning. Represents the complexity of the distributed federated learning model after pruning.

[0011] As a preferred embodiment of the present invention, step S4 further includes: S401, after each round of training, each node dynamically selects important parameters for compression and synchronization based on the difference of the distributed federated learning model, and adopts a gradual fusion strategy. During global update, the updates from each node are gradually integrated in a weight adjustment manner, and the expression is: ; in, Represents the global model in Wheel parameters, represents the learning rate, Representation Node The weight in round t, Represents the difference parameter after compression, Representation Node Parameters.

[0012] Different from the prior art, the above technical solution has the following beneficial effects: (1) Some network anomaly detection methods mostly rely on a single-modal data source. The detection accuracy and comprehensiveness of this method are often limited by the singleness of the data source and cannot capture complex abnormal behaviors. This method introduces multimodal data sources, such as text, vision, and voice, to capture complex abnormal behaviors in the network environment from different dimensions. The multimodal data fusion method enhances the comprehensiveness and accuracy of network anomaly detection by combining the advantages of each modality, and solves the problem that single-modal data cannot cover complex attack scenarios; (2) The federated autonomous learning method of this method allows each distributed node to independently determine the optimal learning strategy based on its own local data distribution and characteristics, without relying on the unified guidance of the global model. This autonomous learning mechanism can ensure that each node performs the most effective model update for its unique data structure during the training process, thereby reducing data annotation costs and improving the learning efficiency and performance of the local model. At the same time, each node can reduce frequent parameter exchanges with other nodes while making full use of its own data resources, thereby reducing communication overhead. (3) In a distributed network environment, different nodes may have differences in hardware configuration, computing power, data type, etc. The federated autonomous learning method proposed in this method can make adaptive adjustments based on the heterogeneity of each node. Nodes with low computing power can choose lightweight learning strategies, while high-performance nodes can perform more local training tasks, ensuring efficient collaboration of the entire federated system. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a framework diagram of the method described in the specific implementation method. DETAILED DESCRIPTION

[0014] In order to explain the technical content, structural features, achieved objectives and effects of the technical solution in detail, the following is a detailed description in conjunction with specific embodiments and accompanying drawings.

[0015] like Figure 1 As shown, this embodiment provides a network anomaly detection method based on multimodal federated active learning, characterized in that it includes the steps of: S1, through multimodal feature aggregation, data of several modalities are fused to generate a unified feature representation; specifically: data of different modalities such as text, vision, and voice are fused. The data source of each modality represents information of different dimensions in the network, such as: system log files are text modalities, video stream data are visual modalities, and voice communication data are voice modalities. After feature extraction and preprocessing of these modal data, multimodal fusion strategies are used to combine information of different modalities to generate a unified feature representation.

[0016] S2, introduces an active learning mechanism in the federated learning framework, allowing distributed nodes to autonomously select learning strategies based on the characteristics and distribution of their own data; this can significantly reduce global synchronization and dependence on labeled data. Each node selects the distributed federated learning model structure that best suits the local data for independent training by analyzing the characteristics of local data, such as specific network traffic patterns, user behavior logs, etc. During the autonomous learning process, each node can autonomously evaluate the performance of the local model and dynamically adjust the training strategy based on data changes without relying on the unified guidance of the global model. For some nodes, this mechanism allows them to continue to improve the performance of local models through unsupervised learning strategies without labeled data. In addition, nodes will not synchronize with the global model frequently when updates are small, effectively reducing communication frequency and overhead.

[0017] S3, through adaptive learning optimization strategy to deal with the node heterogeneity in the distributed environment; specifically, each node dynamically selects the appropriate task load and model complexity according to its own hardware configuration, computing resources, storage capacity and network bandwidth. The system can assign tasks according to the resource status of the node: for nodes with low computing power, lightweight model training tasks are assigned; for high-performance nodes, more complex training tasks are assigned. This adaptive learning optimization strategy ensures efficient collaboration of all nodes in a heterogeneous environment and fully utilizes the hardware capabilities of each node.

[0018] S4, optimizes communication efficiency and reduces the frequency of global synchronization through a progressive federated synchronization mechanism. Specifically, this method is different from the synchronization method in traditional federated learning. This method proposes an innovative progressive federated synchronization mechanism to optimize communication efficiency and reduce the frequency of global synchronization. This mechanism not only relies on model difference evaluation, but also introduces model compression technology and gradual fusion strategy, which can further improve communication efficiency and synchronization effect.

[0019] In the specific implementation process of step S1 in the above embodiment, step S1 includes the following steps: S101, first, extract features from data sources of different modalities. These features represent information of different dimensions in the network, such as text features in system logs, visual features in surveillance videos, and audio features in voice communications. For each modality of data, a specific feature extraction method is used, and these features are represented and normalized so that they can be processed uniformly. For text features, they are divided into context levels and multi-level features are extracted. Events in system logs can be divided into different levels according to time, location, and operation type. The context of each level is represented by a bidirectional encoder representation model to form a hierarchical context embedding representation of the text modality. , and uses the attention mechanism to enable the system to adaptively select the most informative text features between different levels , the expression is: ; in, represents a time-dependent adaptive weight that is dynamically adjusted as the time step t changes. represents the self-attention mechanism, Represents a dynamic weighting coefficient related to the attention weight, which is responsible for adjusting the influence of the attention mechanism when multiple layers of context are aggregated. This coefficient is updated by gradient descent during the model training phase and can be adaptively adjusted according to task-specific requirements; S102, for video stream data, although the traditional convolutional neural network can extract visual features, it is difficult to process dynamic scene changes in surveillance videos or images. This embodiment proposes a multi-view feature learning model based on geometric consistency constraints, which can process complex visual scenes in real time, especially multi-view and scene changes in surveillance videos, and the expression is: ; in, It represents the visual feature representation after multi-view fusion, and it represents the global representation formed by integrating the visual features extracted from different camera perspectives. Indicates that this is the adaptive weight coefficient of the i-th camera perspective, Represents the geometric transformation matrix, N represents the total number of viewing angles, and is used to align the features of the i-th camera viewing angle to the geometric space consistent with the j-th camera viewing angle. represents the feature representation extracted from the i-th camera view, represents the feature representation extracted from the j-th camera view, Used to balance the geometric consistency loss and the overall loss in the feature fusion process. It represents the geometric consistency constraint loss function, which is used to minimize the geometric error between multiple viewpoint features, ensure that the features extracted from different viewpoints can be aligned, and ensure that the model obtains consistent feature representation from different viewpoints.

[0020] S103, for speech data, traditional speech modeling methods mainly rely on fixed convolutional or recursive network structures, but have limited performance when processing speech signals with strong background noise, emotional changes, and multiple speaker modes. In order to more accurately capture the dynamic features in speech and enhance the adaptability to complex scenarios, this embodiment proposes a speech modeling method based on differentiable signal decomposition and neural modal alignment mechanism, which is expressed as: ; in, represents the speech feature representation, Represents differentiable signal decomposition, which decomposes the input speech into multiple components , represents the weights of different components, represents the frequency alignment operation, represents the alignment weight, DTW represents the time domain alignment operation, T represents the time alignment parameter, represents the contribution of each semantic component, and M represents the number of semantic components.

[0021] S104, based on the adaptive fusion mechanism of modal differentiation collaborative modeling, actively utilizes modal differences, and through the "modal collaborative conflict resolution" mechanism, makes the differences between modalities an advantage in the fusion process, rather than simply pursuing alignment. This embodiment not only makes improvements at the feature alignment level, but also deeply utilizes the differences between modalities to dynamically adjust the fusion of each modal feature to improve the model's ability to understand complex data. The expression is: ; in, and represents the characteristics of any mode, Indicates different modal indexes, and Represents the collaborative learning enhancement weight, which controls the influence of other modules on the current k-modal enhancement. Represents feature collaborative learning, represents the weight of controlling conflicting features, Indicates conflict characteristics, It is the fused multi-modal feature.

[0022] In the specific implementation process of step S2 in the above embodiment, step S2 includes the following steps: S201, in the distributed federated learning framework, each node autonomously selects a learning strategy. This strategy is not based on a single goal, such as model accuracy, but combines the node's computing resources, network conditions, local data characteristics, such as data distribution, quality, and noise level, to determine the optimal learning strategy. Each node regards the learning strategy selection problem as a multi-objective optimization problem, assigns weights to different goals through an adaptive algorithm, and finally autonomously selects the learning strategy that best suits the current data and system conditions. The node's multi-objective optimization function expression is: ; in, represents the learning strategy of node b, Indicates The influence weight of each factor is is the fused multimodal feature, Representation and Strategy Related The measurement between factors, such as the accuracy, resource consumption, synchronization cost, etc. of the distributed federated learning model. This function contains multiple sub-goals, such as minimizing computing resource consumption, minimizing communication frequency, minimizing the impact of data noise, etc. Each node dynamically adjusts the weight of each factor according to its current state. ,For example: when the node computing resources are insufficient, the weight of the resource consumption target can be increased, so as to select a learning strategy with lower computing overhead; S202, in traditional federated learning, nodes usually rely on the update guidance of the global model. In this embodiment, each node can dynamically evaluate the performance of the local model through a self-supervised learning mechanism and adjust the model parameters in real time according to the changes in the local data distribution, thereby reducing the dependence on labeled data. Through the self-supervised signal, the model of each node can generate pseudo labels and continue to learn through unlabeled data. The update method of the model is expressed as: ; in, Indicates that node b is The parameters after round training, represents the current training parameters, Represents the learning rate, which is used to control the update of parameters. represents the loss function, represents the error on the unlabeled data x, represents the pseudo-label generated by self-supervision, represents the gradient of the loss function, represents an unlabeled dataset; S203, in federated learning, the synchronization frequency between nodes and the global model directly affects the communication overhead and system efficiency; in the traditional federated learning framework, each node needs to frequently synchronize parameters with the global model, which consumes a lot of network bandwidth and computing resources. In order to optimize the communication frequency, this embodiment uses a dynamic evaluation mechanism of model changes and an adaptive communication control strategy to dynamically adjust the communication frequency according to the size of the local model changes, thereby reducing unnecessary global synchronization, reducing communication overhead, and improving system efficiency. The adaptive communication optimization expression based on the model change is: ; in, Representation Node The communication trigger probability in round t is used to decide whether to synchronize with the global model. Indicates the parameter change, Indicates the set change threshold, To express the adjustment parameters, we control the steepness of the communication probability function so that when the model changes greatly, the communication probability increases rapidly. Represents an exponential function with a natural number e as base.

[0023] In the specific implementation process of step S3 in the above embodiment, step S3 includes the following steps: S301, based on the adaptive learning optimization strategy, the present invention dynamically adjusts the task allocation strategy through the Q-learning learning algorithm, and no longer relies solely on predefined rules or static parameters to allocate tasks, but dynamically adjusts the task allocation through the algorithm. Through node feedback, such as computing speed, resource consumption, etc., the system can learn the resource status of each node, and find the optimal task allocation strategy through trial and error and learning to ensure that the resource utilization of heterogeneous nodes is maximized. The expression is: ; in, Representation Node The task allocation strategy Representation Node computing resources, storage resources, and network bandwidth, Indicates the current node task Execute action task allocation The long-term cumulative reward function is is a discount factor used to weigh immediate rewards and long-term rewards, Represents the mathematical expectation. Through the Q-learning algorithm, this embodiment can dynamically adjust the task load to ensure that the resources of each node are optimally utilized. The task allocation of each node not only depends on the current resource status, but also optimizes the strategy based on the feedback of past task execution.

[0024] S302, adaptive model complexity adjustment based on hierarchical model pruning. This embodiment uses an adaptive model complexity adjustment mechanism based on the resource status of each node. This method not only considers the hardware resources of the node, but also dynamically prunes the non-critical layers in the model, such as neuron connections with smaller weights, according to the resource status of each node to generate a lightweight model version. The node can prune some layers or channels of the neural network according to its own computing power to reduce the model complexity, so that low-resource nodes can also participate in complex tasks. The expression is: ; in, represents the initial complexity of the distributed federated learning model, Representation Node computing resources, storage resources, and network bandwidth, Indicates that the weight parameter determines the impact of each resource on pruning. represents the complexity of the distributed federated learning model after pruning. If computing resources are limited, storage and bandwidth are also limited, It will be close to 1, which means that the pruning ratio of the distributed federated learning model is large, generating a lightweight distributed federated learning model. On the contrary, if the computing resources are sufficient, the storage and bandwidth are sufficient, It will be close to 0, which means that the model pruning ratio is small and the complete model structure is retained.

[0025] In the specific implementation process of step S4 in the above embodiment, step S4 includes the following steps: S401, this embodiment greatly optimizes the communication efficiency in federated learning and reduces the frequency of global synchronization through model compression and gradual fusion strategies. Specifically, after each round of training, the node no longer synchronizes all model parameters immediately, but dynamically selects important parameters for compression and synchronization based on model differences. At the same time, the system adopts a gradual fusion strategy. When the global model is updated, the model updates from each node are gradually integrated in a more flexible weight adjustment method, thereby reducing unnecessary synchronization overhead and ensuring the synchronization effect and model convergence speed. The expression is: ; in, Represents the global model in Wheel parameters, represents the learning rate, Representation Node The weight in round t, Represents the difference parameter after compression, Representation Node Parameters.

[0026] In order to prove the effectiveness of this method, a public dataset was used for verification. Specifically, the Internet of Things (IoT) in smart cities, the network dataset IOT-23 generates data from various edge devices, including smart sensors, cameras, vehicles, air quality detectors, and network traffic monitors. This type of dataset is characterized by multimodality and is suitable for anomaly detection tasks in the field of network security. It can help detect distributed denial of service (DDoS) attacks, data leaks, network congestion, and other network threats. The performance comparison experimental results of different network anomaly detection methods are shown in Table 1: Table 1: Performance comparison experimental results of different network anomaly detection methods

[0027] The experimental results show that this method performs significantly better than other methods in terms of network anomaly detection performance. As can be seen from the table, this method achieves 97.1%, 95.1% and 96.1% in accuracy, recall and F1 value, which are much higher than other models. This shows that this method can effectively capture complex abnormal behaviors and improve the comprehensiveness and accuracy of detection by introducing multimodal federated active learning. In contrast, traditional methods such as BiGRU, Siamese-BiGRU and BERT+WMD perform relatively poorly in these indicators, with accuracy and recall generally between 70% and 83%, and F1 values ​​between 66.8% and 78.4%. This shows that the model of this method can not only process multimodal data in a distributed environment, but also greatly improve the overall detection efficiency while reducing missed reports and false positives.

[0028] It should be noted that, although the above embodiments have been described in this article, the patent protection scope of the present invention is not limited thereby. Therefore, based on the innovative concept of the present invention, changes and modifications made to the embodiments described herein, or equivalent structures or equivalent process changes made using the contents of the present invention specification and drawings, directly or indirectly applying the above technical solutions to other related technical fields, are all included in the patent protection scope of the present invention.

Claims

1. A network anomaly detection method based on multimodal federated active learning, characterized in that: Includes steps: S1, through multimodal feature aggregation, the data of several modalities are fused to generate a unified feature representation; S2, introduces active learning mechanism into the federated learning framework, allowing distributed nodes to autonomously select learning strategies based on the characteristics and distribution of their own data; S3, copes with node heterogeneity in distributed environments through adaptive learning optimization strategies; S4, optimizes communication efficiency and reduces the frequency of global synchronization through a progressive federated synchronization mechanism.

2. The network anomaly detection method based on multimodal federated active learning according to claim 1 is characterized in that: The several modalities include: a textual modality including log files, a visual modality including data stream data, and a voice modality including voice communication data.

3. The network anomaly detection method based on multimodal federated active learning according to claim 1 is characterized in that: S101, several modalities including text modality, extract text features from the data source of the text modality, divide the text features into context levels, and represent the context of each level through a bidirectional encoder representation model to form a hierarchical context embedding representation of the text modality , and use the attention mechanism to adaptively select informative text feature representations between different layers , the expression is: ； in, represents a time-dependent adaptive weight that is dynamically adjusted as the time step t changes. represents the self-attention mechanism, Represents the dynamic weighting coefficient related to the attention weight; S102, several modalities including visual modality, extracting data stream data from the data source of the visual modality, and fusing the data stream data through a multi-view feature learning model, the expression is: ； in, represents the visual feature representation after multi-view fusion, represents the adaptive weight coefficient of the i-th view, Represents the geometric transformation matrix, N represents the total number of viewing angles, and is used to align the features of the i-th camera viewing angle to the geometric space consistent with the j-th camera viewing angle. represents the feature representation extracted from the i-th camera view, represents the feature representation extracted from the j-th camera view, It is used to balance the geometric consistency loss and the overall loss in the feature fusion process. represents the geometric consistency constraint loss function; S103, several modules include speech modality, extracting speech communication data from the data source of speech modality, and fusing the speech communication data through differentiable signal decomposition and neural modality alignment mechanism, the expression is: ； in, represents the speech feature representation, Represents differentiable signal decomposition, which decomposes the input speech into several components , represents the weights of different components, represents the frequency alignment operation, represents the alignment weight, DTW represents the time domain alignment operation, T represents the time alignment parameter, represents the contribution of each semantic component, and M represents the number of semantic components; S104, an adaptive fusion mechanism based on modal differentiation collaborative modeling, actively utilizes modal differences and adjusts the fusion of modal features through modal collaborative conflict resolution mechanism. The expression is: ； in, and represents the characteristics of any mode, Indicates different modal indexes, and Represents the collaborative learning enhancement weight, which controls the influence of other modules on the current k-modal enhancement. Represents feature collaborative learning, represents the weight of controlling conflicting features, Indicates conflict characteristics, Represents the fused multimodal features.

4. The network anomaly detection method based on multimodal federated active learning according to claim 3 is characterized by: The text features include system log features, and the system log features are divided into context levels according to time, location and operation type.

5. The network anomaly detection method based on multimodal federated active learning according to claim 3 is characterized by: S201, in the distributed federated learning model, each node independently selects a learning strategy to perform multi-objective optimization of the node. The multi-objective optimization function of the node The expression is: ； in, represents the learning strategy of node b, Indicates The influence weight of each factor is represents the fused multimodal features, Representation and Learning Strategies Related The measurement between factors; In S202, each node generates a pseudo label through self-supervision signals and continuously learns and updates through unlabeled data. The expression is: ； in, Indicates that node b is The parameters after round training, represents the current training parameters, Represents the learning rate, which is used to control the update of parameters. represents the gradient of the loss function, represents an unlabeled dataset, represents the loss function, represents the error on the unlabeled data x, represents the pseudo-label generated by self-supervision; S203, dynamically adjust the communication frequency through the dynamic evaluation mechanism of the variation and the adaptive communication control strategy, the expression is: ； in, Representation Node The communication trigger probability in round t is used to decide whether to synchronize with the global model. Indicates the parameter change, is the set change threshold, To adjust the parameters, Represents an exponential function with a natural number e as base.

6. The network anomaly detection method based on multimodal federated active learning according to claim 5 is characterized by: S301, based on the adaptive learning optimization strategy, dynamically adjust the task allocation strategy through the Q-learning learning algorithm, the expression is: ； in, Representation Node The task allocation strategy Representation Node computing resources, storage resources, and network bandwidth, Indicates the current node task Execute action task allocation The long-term cumulative reward function is represents the discount factor, which is used to weigh the immediate reward and long-term reward, represents the mathematical expectation; S302, according to the resource status of each node, dynamically prune the non-critical layers in the model to make it lightweight, the expression is: ； in, represents the initial complexity of the distributed federated learning model, Representation Node computing resources, storage resources, and network bandwidth, Indicates the weight parameter that determines the impact of each resource on pruning. Represents the complexity of the distributed federated learning model after pruning.

7. The network anomaly detection method based on multimodal federated active learning according to claim 6 is characterized by: S401, after each round of training, each node dynamically selects important parameters for compression and synchronization based on the differences in the distributed federated learning model, and adopts a gradual fusion strategy. During global updates, the updates from each node are gradually integrated in a weight adjustment manner. The expression is: ； in, Represents the global model in Wheel parameters, represents the learning rate, Representation Node The weight in round t, Represents the difference parameter after compression, Representation Node Parameters.

Images