Edge anomaly detection method based on global-personalized collaborative federated learning

Abstract

The invention relates to an edge anomaly detection method based on global-personalized collaborative federated learning, and the method comprises the steps: enabling an edge server to generate a global model, and then releasing the global model; the client initializes a local model according to the global model; converting the traffic features into global and personalized feature vectors by using a condition calculation component at a client; the client introduces comparison and magnitude loss into a global category embedding module, guides the learning of global feature vectors according to the embedding of sample categories, and guides the learning of personalized feature vectors by using cross entropy loss; the client side judges whether loss replacement is executed or not by calculating the cosine similarity of the cross entropy loss and the total loss updating gradient, and then model training is accelerated; the client uploads the update of the model to the edge server after completing local training; performing global aggregation by the edge server, and updating a global model; and repeating the steps until the anomaly detection model converges. The method is beneficial for improving the edge anomaly detection precision and efficiency.

Description

Technical Field

[0001] The present invention relates to the field of edge anomaly detection technology, and in particular to an edge anomaly detection method based on global-personalized collaborative federated learning. Background Art

[0002] The rapid development of 5G technology has provided strong support for Internet-of-Things (IoT)-related application fields (such as autonomous driving, smart cities, and environmental monitoring), and has also led to a sharp increase in the number of IoT devices. IoT devices contain a large amount of private data, and their security and reliability are critical to both individuals and groups. In recent years, the number of IoT device attacks has been increasing, so anomaly detection plays an important role in ensuring IoT security. In classic cloud computing, massive traffic data is transmitted from IoT devices to the remote cloud for detection, which not only consumes a lot of network bandwidth, but also causes significant latency, seriously affecting user experience. To alleviate this problem, Mobile Edge Computing (MEC) reduces the dependence of IoT devices on the remote cloud by sinking computing and storage resources to the edge of the network, thereby improving bandwidth utilization and transmission latency. In MEC, IoT devices need to upload their data to the edge server for processing, and there is still a privacy leakage problem. At the same time, traffic data is distributed on different IoT devices. If the anomaly detection model is trained locally, it faces the problem of data islands. The emerging Federated Learning (FL) is seen as an effective way to solve the problems of privacy leakage and data silos, and has been initially applied to the field of edge anomaly detection.

[0003] Figure 1 The classic FL is shown. Figure 1 In (a), all clients have the same computing power and aggregation weight, and the edge server provides a unified global model. Figure 1 In (b), the client aggregation weights are different, and the edge server provides a personalized model for the client. Figure 1As shown in (a), in the classic FL (such as FedAvg), the IoT device (i.e., the client) first downloads the global model from the edge server and trains the local model using local data. Then, the local model parameters are uploaded to the edge server, which averages all local model parameters to update the global model and enter the next round of training. Therefore, the IoT device does not need to directly upload its local data to the edge server, protecting user privacy and security. However, the classic FL provides a unified global model for all clients, which does not take into account the differences between different clients. At the same time, the classic FL targets independent and identically distributed (IID) data and is difficult to effectively process non-independent and identically distributed (Non-IID) data. Figure 1 As shown in (b), as a new paradigm of FL, personalized FL (pFL) can provide a personalized model for each client. pFL can personalize the model according to the client's local data, thereby improving the performance of the model on different clients.

[0004] For edge anomaly detection, most existing studies do not consider the Non-IID data problem (i.e., statistical heterogeneity) and use FedAvg to train the global model, which makes it difficult to achieve expected performance on different IoT devices. In addition, in complex edge environments, the diversity of IoT devices causes large differences in their computing power (i.e., system heterogeneity). IoT devices with insufficient computing power train models more slowly, resulting in a significant increase in the time to complete an FL aggregation, which seriously affects the training efficiency of the global model. Therefore, it is necessary to design an efficient pFL framework suitable for heterogeneous edge environments to accelerate model training while improving the accuracy of edge anomaly detection. Summary of the invention

[0005] The purpose of the present invention is to provide an edge anomaly detection method based on global-personalized collaborative federated learning, which is conducive to improving the accuracy and efficiency of edge anomaly detection.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is: an edge anomaly detection method based on global-personalized collaborative federated learning, comprising the following steps:

[0007] (1) After the edge server generates the global model, it delegates the global model to the client; the client initializes the local model based on the global model;

[0008] (2) Using the conditional computation component on the client side to convert traffic features into global and personalized feature vectors;

[0009] (3) The client introduces contrast and magnitude losses in the global category embedding module and guides the learning of the global feature vector based on the embedding of the sample category, and uses the cross entropy loss to guide the learning of the personalized feature vector;

[0010] (4) The client determines whether to perform loss replacement by calculating the cosine similarity of the cross entropy loss and the overall loss update gradient, thereby accelerating the training of the model;

[0011] (5) After completing local training, the client uploads the model update to the edge server; the edge server performs global aggregation and updates the global model; the above steps are repeated until the anomaly detection model converges.

[0012] Furthermore, for a system with N IoT devices, i.e., clients, with statistical and systematic heterogeneity, for each round of FL learning, the edge server randomly selects a subset of clients S r , its client downloads the global model from the edge server; then, the client trains the model using its local data; the training dataset of client i is recorded as where y j ∈{1,2,...,Y} is the sample x j The label of , Y represents the number of abnormal types; therefore, the optimization objective of the local model is:

[0013]

[0014] Among them, ξ i represents the loss function of client i; W i and F i denote the personalized model and local objective function of client i respectively;

[0015] After completing the local update, the model parameters will be uploaded to the edge server. Finally, the edge server updates the global model by aggregating the client models. Considering the differences between different clients, a personalized anomaly detection model is customized for the client. Therefore, the optimization goal of the global model is:

[0016]

[0017] in, represents the weight of client i, {W1,W2,...,W Sr} is represented by S r A personalized model customized for each client.

[0018] Furthermore, the last fully connected (FC) layer of the client's local model is used as the head The rest is used as feature extractor ψ(·; W f ); First, ψ(·; Wf ) Extract training sample X of client i i features and maps them into a new feature vector X' i ; For the local training step t = 1, the cosine similarity cosim(·) is not calculated, but the feature conversion module FeT is directly used to convert X' i Convert to global eigenvector and personalized feature vector then, and are input into the global category embedding module GCE and for global and personalized learning; in GCE, is guided to be close to its corresponding global category embedding GCE(Y i ; E) and freeze global category embeddings To learn Finally, each client performs personalized training and fine-tunes χ to learn ; To improve training efficiency, for local training step t>1, the cosim(·) of the cross entropy loss and the overall loss update gradient in step t-1 is calculated to decide whether to replace the loss function; In each round of training, the client shares the category embedding E, the conditional computation component T, and the feature extractor ψ, and the client copies the frozen category embedding after receiving E

[0019] Furthermore, the implementation method of the feature conversion module FeT is:

[0020] Feature extractor ψ(·; W f ) The training sample X i Mapped to X' i ; In order to coordinate global and personalized learning, X' i Converted to global eigenvectors and personalized feature vector

[0021]

[0022] Among them, σ is the ReLU activation function, μ is the Hadamard product; α, β, α i and β i is the mapping parameter, which is generated by FeT:

[0023]

[0024] Among them, FeT contains two learnable components T with the same structure but different parameters. α and T β ; T α and Tβ It consists of FC layer, ReLU activation layer and normalization layer; g and p i is the global and personalized conditional input generated by GCE; by combining g and p i Input to T α and T β Generate α, β, α i and β i .

[0025] Furthermore, the implementation method of the global category embedding module GCE is:

[0026] GCE contains a learnable category embedding E for generating g and p i , to guide global learning; before each round of training E, the client copies E to generate frozen category embeddings They are used to generate the global category embedding GCE(y;E) and frozen global category embedding for each abnormal category y First, by averaging all anomaly categories To generate global conditional input:

[0027]

[0028] Among them, Y represents the number of abnormal categories;

[0029] For personalized conditions, enter p i , which contains the local Non-IID data information of client i; the proportion of abnormal category y on client i is:

[0030]

[0031] Among them, Φ{·} is the indicator function;

[0032] Therefore, the personalized condition input p i Defined as:

[0033]

[0034] Next, GCE will guide the global feature vector The learning process of X is as follows; Specifically, two additional loss terms are introduced for model update; Since the client shares the global category embedding GCE(y; E), it will guide X i G Embedded GCE (Y i ; E), away from other categories of embedding; therefore, the contrast loss is introduced, which is defined as:

[0035]

[0036] in, represents the contrast loss in the local training step t of client i; represents the cosine similarity between a and b;

[0037] guide Close to freezing global category embedding To improve the model training performance; introduce the magnitude loss, which is defined as:

[0038]

[0039] in, represents the magnitude loss of client i in the t-th step of local training.

[0040] Furthermore, the local update is implemented as follows:

[0041] Due to Non-IID data, the client personalizes training of a head For learning feature information; for the anomaly detection task, the cross entropy (CE) loss is defined as:

[0042]

[0043] in, is the CE loss of the local training step t of client i, δ represents the CE loss function;

[0044] Considering contrast, magnitude and CE losses, the total loss of client i at the tth step is:

[0045]

[0046] Therefore, the update of the local model is:

[0047]

[0048] Among them, ρ is the learning rate;

[0049] Calculate the updated gradients ▽ξ of the two loss functions i,t and The cosine similarity cosim(·) of ; when the value of cosim(·) is greater than the hyperparameter γ, it is used in the next local update Replacement i,t ; In this case, the update of the local model is translated into:

[0050]

[0051] Compared with the prior art, the present invention has the following beneficial effects: the present invention provides an edge anomaly detection method based on global-personalized collaborative federated learning, which takes into account clients with different computing power and provides a personalized model for the client; the method uses conditional computing components and global category embedding modules to introduce contrast and magnitude losses to guide the learning of personalized and global feature vectors, and uses loss replacement strategies to accelerate model training. The present invention can achieve higher detection accuracy while shortening training time, and shows better scalability and convergence. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a comparison between classic FL and this method;

[0053] Figure 2 is a schematic diagram of a method implementation principle of an embodiment of the present invention;

[0054] Figure 3 is a real-world test bed for FedGPA in an embodiment of the present invention;

[0055] Figure 4 It is a scalability comparison of different FL frameworks on the ACI-IOT-2023 dataset in the embodiments of the present invention;

[0056] Figure 5 This is a comparison of the convergence of different FL frameworks on the ACI-IOT-2023 dataset in the embodiments of the present invention. DETAILED DESCRIPTION

[0057] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0058] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0059] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0060] In order to solve the problems existing in the prior art, the present invention proposes a method for edge anomaly detection based on global-personalized collaborative federated learning (FedGPA). Figure 1 As shown in (c), the FedGPA method proposed in the present invention takes into account clients with different computing powers and provides personalized models for the clients.

[0061] Figure 2 The implementation principle of this method is shown. This method considers clients with different computing power and customizes personalized anomaly detection models for them. In global aggregation, aggregation is performed according to the weight of the client. In local training, the feature conversion module is used to generate global and personalized feature vectors, and then the global class embedding module and the head are used for global-personalized collaborative learning. At the same time, the cosine similarity of the cross entropy loss and the overall loss update gradient is used to determine whether to replace the loss.

[0062] The edge anomaly detection method based on global-personalized collaborative federated learning provided in this embodiment includes the following steps:

[0063] (1) After the edge server generates the global model, it delegates the global model to the client; the client initializes the local model based on the global model;

[0064] (2) Using the conditional computation component on the client side to convert traffic features into global and personalized feature vectors;

[0065] (3) The client introduces contrast and magnitude losses in the global category embedding module and guides the learning of the global feature vector based on the embedding of the sample category, and uses the cross entropy loss to guide the learning of the personalized feature vector;

[0066] (4) The client determines whether to perform loss replacement by calculating the cosine similarity of the cross entropy loss and the overall loss update gradient, thereby accelerating the training of the model;

[0067] (5) After completing local training, the client uploads the model update to the edge server; the edge server performs global aggregation and updates the global model; the above steps are repeated until the anomaly detection model converges.

[0068] In this system, there are N IoT devices (i.e., clients) with statistical and system heterogeneity. For each round of FL learning, the edge server randomly selects a subset of clients S r , its client downloads the global model from the edge server. Then, the client trains the model using its local data. The training dataset of client i is recorded as where y j ∈{1,2,...,Y} is the sample x j The label of , Y represents the number of abnormal types. Therefore, the optimization goal of the local model is:

[0069]

[0070] Among them, ξ i W represents the loss function of client i. i and F i They represent the personalized model and local objective function of client i respectively.

[0071] After completing the local update, the model parameters will be uploaded to the edge server. Finally, the edge server updates the global model by aggregating the client models. Considering the differences between different clients, a personalized anomaly detection model is customized for the client. Therefore, the optimization goal of the global model is:

[0072]

[0073] in, represents the weight of client i, Indicated as S r A personalized model customized for each client.

[0074] Based on the constructed system, this method integrates global and personalized learning to efficiently solve statistical and system heterogeneous problems. Figure 2 As shown, the last fully connected (FC) layer of the client's local model is used as the head The rest is used as feature extractor ψ(·; W f ). First, ψ(·; W f ) Extract training sample X of client i i features and maps them into a new feature vector X' i For the local training step t = 1, the cosine similarity cosim(·) is not calculated, but the feature transformation module (i.e., FeT) is directly used to transform X' i Convert to global eigenvector and personalized feature vector then, and are input into the global category embedding module (i.e., GCE) and For global and personalized learning. In GCE, will be guided to be close to its corresponding global category embedding GCE (Y i ; E) and freeze global category embeddings To learn Finally, each client performs personalized training and fine-tunes χ to learn To improve training efficiency, for local training at step t>1, the cosim(·) of the cross entropy loss and the overall loss update gradient in step t-1 is calculated to decide whether to replace the loss function. In each round of training, the client shares the category embedding E, the conditional computation component T, and the feature extractor ψ, and the client copies the frozen category embedding after receiving E.

[0075] Feature Transformation Module (FeT): Feature extractor ψ(·; W f ) The training sample X i Mapped to X' i To coordinate global and personalized learning, X' i Converted to global eigenvectors and personalized feature vector

[0076]

[0077] Among them, σ is the ReLU activation function, and μ is the Hadamard product. α, β, α i and β i is the mapping parameter, which is generated by FeT:

[0078]

[0079] Among them, FeT contains two learnable components T with the same structure but different parameters. α and T β . T α and T β It consists of FC layer, ReLU activation layer and normalization layer. g and p i are the global and personalized conditional inputs generated by GCE. By replacing g and p i Input to T α and T β Can generate α, β, α i and β i .

[0080] Global Category Embedding Module (GCE): GCE contains a learnable category embedding E for generating g and p i , to guide global learning. Before each round of training E, the client copies E to generate frozen category embeddings They are used to generate the global category embedding GCE(y;E) and frozen global category embedding for each abnormal category y First, by averaging all anomaly categories To generate global conditional input:

[0081]

[0082] Among them, Y represents the number of abnormal categories.

[0083] For personalized conditions, enter p i , which contains the local Non-IID data information of client i. The proportion of abnormal category y on client i is:

[0084]

[0085] Wherein, Φ{·} is an indicator function.

[0086] Therefore, the personalized condition input p i is defined as:

[0087]

[0088] Next, GCE will guide the global feature vector Specifically, two additional loss terms are introduced for model updating. Since the client shares the global category embedding GCE(y; E), it will guide X i G Embedded GCE (Y i ; E), away from other categories of embedding. Therefore, a contrast loss is introduced, which is defined as:

[0089]

[0090] in, represents the contrast loss in the local training step t of client i. Represents the cosine similarity between a and b.

[0091] Further, guide Close to freezing global category embedding To improve the model training performance. Specifically, a magnitude loss is introduced, which is defined as:

[0092]

[0093] in, represents the magnitude loss of client i in the t-th step of local training.

[0094] Local Personalization Update: Due to Non-IID data, the client personalizes training to a head For learning X i PFor the anomaly detection task, the cross entropy (CE) loss is defined as:

[0095]

[0096] in, is the CE loss of the local training step t of client i, and δ represents the CE loss function.

[0097] Considering the contrast, magnitude and CE losses, the total loss of client i at the tth step is:

[0098]

[0099] Therefore, the update of the local model is:

[0100] W i,t+1 =W i,t -ρ▽ξ i,t (12)

[0101] Among them, ρ is the learning rate.

[0102] The above design can effectively solve the statistical heterogeneity problem. However, due to the system heterogeneity problem, clients with insufficient computing power may not be able to complete local model training in time, which seriously affects the aggregation efficiency. To solve this problem, this method designs a loss replacement strategy. Specifically, the updated gradient ▽ξ of the two loss functions is calculated. i,t and When the value of cosim(·) is greater than the hyperparameter γ, it will be used in the next local update. Replacement i,t . In this case, the update of the local model translates to:

[0103]

[0104] Through the designed loss replacement strategy, the efficiency of FL training in the device heterogeneous edge environment can be effectively improved while ensuring high anomaly detection accuracy.

[0105] The training process of this method is shown in Algorithm 1. For the edge server, first initialize the header x of each client i i and model parameters (Line 1). At the beginning of each round, a subset of clients S is selected r And calculate the weight n of each client i (Line 3). Then, the r Client i receives W f,r 、E r and T rThen, ClientUpdate(·) is executed in parallel and the locally trained model parameters are returned (line 5). Finally, the edge server performs global aggregation based on the client weights and the uploaded model parameters, and outputs the client’s personalized model {W1, W2, ..., W N}(Lines 6-7). Each client executes ClientUpdate(·). First, initialize cs and generate g and p according to the global model. i (Line 8). When the local step number t = 1 or cs = 0, the overall loss ξ is used i,t Update each model (lines 10-11). Next, calculate the loss function ξ i,t and Update the cosine similarity of the gradient cosim(·). If the value of cosim(·) is greater than γ, then cs = 1 (line 12). When the local step number t>1 and cs = 1, use renew and (Lines 13-14). Finally, the locally updated and Upload to the edge server (line 15).

[0106]

[0107] Method evaluation:

[0108] like Figure 3 As shown, this embodiment builds a hardware test bed to evaluate the performance of FedGPA in real scenarios, which consists of a workstation and 10 clients of two types. The workstation is regarded as an edge server, equipped with two NVIDIA GeForce GTX 3090 GPUs, an Intel (R) Xeon (R) CPU Silver 4208 @ 2.1GHz, and a memory size of 32GB. Raspberry Pi 4Bs is a client of one type, equipped with a quad-core Broadcom BCM2711SoC @ 1.5GHz and a memory size of 4GB. Jetson TX2s is another client, equipped with a 256-core NVIDIA Pascal (R) GPU and a processing cluster consisting of a dual-core NVIDIA Denver 2CPU and a quad-core Arm Cortex-A57 MPCore processor, and a memory size of 8GB. The operating system of the hardware device is Ubuntu 18.04. In addition, all devices are located in the same local area network and communicate through a 5GHz WiFi network. By building a real-world test bed, an edge environment with statistical and system heterogeneity is simulated.

[0109] In order to evaluate the performance of FedGPA, this embodiment uses the following three benchmark traffic data sets.

[0110] (1) ACI-IOT-2023: This dataset is a typical home IoT network simulation data provided by the U.S. Army Cyber ​​Institute (ACI) in the IoT Research Laboratory (IoTRL). It contains various attack records within a week, covering reconnaissance (Recon), denial of service (DoS) attacks, brute force cracking (Brute Force), and spoofing (Spoofing).

[0111] (2) CIC-IOT-2023: This dataset is a simulation of 105 attacks performed by the Canadian Institute for Cybersecurity (CIC) in an IoT topology consisting of 33 devices. These attacks include DDoS, DoS, reconnaissance, web-based, brute force, spoofing, and Mirai.

[0112] (3)USTC-TFC-2016: This dataset is provided by the University of Science and Technology of China (USTC) and contains network traffic data of different applications. Normal traffic includes Facetime, FTP, and MySQL, and abnormal traffic includes Neris, Nsis, and Virut.

[0113] The original ACI-IoT-2023 dataset contains 1 type of normal traffic and 4 types of abnormal traffic. The original CIC-IOT-2023 dataset contains 1 type of normal traffic and 7 types of abnormal traffic. The original USTC-TFC-2016 dataset contains 10 types of normal traffic and 10 types of abnormal traffic. First, the original traffic data is preprocessed. Specifically, the original traffic data is split into independent sessions according to the five-tuple (i.e., source IP, destination IP, source port, destination port, and transport layer protocol). For each session, the key feature information of the first N packets is extracted, including the payload length (Payload Length) and the inter-arrival time (Inter-arrival Time). In order to maintain the continuity of the training samples, the inter-arrival time of the first packet is set to 0. When extracting the packet features, the DNS packets are discarded and filled with empty packets. This is because the DNS packets mainly contain the mapping information from the domain name to the IP address, which is irrelevant to the abnormal traffic detection. Then, the extracted features are arranged in chronological order to obtain the training samples, and the first 8 packets of each session (i.e., N=8) are extracted for abnormal traffic detection. In the CIC-IOT-2023 dataset, distributed DoS attacks and DoS attacks are combined into one category, and three categories with too few samples are removed. In the USTC-TFC-2016 dataset, 10 types of normal traffic are classified into one category, and four types of abnormal traffic with too few samples are removed. The description of the dataset is shown in Table 1. Finally, a Non-IID local dataset is assigned to each client, where the dataset is sampled by label q i,c ~Dirichlet(α),α=0.3,q i,c It represents the proportion of data with label c assigned to client i. The client local dataset includes training set (60%), validation set (20%) and test set (20%).

[0114] Table 1 Dataset description

[0115]

[0116] The proposed FedGPA is compared with the following four state-of-the-art FL frameworks.

[0117] (1) FedAvg: As one of the most classic FL, FedAvg averages and aggregates the model parameters of all clients and provides a global model for them.

[0118] (2) FedALA: FedALA introduces an adaptive local aggregation (ALA) module that adaptively aggregates the global and local models according to the client's local objectives.

[0119] (3) FedPer: FedPer divides the model into a base layer and a personalized layer. In each round, only the parameters of the base layer are aggregated, and the personalized layer is only updated in the local personalized task.

[0120] (4) GPFL: As a variant of FedPer, GPFL can learn global and personalized feature information simultaneously.

[0121] Based on the above FL framework, this embodiment uses the following three classic deep learning models for anomaly detection.

[0122] (1) 1D-CNN: As a convolutional neural network for processing sequence data, 1D-CNN can automatically extract the key features of the sequence and use it for classification tasks.

[0123] (2) GRU: As an improved recurrent neural network (RNN), GRU controls the flow of information by introducing a gating mechanism to capture long-term dependencies.

[0124] (3)FormerTime: As a Transformer-based method, FormerTime shortens the time series length and efficiently extracts features for classification through multi-scale representation and self-attention mechanism.

[0125] Based on PyTorch 1.11.0, the proposed FedGPA and other state-of-the-art methods are implemented in this paper. To ensure fairness, for all FL frameworks, the number of global communication rounds is set to 100, the number of client local training steps is set to 10, the training batch size is set to 64, the learning rate is set to 0.005, and the customer participation rate is set to 0.8. For each anomaly detection model, the 1D-CNN consists of 2 convolutional layers and 2 fully connected layers, where each convolutional layer consists of a convolutional layer, a normalization layer, a ReLU activation layer, and a pooling layer, with a convolution kernel size of 3, a stride of 2, a padding of 1, and a step size of 2 for the pooling layer. The GRU consists of 2 hidden layers and 2 FC layers, where the hidden layer size is 32. For FormerTime, the number of self-attention heads is 4, the number of layers is 3, the hidden size of each layer is 64, the stride is 2, the slice size is 2, and the dropout rate is 0.2. Based on the above settings, different anomaly detection models are run on different FL frameworks and the accuracy and training time of each model are tested to evaluate the performance of FedGPA proposed in the present invention.

[0126] Table 2 Comparison of detection accuracy (%) and training time (s) of different models on different FL frameworks

[0127]

[0128] As shown in Table 2, this embodiment compares the accuracy and efficiency of different anomaly detection models on different FL frameworks. For the ACI-IOT-2023 dataset, FedGPA has an accuracy of 98.67% when using 1D-CNN, which is almost the same as GPFL with the highest accuracy, but the training time is only about 60% of GPFL. When using GRU and FormerTime, FedAvg only achieved accuracies of 65.97% and 88.74%, respectively. In contrast, FedGPA not only achieved the highest accuracies of 99.89% and 99.90%, respectively, but also had comparable training time to FedAvg. Although GPFL achieved similar accuracy to FedGPA, the training time was significantly longer. For FedPer and FedALA, although their training time was slightly shorter than FedGPA when using 1D-CNN or GRU, FedGPA showed a clear advantage in accuracy. In particular, when using GRU, FedGPA improves the accuracy by about 19.76% compared with FedPer, and when using FormerTime, FedGPA outperforms them in both performance indicators. For the CIC-IOT-2023 dataset, when using GRU, FedGPA's accuracy is slightly lower than GPFL, but the training time is reduced by 5.67s. Compared with other methods, when using 1D-CNN and FormerTime, FedGPA's accuracy is improved by about 0.05-19.05%. For the USTC-TFC-2016 dataset, FedGPA achieves the highest accuracy when using different anomaly detection models and is close to FedAvg in training time. Overall, FedGPA significantly reduces model training time while ensuring high detection accuracy. FedAvg provides a global model for all clients, so it cannot adapt to dynamic and heterogeneous edge environments. FedALA aggregates the model locally for initialization, and FedPer splits the model for personalized fine-tuning. They all provide personalized models for clients but do not combine global collaborative learning, so they cannot fully utilize the global feature information of different clients. Similar to FedPer, GPFL splits the model and considers the feature information of other clients, but it does not consider the device heterogeneity in complex edge environments, resulting in clients with smaller computing power being unable to participate in federated aggregation in a timely manner. FedGPA proposed in the present invention achieves excellent performance, thanks to its global-personalized collaborative learning, and improves training efficiency through loss replacement.

[0129] like Figure 4 As shown in Figure 2, this embodiment tests the scalability of different FL frameworks under different numbers of clients on the ACI-IOT-2023 dataset. In general, the FedGPA proposed in the present invention shows superior and stable anomaly detection accuracy in scenarios with different numbers of clients. Specifically, Figure 4As shown in (b), as the number of clients changes, the accuracy of FedGPA fluctuates by about 8.8% when using GRU. In comparison, FedAvg, FedPer, FedALA, and GPFL fluctuate by about 25.33%, 16.58%, 33.07%, and 18.99%, respectively, and their accuracy is lower than that of FedGPA. Figure 4 As shown in (a) and (c), when other anomaly detection models are used, FedGPA can still maintain a stable high accuracy, while other FL frameworks have a certain degree of performance fluctuation. For example, when 1D-CNN is used on FedALA, there is a performance fluctuation of about 43.39%. As the number of clients increases, the accuracy of most FL frameworks increases first and then decreases. This is because when the number of clients is small, the training data is limited, and the anomaly detection model cannot be fully trained or overfits. When the number of clients is large, statistical and system heterogeneity problems become more prominent, resulting in training deviations, which in turn affects the anomaly detection performance.

[0130] like Figure 5 As shown in Figure 2, this embodiment tests the convergence of different FL frameworks from the perspective of average training loss on the ACI-IOT-2023 dataset. In the FedGPA proposed in the present invention, the contrast loss ξ is introduced. cl and magnitude loss ξ ml To ensure the fairness of the comparison, these two losses are hidden and only the cross entropy loss is considered. The results show that FedGPA can achieve faster convergence than FedAvg and show lower training loss compared with other FL frameworks. Figure 5 As shown in (a), compared with FedAvg, FedGPA has a significant convergence advantage in the first 10 rounds. After 30 rounds, FedGPA can converge to the lowest loss and tend to be stable. Figure 5 As shown in (b), FedPer converges more stably than other FL frameworks. However, the detection accuracy of FedPer is only about 80.13% because FedPer suffers from overfitting. Figure 5 As shown in (c), FedALA exhibits large performance fluctuations. Due to statistical and system heterogeneity, the training process of each FL framework exhibits a certain degree of fluctuation. The FedGPA proposed in the present invention can adapt well to different anomaly detection models and make them fast and stable, verifying its advantage in convergence.

[0131] Table 3 Effect of hyperparameter γ on FedGPA performance

[0132]

[0133] As shown in Table 3, this embodiment tests the effect of the hyperparameter γ on the performance of FedGPA, where 1D-CNN is used as an anomaly detection model. γ represents the threshold of cosine similarity cosim(·). The results show that the model accuracy is positively correlated with γ, while the training time is negatively correlated with γ. When γ = 0.95, the accuracy of FedGPA reaches a peak of 98.67% and 97.83% on both the ACI-IOT-2023 and CIC-IOT-2023 datasets, respectively, and the training time is 13.77s and 5.47s, respectively. Although the training time is slightly increased compared to other values ​​of γ, this is acceptable while ensuring high accuracy. For the USTC-TFC-2016 dataset, when γ = 0.98, FedGPA achieves the highest accuracy of 96.05%, which is only 0.01% higher than when γ = 0.95, but the training time increases by 0.83s. Therefore, the setting of γ should appropriately balance model accuracy and training efficiency according to actual needs.

[0134] The edge anomaly detection method based on global-personalized collaborative FL proposed in the present invention can perform global and personalized learning at the same time by designing feature conversion modules, global category embedding modules and loss replacement strategies, solving statistical and system heterogeneity problems and improving the training efficiency of the model. Based on real-world test beds and traffic data sets, a large number of experiments have verified the superiority of FedGPA proposed in the present invention. The results show that compared with the most advanced (state-of-the-art) methods, FedGPA can shorten the training time while maintaining high detection accuracy. At the same time, FedGPA shows better scalability and convergence.

[0135] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0136] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0137] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0138] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0139] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A method for edge anomaly detection based on global-personalized collaborative federated learning, characterized in that: The following steps are involved: (1) After the edge server generates the global model, it delegates the global model to the client; the client initializes the local model based on the global model; (2) Using the conditional computation component on the client side to convert traffic features into global and personalized feature vectors; (3) The client introduces contrast and magnitude losses in the global category embedding module and guides the learning of the global feature vector based on the embedding of the sample category, and uses the cross entropy loss to guide the learning of the personalized feature vector; (4) The client determines whether to perform loss replacement by calculating the cosine similarity of the cross entropy loss and the overall loss update gradient, thereby accelerating the training of the model; (5) After completing local training, the client uploads the model update to the edge server; the edge server performs global aggregation and updates the global model; the above steps are repeated until the anomaly detection model converges.

2. The edge anomaly detection method based on global-personalized collaborative federated learning according to claim 1 is characterized in that: For a system with N statistically and systematically heterogeneous IoT devices, i.e., clients, for each round of FL learning, the edge server randomly selects a subset of clients S r , its client downloads the global model from the edge server; then, the client trains the model using its local data; the training dataset of client i is recorded as where y j ∈{1,2,...,Y} is the sample x j The label of , Y represents the number of abnormal types; therefore, the optimization objective of the local model is: Among them, ξ i represents the loss function of client i; W i and F i denote the personalized model and local objective function of client i respectively; After completing the local update, the model parameters will be uploaded to the edge server. Finally, the edge server updates the global model by aggregating the client models. Considering the differences between different clients, a personalized anomaly detection model is customized for the client. Therefore, the optimization goal of the global model is: in, represents the weight of client i, Indicated as S r A personalized model customized for each client.

3. The edge anomaly detection method based on global-personalized collaborative federated learning according to claim 2 is characterized in that: The last fully connected (FC) layer of the client’s local model is used as the head χ(·; W i h ), and the rest is used as feature extractor ψ(·; W f ); First, ψ(·; W f ) Extract training sample X of client i i features and maps them into a new feature vector X i '; For the local t = 1 step training, the cosine similarity cosim(·) is not calculated, but the feature conversion module FeT is directly used to convert X i 'Convert to global feature vector and personalized feature vector then, and are input into the global category embedding module GCE and for global and personalized learning; In GCE, is guided to be close to its corresponding global category embedding GCE(Y i ; E) and freeze global category embeddings To learn Global feature information of Finally, each client performs personalized training and fine-tunes χ to learn Personalized feature information; To improve training efficiency, for local training step t>1, the cosim(·) of the cross entropy loss and the overall loss update gradient in step t-1 is calculated to decide whether to replace the loss function; in each round of training, the client shares the category embedding E, the conditional computation component T, and the feature extractor ψ, and the client copies the frozen category embedding after receiving E 4. The edge anomaly detection method based on global-personalized collaborative federated learning according to claim 3 is characterized in that: The implementation method of the feature conversion module FeT is: Feature extractor ψ(·; W f ) The training sample X i Mapped to X i '; In order to coordinate global and personalized learning, X i 'Convert to global eigenvectors respectively and personalized feature vector Among them, σ is the ReLU activation function, μ is the Hadamard product; α, β, α i and β i is the mapping parameter, which is generated by FeT: Among them, FeT contains two learnable components T with the same structure but different parameters. α and T β ; T α and T β It consists of FC layer, ReLU activation layer and normalization layer; g and p i is the global and personalized conditional input generated by GCE; by combining g and p i Input to T α and T β Generate α, β, α i and β i .

5. The edge anomaly detection method based on global-personalized collaborative federated learning according to claim 4 is characterized in that: The implementation method of the global category embedding module GCE is: GCE contains a learnable category embedding E for generating g and p i , to guide global learning; before each round of training E, the client copies E to generate frozen category embeddings They are used to generate the global category embedding GCE(y;E) and frozen global category embedding for each abnormal category y First, by averaging all anomaly categories To generate global conditional input: Among them, Y represents the number of abnormal categories; For personalized conditions, enter p i , which contains the local Non-IID data information of client i; the proportion of abnormal category y on client i is: Among them, Φ{·} is the indicator function; Therefore, the personalized condition input p i Defined as: Next, GCE will guide the global feature vector Specifically, two additional loss terms are introduced for model updating. Since the client shares the global category embedding GCE(y; E), it will guide Embedded GCE (Y i ; E), away from other categories of embedding; therefore, the contrast loss is introduced, which is defined as: in, represents the contrast loss in the local training step t of client i; represents the cosine similarity between a and b; guide Close to freezing global category embedding To improve the model training performance; introduce the magnitude loss, which is defined as: in, represents the magnitude loss of client i in the t-th step of local training.

6. The edge anomaly detection method based on global-personalized collaborative federated learning according to claim 5 is characterized in that: The implementation method of local update is: Due to the Non-IID data, the client personalizes the training of a head χ(·; W i h ) for learning feature information; for the anomaly detection task, the cross entropy (CE) loss is defined as: in, is the CE loss of the local training step t of client i, δ represents the CE loss function; Considering contrast, magnitude and CE losses, the total loss of client i at the tth step is: Therefore, the update of the local model is: Among them, ρ is the learning rate; Calculate the updated gradients of the two loss functions and The cosine similarity cosim(·) of ; when the value of cosim(·) is greater than the hyperparameter γ, it is used in the next local update Replacement i,t ; In this case, the update of the local model is translated into:

Images