Edge Anomaly Detection Method Based on Global-Personalized Collaborative Federated Learning

Abstract

This invention relates to an edge anomaly detection method based on global-personalized collaborative federated learning, comprising: an edge server generating a global model and then distributing it; a client initializing a local model based on the global model; the client using a conditional computation component to convert traffic features into global and personalized feature vectors; the client introducing contrastive and magnitude losses in the global category embedding module and guiding the learning of global feature vectors based on the embedding of sample categories, and using cross-entropy loss to guide the learning of personalized feature vectors; the client determining whether to perform loss replacement by calculating the cosine similarity of the gradient updates of the cross-entropy loss and the overall loss, thereby accelerating model training; the client uploading the updated model to the edge server after completing local training; the edge server performing global aggregation and updating the global model; repeating the above steps until the anomaly detection model converges. This method is beneficial for improving the accuracy and efficiency of edge anomaly detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of edge anomaly detection, and particularly relates to an edge anomaly detection method based on global-personalized collaborative federated learning. BACKGROUND

[0002] The rapid development of 5G technology provides strong support for Internet-of-Things (IoT) related application fields (such as autonomous driving, smart city and environmental monitoring), and also leads to a sharp increase in the number of IoT devices. IoT devices contain a large amount of private data, and their security and reliability are crucial to individuals and groups. In recent years, the number of cases of IoT devices being attacked has been on the rise, so anomaly detection plays an important role in ensuring IoT security. In classic cloud computing, massive traffic data is transmitted from IoT devices to remote clouds for detection, which not only consumes a large amount of network bandwidth, but also causes significant delay, seriously affecting user experience. In order to alleviate this problem, Mobile Edge Computing (MEC) reduces the dependence of IoT devices on remote clouds by sinking computing and storage resources to the network edge, thereby improving bandwidth utilization and transmission delay. In MEC, IoT devices need to upload their data to edge servers for processing, which still has the problem of privacy leakage. At the same time, traffic data is distributed across different IoT devices, and if an anomaly detection model is trained locally, it faces the problem of data silos. Emerging federated learning (FL) is considered an effective way to solve the problems of privacy leakage and data silos, and has been preliminarily applied to the field of edge anomaly detection.

[0003] Figure 1 A classic FL is shown. In (a) of FIG. 1, all clients have the same computing power and aggregation weights, and the edge server provides a unified global model. In (b) of FIG. 1, the aggregation weights of the clients are different, and the edge server provides personalized models for the clients. As shown in (c) of FIG. 1, the edge server provides a global model and personalized models for the clients. Figure 1 Figure 1 Figure 1 ​​As shown in (a), in classic FL (e.g., FedAvg), the IoT device (i.e., the client) first downloads the global model from the edge server and trains a local model using local data. Then, the local model parameters are uploaded to the edge server, which averages all local model parameters to update the global model and proceed to the next round of training. Therefore, the IoT device does not need to directly upload its local data to the edge server, protecting user privacy. However, classic FL provides a uniform global model for all clients, failing to consider the differences between different clients. Furthermore, classic FL is designed for independent and identically distributed (IID) data, making it difficult to effectively handle non-independent and identically distributed (Non-IID) data. Figure 1 As shown in (b), personalized FL (pFL), a new paradigm of FL, can provide a personalized model for each client. pFL can personalize the model based on the client's local data, thereby improving the model's performance on different clients.

[0004] For edge anomaly detection, most existing studies have not considered the issue of Non-IID data (i.e., statistical heterogeneity) and use FedAvg for global model training, resulting in performance that falls short of expectations across different IoT devices. Furthermore, in complex edge environments, the diversity of IoT devices leads to significant differences in their computing power (i.e., system heterogeneity). IoT devices with insufficient computing power train models slowly, significantly increasing the time required to complete a single FL aggregation, which severely impacts the training efficiency of the global model. Therefore, it is necessary to design an efficient pFL framework suitable for heterogeneous edge environments to improve edge anomaly detection accuracy while accelerating model training. Summary of the Invention

[0005] The purpose of this invention is to provide an edge anomaly detection method based on global-personalized collaborative federated learning, which is beneficial to improving the accuracy and efficiency of edge anomaly detection.

[0006] To achieve the above objectives, the technical solution adopted by this invention is: an edge anomaly detection method based on global-personalized collaborative federated learning, comprising the following steps:

[0007] (1) After the edge server generates the global model, it distributes the global model; the client initializes the local model based on the global model.

[0008] (2) Use the conditional calculation component on the client side to convert traffic features into global and personalized feature vectors;

[0009] (3) The client introduces contrast and magnitude loss in the global category embedding module and guides the learning of global feature vectors based on the embedding of sample categories, and uses cross-entropy loss to guide the learning of personalized feature vectors.

[0010] (4) The client determines whether to perform loss replacement by calculating the cosine similarity of the gradient update between the cross-entropy loss and the overall loss, thereby accelerating the training of the model;

[0011] (5) After the client completes local training, it uploads the model update to the edge server; the edge server performs global aggregation and updates the global model; repeat the above steps until the anomaly detection model converges.

[0012] Furthermore, for a system with N IoT devices exhibiting statistical and systemic heterogeneity (i.e., clients), for each round of FL learning, the edge server randomly selects a subset S of clients. r The client downloads the global model from the edge server; then, the client trains the model using its local data; the training dataset for client i is denoted as . Where y j ∈{1,2,...,Y} is a sample x j The label Y represents the number of exception types; therefore, the optimization objective of the local model is:

[0013]

[0014] Where, ξ i W represents the loss function for client i; i and F i Let i represent the personalized model and the local objective function of client i, respectively.

[0015] After the local update is complete, the model parameters will be uploaded to the edge server. Finally, the edge server updates the global model by aggregating the models from different clients. Considering the differences between different clients, personalized anomaly detection models are customized for each client. Therefore, the optimization objective of the global model is:

[0016]

[0017] in, Let {W1, W2, ..., W} represent the weights of client i. Sr} is represented as S r A personalized model customized for each client.

[0018] Furthermore, the last fully connected (FC) layer of the client's local model is used as the header. The remaining part serves as the feature extractor ψ(·;W) f First, ψ(·;Wf Extract training samples X from client i. i Features and map them to a new feature vector X' i For the local training step t=1, the cosine similarity cosim(·) is not calculated; instead, the feature transformation module FeT is used directly to transform X' i Convert to global feature vector and personalized feature vectors then, and They are respectively input into the Global Category Embedding Module (GCE) and To enable global and personalized learning; in GCE, Guided to approach its corresponding global category embedding GCE(Y) i ;E) and frozen global category embedding To learn The global feature information is obtained; finally, each client undergoes personalized training, and the χ² value is fine-tuned to learn. Personalized feature information; to improve training efficiency, for local training at step t>1, cosim(·) of the cross-entropy loss and overall loss update gradient in step t-1 is calculated to determine whether to replace the loss function; in each round of training, the client shares the class embedding E, the conditional computation component T, and the feature extractor ψ, and the client obtains the frozen class embedding after receiving E.

[0019] Furthermore, the implementation method of the feature transformation module FeT is as follows:

[0020] Feature extractor ψ(·;W f ) train sample X i Mapped to X' i To coordinate global and personalized learning, X' i Convert them into global feature vectors respectively and personalized feature vectors

[0021]

[0022] Where σ is the ReLU activation function, μ is the Hadamard product; α, β, α i and β i These are the mapping parameters, generated by FeT:

[0023]

[0024] FeT contains two learnable components T with identical structures but different parameters. α and T β ;T α and Tβ It consists of an FC layer, a ReLU activation layer, and a normalization layer; g and p i Global and personalized conditional inputs generated by GCE; by using g and p i Input to T α and T β Generate α, β, α i and β i .

[0025] Furthermore, the implementation method of the Global Category Embedding Module (GCE) is as follows:

[0026] GCE contains learnable class embeddings E for generating g and p. i To guide global learning; before each round of training E, the client copies E to generate frozen class embeddings. The global class embedding GCE(y;E) and the frozen global class embedding are used to generate the global class embedding for each anomaly class y, respectively. First, by averaging all anomaly categories To generate global conditional input:

[0027]

[0028] Where Y represents the number of anomaly categories;

[0029] For personalized conditional input p i It contains local Non-IID data information of client i; the proportion of anomaly category y on client i is:

[0030]

[0031] Where Φ{·} is the indicator function;

[0032] Therefore, personalized conditional input p i Defined as:

[0033]

[0034] Next, GCE will guide the global feature vector. The learning process; specifically, two additional loss terms are introduced for model updates; since the client shares the global class embeddings GCE(y; E), X will be guided... i G Embeddings of GCE (Y) close to its category i E), far from other class embeddings; therefore, a contrastive loss is introduced, which is defined as:

[0035]

[0036] in, This represents the contrastive loss during the t-th step of training on client i. This represents the cosine similarity between a and b;

[0037] guide Near-freeze global category embedding To improve model training performance, an order-of-magnitude loss is introduced, defined as:

[0038]

[0039] in, This represents the magnitude of the loss during the t-th step of training on client i.

[0040] Furthermore, the local update is implemented as follows:

[0041] Due to the use of Non-IID data, the client-side personalized training of a header is performed. For learning The feature information; for anomaly detection tasks, the cross-entropy (CE) loss is defined as:

[0042]

[0043] in, δ is the CE loss of client i during the t-th local training step, where δ represents the CE loss function;

[0044] Considering the comparison, magnitude, and CE loss, the total loss of client i at step t is:

[0045]

[0046] Therefore, the local model is updated as follows:

[0047]

[0048] Where ρ is the learning rate;

[0049] Calculate the update gradients ▽ξ of the two loss functions. i,t and The cosine similarity is calculated using cosim(·); when the value of cosim(·) is greater than the hyperparameter γ, it is used in subsequent local updates. Substitution ξ i,t In this case, the update of the local model is transformed into:

[0050]

[0051] Compared with existing technologies, this invention has the following advantages: This invention provides an edge anomaly detection method based on global-personalized collaborative federated learning. This method considers clients with different computing power and provides personalized models for each client. The method uses a conditional computation component and a global category embedding module to introduce contrastive and magnitude losses to guide the learning of personalized and global feature vectors, and uses a loss replacement strategy to accelerate model training. This invention can achieve higher detection accuracy while shortening training time, and exhibits better scalability and convergence. Attached Figure Description

[0052] Figure 1 This is a comparison between the classic FL method and our method;

[0053] Figure 2 This is a schematic diagram illustrating the implementation principle of the method according to an embodiment of the present invention;

[0054] Figure 3 This is a real-world testbed from FedGPA, as described in this embodiment of the invention.

[0055] Figure 4 This is a comparison of the scalability of different FL frameworks on the ACI-IOT-2023 dataset in this embodiment of the invention;

[0056] Figure 5 This is a comparison of the convergence of different FL frameworks on the ACI-IOT-2023 dataset in this embodiment of the invention. Detailed Implementation

[0057] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0058] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0059] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0060] To address the problems of existing technologies, this invention proposes a federated learning with global-personalized collaboration for edge anomaly detection (FedGPA). Figure 1 As shown in (c), the FedGPA method proposed in this invention takes into account clients with different computing power and provides personalized models for clients.

[0061] Figure 2 The implementation principle of this method is illustrated. This method considers clients with different computing power and customizes personalized anomaly detection models for them. In global aggregation, aggregation is performed based on the client's weights. In local training, a feature transformation module is used to generate global and personalized feature vectors, and then a global-personalized collaborative learning is performed using a global class embedding module and a head. Simultaneously, the cosine similarity of the gradient updates based on the cross-entropy loss and the overall loss determines whether to perform loss replacement.

[0062] The edge anomaly detection method based on global-personalized collaborative federated learning provided in this embodiment includes the following steps:

[0063] (1) After the edge server generates the global model, it distributes the global model; the client initializes the local model based on the global model.

[0064] (2) Use the conditional calculation component on the client side to convert traffic features into global and personalized feature vectors;

[0065] (3) The client introduces contrast and magnitude loss in the global category embedding module and guides the learning of global feature vectors based on the embedding of sample categories, and uses cross-entropy loss to guide the learning of personalized feature vectors.

[0066] (4) The client determines whether to perform loss replacement by calculating the cosine similarity of the gradient update between the cross-entropy loss and the overall loss, thereby accelerating the training of the model;

[0067] (5) After the client completes local training, it uploads the model update to the edge server; the edge server performs global aggregation and updates the global model; repeat the above steps until the anomaly detection model converges.

[0068] In this system, there are N IoT devices (i.e., clients) with statistical and system heterogeneity. For each round of FL learning, the edge server randomly selects a subset S of clients. r The client downloads the global model from the edge server. Then, the client trains the model using its local data. The training dataset for client i is denoted as . Where y j ∈{1,2,...,Y} is a sample x j The label Y represents the number of exception types. Therefore, the optimization objective of the local model is:

[0069]

[0070] Where, ξ i Let W represent the loss function for client i. i and F i Let i represent the personalized model and the local objective function of client i, respectively.

[0071] After the local update is complete, the model parameters are uploaded to the edge server. Finally, the edge server updates the global model by aggregating the models from different clients. Considering the differences between different clients, personalized anomaly detection models are customized for each client. Therefore, the optimization objective of the global model is:

[0072]

[0073] in, This represents the weight of client i. Represented as S r A personalized model customized for each client.

[0074] Based on the constructed system, this method integrates global and personalized learning to efficiently solve statistical and system heterogeneous problems. For example... Figure 2 As shown, the last fully connected (FC) layer of the client's local model is used as the header. The remaining part serves as the feature extractor ψ(·;W) f First, ψ(·;W f Extract training samples X from client i. i Features and map them to a new feature vector X' i For the local training step t=1, the cosine similarity cosim(·) is not calculated; instead, the feature transformation module (i.e., FeT) is used directly to transform X'. i Convert to global feature vector and personalized feature vectors then, and They are respectively input into the Global Category Embedding Module (i.e., GCE) and To enable both global and personalized learning. In GCE, The object will be guided to embed GCE(Y) close to its corresponding global category. i ;E) and frozen global category embedding To learn The global feature information is then obtained. Finally, each client undergoes personalized training, fine-tuning χ² to learn... Personalized feature information. To improve training efficiency, for local training at step t>1, cosim(·) of the cross-entropy loss and overall loss update gradient in step t-1 is calculated to determine whether to replace the loss function. In each round of training, the client shares the class embedding E, the conditional computation component T, and the feature extractor ψ, and the client obtains the frozen class embedding after receiving E.

[0075] Feature Transformation Module (FeT): Feature Extractor ψ(·; W) f ) train sample X i Mapped to X' i To coordinate global and personalized learning, X' i Convert them into global feature vectors respectively and personalized feature vectors

[0076]

[0077] Where σ is the ReLU activation function, and μ is the Hadamard product. α, β, α i and β i These are the mapping parameters, generated by FeT:

[0078]

[0079] FeT contains two learnable components T with identical structures but different parameters. α and T β T α and T β It consists of an FC layer, a ReLU activation layer, and a normalization layer. g and p i These are global and personalized conditional inputs generated by GCE. (This is achieved by using g and p...) i Input to T α and T β It can generate α, β, α i and β i .

[0080] Global Class Embedding Module (GCE): GCE contains learnable class embeddings E, used to generate g and p. i This guides the overall learning process. Before each round of training E, the client copies E to generate frozen class embeddings. The global class embedding GCE(y;E) and the frozen global class embedding are used to generate the global class embedding for each anomaly class y, respectively. First, by averaging all anomaly categories To generate global conditional input:

[0081]

[0082] Where Y represents the number of anomaly categories.

[0083] For personalized conditional input p i It contains the local Non-IID data information of client i. The proportion of anomaly category y on client i is:

[0084]

[0085] Where Φ{·} is the indicator function.

[0086] Therefore, personalized conditional input p i Defined as:

[0087]

[0088] Next, GCE will guide the global feature vector. The learning process. Specifically, two additional loss terms are introduced for model updates. Since the clients share the global class embeddings GCE(y;E), X will be guided... i G Embeddings of GCE (Y) close to its category i ;E), far from other class embeddings. Therefore, a contrastive loss is introduced, defined as:

[0089]

[0090] in, This represents the contrastive loss during the t-th step of training on client i. This represents the cosine similarity between a and b.

[0091] Furthermore, guide Near-freeze global category embedding To improve model training performance, a loss of magnitude is introduced, defined as:

[0092]

[0093] in, This represents the magnitude of the loss during the t-th step of training on client i.

[0094] Local Personalization Update: Due to Non-IID data, the client-side personalizes the training of a head. Used for learning X i PThe feature information. For anomaly detection tasks, the cross-entropy (CE) loss is defined as:

[0095]

[0096] in, δ is the CE loss of client i during the t-th local training step, where δ represents the CE loss function.

[0097] Considering the comparison, magnitude, and CE loss, the total loss of client i at step t is:

[0098]

[0099] Therefore, the local model is updated as follows:

[0100] W i,t+1 =W i,t -ρ▽ξ i,t (12)

[0101] Where ρ is the learning rate.

[0102] The above design effectively solves the statistical heterogeneity problem. However, due to system heterogeneity, clients with insufficient computing power may not be able to complete local model training in a timely manner, which severely affects aggregation efficiency. To address this issue, this method designs a loss replacement strategy. Specifically, the update gradients ∇ξ of the two loss functions are calculated. i,t and The cosine similarity is calculated using `cosim(·)`. When the value of `cosim(·)` is greater than the hyperparameter γ, it will be used in subsequent local updates. Substitution ξ i,t In this case, the update of the local model becomes:

[0103]

[0104] The designed loss replacement strategy can effectively improve the efficiency of FL training in heterogeneous edge environments while ensuring high anomaly detection accuracy.

[0105] The training process of this method is shown in Algorithm 1. For the edge server, the header χ of each client i is first initialized. i and model parameters (Line 1). At the start of each round, select a subset S of clients. r And calculate the weight n for each client. i (Line 3). Following this, belonging to S... r The client i receives W f,r E r and T rThen, ClientUpdate() is executed in parallel, returning the locally trained model parameters (line 5). Finally, the edge server performs global aggregation based on the client weights and the uploaded model parameters, and outputs the client's personalized model {W1, W2, ..., W...} after the aggregation rounds are completed. N (Lines 6-7). Each client executes ClientUpdate(·). First, initialize cs and then initialize each module according to the global model to generate g and p. i (Line 8). When the local step t = 1 or cs = 0, use the total loss ξ. i,t Update each model (lines 10-11). Next, calculate the loss function ξ. i,t and Update the cosine similarity of the gradient, cosim(·). If the value of cosim(·) is greater than γ, then cs = 1 (line 12). When the local step t > 1 and cs = 1, use... renew and (Lines 13-14). Finally, update locally. and Upload to the edge server (line 15).

[0106]

[0107] Method evaluation:

[0108] like Figure 3 As shown, this embodiment constructs a hardware testbed to evaluate the performance of FedGPA in a real-world scenario, consisting of a workstation and 10 clients of two types. The workstation is considered an edge server, equipped with two NVIDIA GeForce GTX 3090 GPUs, an Intel(R) Xeon(R) CPU Silver 4208@2.1GHz, and 32GB of RAM. Raspberry Pi 4Bs serve as one type of client, equipped with a quad-core Broadcom BCM2711 SoC@1.5GHz and 4GB of RAM. Jetson TX2s serve as another type of client, equipped with a 256-core NVIDIA Pascal(R) GPU and a processing cluster consisting of a dual-core NVIDIA Denver 2 CPU and a quad-core Arm Cortex-A57 MPCore processor, along with 8GB of RAM. The operating system for the hardware is Ubuntu 18.04. Furthermore, all devices are located on the same local area network and communicate via a 5GHz WiFi network. By building a real-world testbed, an edge environment with statistical and system heterogeneity is simulated.

[0109] To evaluate the performance of FedGPA, this embodiment uses the following three benchmark traffic datasets.

[0110] (1) ACI-IOT-2023: This dataset is a typical home IoT network simulation provided by the U.S. Army Cyber ​​Research Institute (ACI) at the IoT Research Lab (IoTRL). It contains various attack records within a week, covering recon, denial-of-service (DoS) attacks, brute force, and spoofing.

[0111] (2) CIC-IOT-2023: This dataset is a simulation of 105 attacks executed by the Canadian Cyber ​​Security Institute (CIC) in an IoT topology consisting of 33 devices. These attacks include DDoS, DoS, reconnaissance, web-based attacks, brute-force attacks, spoofing, and Mirai.

[0112] (3)USTC-TFC-2016: This dataset was provided by the University of Science and Technology of China (USTC) and contains network traffic data for different applications. Normal traffic includes Facetime, FTP and MySQL, while abnormal traffic includes Neris, Nsis and Virut.

[0113] The original ACI-IoT-2023 dataset contains 1 type of normal traffic and 4 types of abnormal traffic. The original CIC-IOT-2023 dataset contains 1 type of normal traffic and 7 types of abnormal traffic. The original USTC-TFC-2016 dataset contains 10 types of normal traffic and 10 types of abnormal traffic. First, the original traffic data is preprocessed. Specifically, the original traffic data is split into independent sessions according to a 5-tuple (i.e., source IP, destination IP, source port, destination port, and transport layer protocol). For each session, key feature information of the first N packets is extracted, including payload length and inter-arrival time. To maintain the continuity of the training samples, the inter-arrival time of the first packet is set to 0. When extracting packet features, DNS packets are discarded and padded with empty packets. This is because DNS packets mainly contain domain name to IP address mapping information, which is irrelevant to abnormal traffic detection. Next, the extracted features are arranged in chronological order to obtain training samples, and the first 8 packets of each session (i.e., N=8) are extracted for abnormal traffic detection. In the CIC-IOT-2023 dataset, distributed DoS attacks and DoS attacks were merged into one class, and three categories with insufficient samples were removed. In the USTC-TFC-2016 dataset, 10 types of normal traffic were grouped into one class, and four types of abnormal traffic with insufficient samples were removed. The datasets are described in Table 1. Finally, a Non-IID local dataset was assigned to each client, where the dataset was sampled by label q. i,c ~Dirichlet(α),α=0.3,q i,c This represents the proportion of data labeled c assigned to client i. The client's local dataset contains a training set (60%), a validation set (20%), and a test set (20%).

[0114] Table 1. Dataset Description

[0115]

[0116] The FedGPA proposed in this invention is compared with the following four state-of-the-art FL frameworks.

[0117] (1) FedAvg: As one of the most classic FL models, FedAvg aggregates the model parameters of all clients on average and provides a global model for them.

[0118] (2) FedALA: FedALA introduces an Adaptive Local Aggregation (ALA) module that can adaptively aggregate global and local models for local targets of the client.

[0119] (3) FedPer: FedPer divides the model into a base layer and a personalization layer. In each round, only the parameters of the base layer are aggregated, and the personalization layer is only updated in the local personalization task.

[0120] (4) GPFL: As a variant of FedPer, GPFL can learn both global and personalized feature information at the same time.

[0121] Based on the FL framework described above, this embodiment uses the following three classic deep learning models for anomaly detection.

[0122] (1) 1D-CNN: As a convolutional neural network used to process sequential data, 1D-CNN can automatically extract key features of a sequence and use them for classification tasks.

[0123] (2) GRU: As an improved recurrent neural network (RNN), GRU controls the flow of information by introducing a gating mechanism to capture long-term dependencies.

[0124] (3)FormerTime: As a Transformer-based method, FormerTime shortens the temporal length and efficiently extracts features for classification through multi-scale representation and self-attention mechanism.

[0125] Based on PyTorch 1.11.0, this invention implements the proposed FedGPA and other state-of-the-art methods. To ensure fairness, for all FL frameworks, the global communication rounds are set to 100, the client-side local training steps to 10, the training batch size to 64, the learning rate to 0.005, and the client participation rate to 0.8. For each anomaly detection model, the 1D-CNN consists of 2 convolutional layers and 2 fully connected layers. Each convolutional layer consists of a convolutional layer, a normalization layer, a ReLU activation layer, and a pooling layer. The kernel size is 3, the stride is 2, and the padding is 1. The stride of the pooling layer is 2. The GRU consists of 2 hidden layers and 2 fully connected layers, with the hidden layer size being 32. For FormerTime, the number of self-attention heads is 4, the number of layers is 3, the hidden size of each layer is 64, the stride is 2, the slice size is 2, and the dropout rate is 0.2. Based on the above settings, different anomaly detection models were run on different FL frameworks and the accuracy and training time of each model were tested to evaluate the performance of the FedGPA proposed in this invention.

[0126] Table 2 Comparison of detection accuracy (%) and training time (s) of different models on different FL frames

[0127]

[0128] As shown in Table 2, this embodiment compares the accuracy and efficiency of different anomaly detection models on different FL frameworks. For the ACI-IOT-2023 dataset, FedGPA achieved an accuracy of 98.67% using 1D-CNN, which is comparable to the highest-accurate GPFL, but its training time is only about 60% of GPFL's. When using GRU and FormerTime, FedAvg achieved accuracies of only 65.97% and 88.74%, respectively. In contrast, FedGPA not only achieved the highest accuracy of 99.89% and 99.90%, respectively, but also had a comparable training time to FedAvg. Although GPFL achieved similar accuracy to FedGPA, its training time was significantly longer. For FedPer and FedALA, although their training time was slightly shorter than FedGPA when using 1D-CNN or GRU, FedGPA showed a clear advantage in accuracy. Specifically, when using GRU, FedGPA achieves approximately 19.76% higher accuracy than FedPer, and when using FormerTime, FedGPA outperforms both in terms of performance metrics. For the CIC-IOT-2023 dataset, when using GRU, FedGPA's accuracy is slightly lower than GPFL, but its training time is reduced by 5.67 seconds. Compared to other methods, FedGPA improves accuracy by approximately 0.05%–19.05% when using 1D-CNN and FormerTime. For the USTC-TFC-2016 dataset, FedGPA achieves the highest accuracy across different anomaly detection models and is close to FedAvg in training time. Overall, FedGPA significantly reduces model training time while maintaining high detection accuracy. FedAvg provides a global model to all clients, thus it cannot adapt to dynamic and heterogeneous edge environments. FedALA initializes by aggregating models locally, while FedPer splits the model for personalized fine-tuning. All of these approaches provide personalized models for clients but lack global collaborative learning, thus failing to fully utilize the global feature information from different clients. Similar to FedPer, GPFL splits the model and considers feature information from other clients, but it doesn't account for device heterogeneity in complex edge environments, causing clients with lower computing power to be unable to participate in federated aggregation in a timely manner. The FedGPA proposed in this invention achieves superior performance thanks to its global-personalized collaborative learning and improves training efficiency through loss substitution.

[0129] like Figure 4 As shown, this embodiment tests the scalability of different FL frameworks with varying numbers of clients on the ACI-IOT-2023 dataset. Overall, the FedGPA proposed in this invention demonstrates superior and stable anomaly detection accuracy in scenarios with varying numbers of clients. Specifically, as... Figure 4As shown in (b), FedGPA's accuracy fluctuated by approximately 8.8% when using GRU, as the number of clients changed. In comparison, FedAvg, FedPer, FedALA, and GPFL exhibited performance fluctuations of approximately 25.33%, 16.58%, 33.07%, and 18.99%, respectively, and their accuracy was all lower than FedGPA. Figure 4 As shown in (a) and (c), FedGPA maintains stable high accuracy when using other anomaly detection models, while other FL frameworks exhibit some degree of performance fluctuation. For example, when using 1D-CNN on FedALA, a performance fluctuation of approximately 43.39% occurred. With the increase in the number of clients, the accuracy of most FL frameworks initially increases and then decreases. This is because when the number of clients is small, training data is limited, and the anomaly detection model cannot be fully trained or may overfit. With a large number of clients, statistical and system heterogeneity issues become more prominent, leading to training bias and consequently affecting anomaly detection performance.

[0130] like Figure 5 As shown, this embodiment tests the convergence of different FL frameworks on the ACI-IOT-2023 dataset from the perspective of average training loss. In the FedGPA proposed in this invention, a contrastive loss ξ is introduced. cl And magnitude loss ξ ml To ensure fairness in the comparison, these two losses are omitted, and only the cross-entropy loss is considered. The results show that FedGPA achieves faster convergence than FedAvg and exhibits lower training loss compared to other FL frameworks. Figure 5 As shown in (a), FedGPA demonstrates a significant convergence advantage over FedAvg in the first 10 rounds. After 30 rounds, FedGPA converges to the minimum loss and tends to stabilize. Figure 5 As shown in (b), FedPer's convergence is more stable compared to other FL frameworks. However, FedPer's detection accuracy is only about 80.13%, indicating overfitting. Figure 5 As shown in (c), FedALA exhibits significant performance fluctuations. Due to statistical and system heterogeneity, the training process of each FL framework shows a certain degree of fluctuation. The FedGPA proposed in this invention can adapt well to different anomaly detection models and stabilize them quickly, verifying its advantages in convergence.

[0131] Table 3. Effect of hyperparameter γ on FedGPA performance

[0132]

[0133] As shown in Table 3, this embodiment tested the impact of the hyperparameter γ on the performance of FedGPA, where 1D-CNN was used as the anomaly detection model. γ represents the threshold of the cosine similarity cosim(·). The results show that model accuracy is positively correlated with γ, while training time is negatively correlated with γ. When γ = 0.95, FedGPA's accuracy reached its peak on both the ACI-IOT-2023 and CIC-IOT-2023 datasets, at 98.67% and 97.83%, respectively, with training times of 13.77s and 5.47s, respectively. Although the training time increased slightly compared to other γ values, this is acceptable while maintaining high accuracy. For the USTC-TFC-2016 dataset, when γ = 0.98, FedGPA achieved the highest accuracy of 96.05%, only 0.01% higher than when γ = 0.95, but the training time increased by 0.83s. Therefore, the setting of γ should appropriately balance model accuracy and training efficiency according to actual needs.

[0134] This invention proposes a global-personalized collaborative edge anomaly detection method based on Functional Linear Analysis (FL). By designing a feature transformation module, a global category embedding module, and a loss replacement strategy, it can simultaneously perform global and personalized learning, solving statistical and system heterogeneity problems and improving model training efficiency. Extensive experiments using real-world testbeds and traffic datasets validate the superiority of the proposed FedGPA method. Results show that compared to state-of-the-art methods, FedGPA maintains high detection accuracy while reducing training time. Furthermore, FedGPA exhibits better scalability and convergence.

[0135] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0137] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0138] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0139] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. An edge anomaly detection method based on global-personalized collaborative federated learning, characterized in that, The method comprises the following steps: (1) After the edge server generates a global model, the global model is deployed; the client initializes a local model according to the global model; (2) The traffic characteristics are converted into global and personalized feature vectors by using a conditional calculation component on the client side; (3) The client introduces contrast and magnitude loss in the global category embedding module and guides the learning of the global feature vector according to the embedding of the sample category, and uses the cross-entropy loss to guide the learning of the personalized feature vector; (4) The client judges whether to perform loss replacement by updating the cosine similarity of the gradient through the calculation of the cross-entropy loss and the total loss, thereby accelerating the training of the model; (5) After the client completes the local training, the update of the model is uploaded to the edge server; The edge server performs global aggregation and updates the global model; the above steps are repeated until the anomaly detection model converges; For a system with N IoT devices, i.e., clients, with statistical and system heterogeneity, for each round of FL learning, the edge server randomly selects a client subset S r , whose clients download the global model from the edge server; then, the clients train the model using their local data; the training dataset of client i is denoted as where y j ∈ {1, 2,..., Y} is the label of sample x j ; Y represents the number of anomaly types; thus, the optimization objective of the local model is: where, ξ i represents the loss function of the client i; W i and F i respectively represent the personalized model and the local objective function of the client i; After the local update is completed, the model parameters are uploaded to the edge server; finally, the edge server updates the global model by aggregating the models of the clients; considering the differences of different clients, a personalized anomaly detection model is customized for the clients; therefore, the optimization goal of the global model is: wherein, represents the weight of the client i, represents the personalized model customized for S r clients.

2. The edge anomaly detection method based on global-personalized collaborative federated learning according to claim 1, characterized in that, the last fully connected (FC) layer of the local model of the client as the head χ(·; W i h ) and the rest as the feature extractor ψ(·; W f ); first, ψ(·; W f ) extracts the training samples X i features of the client i and maps them as new feature vectors X i '; for the local t-th step training, the cosine similarity cosim(·) is not calculated, but X i ' is directly converted into global feature vector and personalized feature vector Then, and are input into the global class embedding module GCE and χ(·; W i h , respectively, for global and personalized learning; In the GCE, guided to be close to its corresponding global class embedding GCE(Y i ; E) and the frozen global class embedding to learn global feature information; Finally, each client performs personalized training, fine-tuning χ to learn personalized feature information; To improve the training efficiency, for the local t-th step training, the cosim(·) of the cross-entropy loss and the overall loss update gradient in the t-1-th step is calculated to determine whether to replace the loss function; in each round of training, the client shares the class embedding E, the conditional calculation component T and the feature extractor ψ, and the client copies the frozen class embedding after receiving E 3. The edge anomaly detection method based on global-personalized collaborative federated learning according to claim 2, characterized in that, The implementation method of the feature conversion module FeT is: feature extractor ψ(·; W f ) maps the training sample X i into X i ; in order to coordinate global and personalized learning, X i are transformed into global feature vector and personalized feature vector where σ is the ReLU activation function, and is the Hadamard product; a, b, a i and b i are mapping parameters, which are generated by the FeT: where FeT contains two learnable components T α and T β ; T α and T β are composed of FC layers, ReLU activation layers and normalization layers; g and p i are global and personalized condition inputs generated by GCE; by inputting g and p i to T α and T β , α, β, α i and β i are generated.

4. The edge anomaly detection method based on global-personalized collaborative federated learning according to claim 3, characterized in that, The implementation method of the global category embedding module GCE is: GCE includes learnable class embeddings E for generating g and p i to guide global learning; before each round of training E, the client copies E to generate frozen class embeddings global class embeddings GCE(y; E) and frozen global class embeddings First, global conditional input is generated by averaging all abnormal class ​ Wherein Y represents the number of abnormal categories; For personalized condition input p i which contains the local Non-IID data information of the client i; the proportion of the exception category y on the client i is: Wherein Φ{·} is an indicator function; Thus, the personalized condition input p i is defined as: Next, GCE will guide the learning process of global feature vectors ; specifically, two extra loss terms are introduced for model update; since clients share global class embeddings GCE(y; E), they will be guided to be close to their class embeddings GCE(Y i ; E) and far away from other class embeddings; therefore, a contrastive loss is introduced, which is defined as: wherein, represents the contrastive loss in the t-th step of training locally for the client i; represents the cosine similarity between a and b; Guiding Approaching frozen global class embeddings To boost the model training performance; introduce a magnitude loss, which is defined as: wherein, represents the magnitude loss in the t-th step of training locally at client i.

5. The edge anomaly detection method based on global-personalized collaborative federated learning according to claim 4, characterized in that, The implementation method of the local update is: Due to Non-IID data, the client personalizes a head χ(·; W i h ) for learning the feature information; for the anomaly detection task, the cross-entropy (CE) loss is defined as: wherein, is the CE loss of the client i at the t-th step of training, and δ denotes the CE loss function. Considering the contrast, magnitude and CE loss, the total loss of the client i at the t-th step is: Therefore, the update of the local model is: Wherein ρ is a learning rate; updating gradients of the two loss functions and the cosine similarity cosim(·) of ; when the value of cosim(·) is greater than a hyperparameter γ, use i,t instead of ξ i,t in the next local update; in this case, the update of the local model translates to:

Images