Deep learning model training method in mobile edge computing and corresponding system

Abstract

A method for training a deep learning model in mobile edge computing and a corresponding system are disclosed. The method comprises: obtaining a trained initial model, modifying the initial model to obtain a current task model; distributing the current task model and parameters of the initial model to a plurality of clients for initial training by the plurality of clients respectively; selecting a client combination with the highest prediction accuracy based on initial training results of the plurality of clients; and obtaining formal training results of each client in the client combination and generating parameters of the current task model. The present application realizes model training under the condition that the data of the client itself can only be used for local training through knowledge transfer of the existing model and the summary of local training of the client, and further improves the training efficiency and eliminates unreliable clients by screening the clients participating in the formal model training.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to mobile edge computing, and in particular to a deep learning model training method in mobile edge computing and a mobile edge computing system using the method. BACKGROUND

[0002] In order to achieve the performance of high bandwidth and low latency of communication, the existing cloud core network architecture cannot meet the requirements, and therefore the concept of MEC (Mobile Edge Computing) is proposed. Figure 1 A network architecture covering MEC is shown. As shown in the figure, MEC technology combines IT service environment and cloud computing technology at the network edge, improves the computing and storage capacity of the edge network, reduces the network operation and service delivery delay, and improves the user service quality experience, so it will be widely used in communication networks.

[0003] Deep learning (DL) has been used in many application scenarios of MEC due to its strong learning ability and wide range of applications. However, in many application scenarios, privacy protection makes it impossible to collect private data from mobile devices to edge servers for centralized model training, which hinders the wide application of deep learning in the field of mobile edge computing. Although federated learning (FL) has been proposed for coordinating model training across mobile devices, deep learning model training based on federated learning still has problems such as low training efficiency and poor training results.

[0004] Therefore, an improved scheme for training deep learning models in mobile edge computing is needed. SUMMARY

[0005] One of the technical problems to be solved by the present disclosure is to provide a deep learning model training method in mobile edge computing, which realizes model training under the condition that only local training can be used for client-side data by migrating the knowledge of existing models and summarizing the local training of clients, and further improves the training efficiency and eliminates unreliable and low-quality clients by screening the clients participating in formal model training.

[0006] According to a first aspect of the present disclosure, a deep learning model training method in mobile edge computing is provided, comprising: obtaining a trained initial model, modifying the initial model to obtain a current task model; downlinking the current task model and the parameters of the initial model to a plurality of clients for initial training by the plurality of clients respectively; selecting the combination of clients with the highest prediction accuracy based on the initial training results of the plurality of clients; and obtaining the formal training results of each client in the client combination and generating the parameters of the current task model.

[0007] Optionally, modifying the initial model to obtain the current task model includes: modifying the fully connected layer of the initial model based on the classification requirements of the current task to obtain the current task model containing the new fully connected layer, and distributing the current task model and the parameters of the initial model to multiple clients includes: providing the current task model and the initial parameters other than the modified layer to the multiple clients, wherein the multiple clients perform training to update the parameters of the new fully connected layer while maintaining the initial parameters of the unmodified layer.

[0008] Optionally, based on the initial training results of the multiple clients, selecting the client combination with the highest prediction accuracy includes: using prediction accuracy as a reward and whether a certain client is included as an action, and self-learning to select the client combination from the multiple clients with no more than a threshold number.

[0009] Optionally, selecting the client combination with the highest prediction accuracy based on the initial training results of the multiple clients includes: setting a threshold number of client combinations to be selected; and setting the threshold number as the learning condition for the self-learning.

[0010] Optionally, the method further includes: determining unreliable clients in certain client combinations based on the degradation of prediction accuracy under certain client combinations.

[0011] Optionally, the method further includes: distributing a trained current task model to other clients among the plurality of clients, excluding the unreliable client, for each of the plurality of clients to make a prediction for the current task.

[0012] Optionally, if the plurality of clients include new clients and / or clients that have collected new data, the client combination is updated and selected based on the current task model parameters, and the current task model parameters are updated and trained.

[0013] According to a second aspect of this disclosure, a mobile edge computing system is provided, comprising: an edge server for performing the method of the first aspect, and a plurality of clients, wherein the plurality of clients perform initial training using their respective initial training datasets, the edge server selects a client portfolio using a validation dataset that does not involve client data, and each client in the client portfolio performs formal training using its own training dataset, wherein the training dataset contains more data than the initial training dataset and the validation dataset.

[0014] Optionally, the edge server identifies unreliable clients by selecting the client combination and issues a warning about the unreliability of the client.

[0015] According to a third aspect of this disclosure, a method for training an intrusion detection model in mobile edge computing is provided, comprising: obtaining an initial intrusion detection model and its parameters trained from a source domain; replacing the fully connected layer of the initial model with a new fully connected layer that conforms to the current task classification to obtain a current task model; distributing the initial parameters of the current task model to multiple clients for each client to perform initial training; obtaining the initial training results of the multiple clients; selecting a client combination with the highest prediction accuracy and not exceeding a threshold number based on reinforcement learning; and generating current task model parameters based on the training results of each client in the client combination for the new fully connected layer.

[0016] According to a fourth aspect of this disclosure, a computing device is provided, comprising: a processor; and a memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method described in the first or third aspect above.

[0017] According to a fifth aspect of this disclosure, a non-transitory machine-readable storage medium is provided, on which executable code is stored, which, when executed by a processor of an electronic device, causes the processor to perform the method described in the first or third aspect above.

[0018] Therefore, this invention utilizes federated learning to protect privacy by training the model locally on the client side and leverages knowledge transfer from existing models to improve training efficiency. Since the edge server cannot know the client's state, reinforcement learning based on the client's initial training results can be used to filter out low-quality and unreliable clients, thereby improving model training performance while saving time and computational costs. Attached Figure Description

[0019] The above and other objects, features and advantages of this disclosure will become more apparent from the more detailed description of exemplary embodiments thereof taken in conjunction with the accompanying drawings, wherein like reference numerals generally denote like parts.

[0020] Figure 1 The network architecture covering MEC is shown.

[0021] Figure 2 A schematic flowchart of a deep learning model training method in mobile edge computing according to an embodiment of the present invention is shown.

[0022] Figure 3 An example is shown where the current model is trained based on an initial trained model.

[0023] Figure 4An illustration is shown of a mobile edge computing system performing the training method of the present invention. Detailed Implementation

[0024] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0025] Mobile edge computing (MEC) technology pushes the computing power of cloud data centers down to the network edge. On the one hand, MEC infrastructure is typically deployed at the network edge, such as wireless base stations, making it more vulnerable to insecure environments. On the other hand, MEC employs technologies such as open application programming interfaces (APIs) and open network function virtualization (NFV), and this openness can easily expose MEC to external attackers. However, compared to cloud centers, edge facilities have limited resources and capabilities, making it difficult to provide the same level of security as cloud data centers. Therefore, security is a critical issue that MEC services need to address. For example, MEC systems need to promptly identify network intrusions. A network intrusion here can refer to a critical event that could lead to the leakage of valuable network resources and jeopardize network security, such as a technical failure. Therefore, intrusion detection (ID) is needed to identify data points, events, and / or observations that deviate from normal behavior and to isolate intrusions before they cause serious damage. Deep learning (DL) has been used to train intrusion detection models in mobile edge computing and can provide high-performance detection models that outperform previous intrusion detection methods, thereby improving the security of mobile edge computing.

[0026] It should be understood that deep learning models need to be trained before they can perform correct reasoning. For example, before deploying a deep learning model that uses a convolutional neural network (CNN) for reasoning (e.g., determining whether a network event input to the model is an intrusion event), the CNN must first be trained.

[0027] The training process for a CNN does not change the structure of the neural network model, but it does update the parameters of each layer of the neural network model. That is, by importing a large amount of training data, the parameters of each layer of the neural network model, such as weights and biases, are determined.

[0028] Training the model means learning (determining) ideal values ​​for all weights and biases using labeled samples. These determined weights and biases then enable high-accuracy inference on the input features during the neural network deployment phase, such as correctly classifying network events, for example, whether they are intrusion events or normal events.

[0029] In supervised learning, machine learning algorithms learn parameters by examining multiple samples and trying to find a model that minimizes loss; this process is called empirical risk minimization.

[0030] Loss is a penalty for poor predictions. That is, loss can be a numerical value representing how accurately the model predicts for a single sample. If the model's predictions are perfectly accurate, the loss is zero; otherwise, the loss will be large. The goal of training a model is to find a set of weights and biases with a "small" average loss from all samples.

[0031] During the training of a neural network, a loss function needs to be defined to quantify whether the current weights and biases can allow the network input to fit all network inputs. Therefore, the goal of training the network can be transformed into minimizing the loss function of the weights and biases. Typically, gradient descent (and backpropagation in multi-layer neural network training) is used to achieve this minimization.

[0032] The backpropagation algorithm involves repeated iterative processes of forward and backward propagation. Forward propagation involves neurons in different layers being connected through weight matrices, allowing stimuli (feature values) to be continuously transmitted from one layer to the next via the activation function of each layer. In backpropagation, the error of the current layer is derived from the error of the next layer. Through these iterative processes of forward and backward propagation, the weights and biases are continuously adjusted until the loss function gradually approaches its minimum, thus completing the training of the neural network.

[0033] For privacy reasons, edge servers cannot collect personal data from clients for centralized model training. Therefore, to protect user privacy during deep learning model training, federated learning (FL), which coordinates model training across mobile devices, can be used. In federated learning, clients upload their locally trained models to the server for aggregation (e.g., uploading model parameters or gradients), and the server does not collect any initial data from users. It should be understood that although the principles of this invention will primarily be explained based on intrusion detection models, the federated learning-based deep model training described above can be used not only for intrusion detection but also for other MEC applications requiring privacy protection, such as healthcare, autonomous driving, and security image analysis. In other words, the training scheme of this invention can be applied to various MEC application scenarios.

[0034] Federated learning leads to frequent, large-scale, and concurrent model transfers, consuming significant communication resources. Furthermore, edge servers cannot monitor the training quality of each client. Therefore, the overhead and efficiency of existing federated learning-based deep learning model training urgently need improvement. To address this, this invention provides a deep learning model training method for mobile edge computing. This method achieves model training when client-side data can only be used for local training by transferring knowledge from existing models and aggregating locally trained data. Moreover, by filtering clients used for formal model training, it further improves training efficiency and eliminates unreliable and low-quality clients.

[0035] Figure 2 A schematic flowchart illustrating a deep learning model training method in mobile edge computing according to an embodiment of the present invention is shown. This method can be executed by an edge server in mobile edge computing.

[0036] In step S210, a trained initial model is obtained, and the initial model is modified to obtain the current task model.

[0037] As we know from the background of deep learning, obtaining a trained initial model means obtaining the model structure of a neural network model and the parameters (e.g., weights and biases) of that neural network model obtained by training it using labeled training data. This "initial" model is relative to the "current" task model; that is, the obtained model can be modified to obtain the "current" model. This initial model can be a model used to perform the same task as the current task model, for example, both being models for intrusion detection. However, the two models may have specific task differences, such as a different number of result classifications.

[0038] Since the parameters of the initial model contain knowledge of extracting intrusion event features from network events, the method of directly conducting subsequent training based on the trained model in this invention can significantly improve training efficiency compared to selecting a model and training parameters from scratch, thereby avoiding unnecessary waste of computing power and communication resources.

[0039] Here, modifying the initial model can be understood as modifying the model itself (rather than its parameters). This results in a current task model with a similar structure and classification task to the initial model, but with slight differences. In one embodiment (for example, combined with the following...), Figure 3 In the described embodiments, the fully connected layer of the initial model can be modified based on the classification requirements of the current task to obtain the current task model containing the new fully connected layer.

[0040] Subsequently, in step S220, the parameters of the current task model and the initial model are sent to multiple clients for each client to perform initial training.

[0041] Since the modification to the initial model in step S210 involves the model itself, such as modifying several layers, when training the current task model based on the modified model, the initial model parameters corresponding to the unmodified layers can be directly used as the parameters for subsequent training. Alternatively, random parameters can be assigned to the modified layers in the initial model, and these random parameters can also be used as the initial parameters for subsequent training of the current task model.

[0042] The method described above for modifying and training a model using an already trained model can be considered a form of transfer learning. Transfer learning (TL) is an ideal technique for accelerating the learning process and improving the communication efficiency of federated learning utilized in this invention. The key to transfer learning is transferring knowledge from an existing domain to a new domain, aiming to accelerate model training, reduce computational costs, and improve performance. As detailed below, this invention achieves privacy-preserving and communication-efficient model training through a combination of FL and TL, namely joint transfer learning (FTL). Specifically, a well-trained model is selected and transferred from the source domain to the target domain as the initial global model for federated learning. Due to knowledge transfer, the federated learning model training exhibits higher initial accuracy, learning speed, and asymptotes. Therefore, the application of transfer learning in federated learning reduces the number of communication rounds required for convergence, thereby improving communication efficiency.

[0043] In neural networks used for deep learning, features progress from general to specific (from low to high layers). Therefore, the transferability of features and classifiers decreases as cross-domain differences increase. This is why this invention requires selecting an initial model with the same (or at least similar) task type. During model transfer, the model itself can be adjusted, for example, modifying the last few fully connected (FC) layers to suit the classification needs of the current task. After model transfer, the model can be fine-tuned; for example, all layers in the current model except for the last few fully connected layers can be frozen, and only the last few FC layers can be trained.

[0044] Figure 3 An example is shown where the current model is trained based on an initial trained model. Here, the initial model can be considered as... Figure 2 The initial model mentioned above is replaced by the current model, which is the "current task model" obtained based on the initial model. Figure 3In the example, the initial model and the current model do not differ in the number of layers or layer distribution structure. Each is a convolutional neural network model consisting of three one-dimensional (1D) convolutional layers (Conv1, Conv2, Conv3), two max-pooling layers (Pool1, Pool2), one batch normalization layer (Norm), one dropout layer, and two fully connected (FC1, FC2) layers. The only difference between the initial and current models is that the current model adjusts the last fully connected layer (i.e., FC2) of the initial model. For example, since the initial model is a five-class classification model for intrusion detection, while the current model is an eight-class classification model for intrusion detection, the FC2 layer needs to be adjusted. In other words, the FC layers of the current model need to be aligned with the number of task classifications.

[0045] Since the order and internal structure of the first three convolutional layers (Conv1, Conv2, Conv3), two max pooling layers (Pool1, Pool2), one batch normalization layer (Norm), one dropout layer, and one fully connected layer (FC1) remain completely unchanged compared to the initial model, the parameters of these layers in the initial model can be directly passed to the current model.

[0046] In one embodiment, training on the client can be performed as shown in the figure, by freezing the initial parameters (i.e., the parameters of all layers except the FC2 layer remain unchanged), and training only the parameters of the FC2 layer. In other words, the current model at this point makes the most of the knowledge contained in the initial model parameters, replacing the last layer only because the classification is different.

[0047] In other embodiments, training performed on the client may, for example, freeze fewer layers, such as freezing Conv1 to the Dropout layer, and training the initial parameters of FC1 and the parameters of the new FC2 layer. In still other embodiments, it may even be possible to train the initial parameters of Conv1 to FC1 and the parameters of the new FC2 layer without freezing any layers.

[0048] It should be understood that, Figure 3 This is just one example of TL. In different examples, the network structure of the neural network can be changed more significantly (e.g., the structure of more layers can be modified), or more initial layers can be fine-tuned, etc. This invention does not limit this.

[0049] Subsequently, in step S230, based on the initial training results of the multiple clients, the client combination with the highest prediction accuracy is selected. In step S240, the formal training results of each client in the client combination are obtained, and the current task model parameters are generated.

[0050] As can be seen from the above, in this invention, not all available clients are directly selected for formal training of the current model. Instead, available clients undergo initial training (e.g., a few rounds of training) to identify high-quality clients, and then federated learning is performed only on these high-quality clients. This is because in mobile edge computing, edge servers cannot access the client's training data and therefore cannot fully control the client's behavior. Consequently, clients may deviate from normal behavior during federated learning. These abnormal clients can be called "unreliable clients." Unreliable behavior may be intentional (e.g., proactive attacks by malicious attackers) or unintentional (e.g., clients with hardware or software defects causing anomalies in data collection or model training). Furthermore, some clients may contain little or duplicate data, and their update quality may be low. These clients with poor training data quality can be called "low-quality clients." These unreliable or low-quality clients will reduce the model's training efficiency and may even cause the model to fail to converge. Therefore, after knowledge transfer through transfer learning, it is necessary to exclude unreliable or low-quality clients and select more useful clients to participate in the federated learning model training. Therefore, by screening clients, the cost of model training can be reduced, the accuracy of model prediction can be improved, and the performance and security of mobile edge computing systems can be guaranteed.

[0051] Specifically, in step S230, a budget constraint can be imposed on the number of clients participating in federated learning, for example, denoted by K. To this end, a selection scheme can be designed to select K clients from the M clients included in the mobile edge computing system to participate in model training (where K ≤ M), with the goal of maximizing the model's prediction accuracy with no more than K clients used for federated learning. For this purpose, a self-learning (i.e., unsupervised) approach can be used for client selection. Specifically, prediction accuracy can be used as a reward, and whether a certain client is included can be used as an action to self-learn and select client combinations from the multiple clients up to a threshold number. Further, the threshold number K can be set as the learning condition for self-learning.

[0052] Reinforcement learning (RL) is a learning process for an agent that takes actions based on its environment to maximize its reward. Specifically, suppose there is a set of states and a set of actions. At each time step t, the agent observes state s. t And execute action a t Then, the state changes from s t Transfer to s t+1 The agent received a reward. tThe state transitions follow a Markov decision process. The goal of reinforcement learning is to learn the mapping from states and actions to long-term reward values, called the optimal value function. To this end, in step S230, reinforcement learning, especially the typical reinforcement learning algorithm Q-learning (i.e., Q-learning, or QL), can be used to select a subset of clients in the target domain for training.

[0053] Therefore, in step S220, all M clients in the system can perform initial training using their respective data. Since the purpose of this initial training is to enable the edge server to select clients based on the parameter results obtained from the initial training (e.g., the parameters of the FC2 layer trained by each client), the amount of training data for this initial training should be small. For example, each client can be instructed to use a small proportion (e.g., 10%) of its total available data as the initial training dataset for training. Simultaneously, the number of training epochs should also be small, for example, 5 iterations. It should be understood that the data used by the clients for model training is their own acquired data, such as data acquired during actual operation. This self-acquired data is used only for local model training (e.g., initial training and formal training as described below) for privacy protection reasons, and the training results uploaded to the server are merely the model parameters, not the privacy data itself used to train the model.

[0054] Subsequently, in step S230, the edge server can test the initial training results uploaded by all M clients (e.g., the parameters of the FC2 layer obtained by each client), using its own validation dataset (the data in the validation dataset does not come from the clients, so there is no privacy issue), and use reinforcement learning to obtain no more than K combinations of clients whose test results are optimal (i.e., the FC2 layer parameters finally generated based on the parameters submitted by these client combinations optimize the model's classification prediction results). In step S240, these clients can be used to perform federated learning and obtain the final training model. For example, these clients can be used to perform multiple rounds of training on all available data, and the FC2 parameters for the current task model can be obtained based on the FC2 parameters uploaded by each of these clients.

[0055] As mentioned earlier, reinforcement learning can be used to select a preferred client combination of no more than K clients from M clients, and the unselected clients can at least be considered as suboptimal clients. Since the poor prediction performance of the individual client training models can be caused by various reasons such as insufficient data, system failure, or malicious obfuscation, the attributes of the unselected clients can be determined based on the changes in prediction accuracy under various client combinations. For example, clients that cause a decrease (or significant degradation) in prediction accuracy can be considered unreliable clients (or suspected unreliable clients), and those clients that cause slow improvement in prediction accuracy are low-quality clients.

[0056] Therefore, based on the model training results, the system can provide alerts for the aforementioned suspected unreliable clients, and then further track and determine the attributes of the suspected unreliable clients.

[0057] The deep learning model trained using the combination of FTL and RL as described above, such as an intrusion detection model, can be distributed to M clients in the mobile edge computing system, enabling these clients to use the intrusion detection model to detect intrusions in network events. In some embodiments, when certain clients are determined to be unreliable, the intrusion detection model trained by this invention may not be sent to these clients, thereby preventing the training results from being obtained by malicious clients.

[0058] In carrying out the present invention, as follows Figure 2 After the model shown is trained, it can be updated when some clients collect more data or new clients join the system. Specifically, the client combination can be updated based on the current task model parameters, or the current task model parameters can be updated during training.

[0059] This invention can also be implemented as a mobile edge computing system, which may include systems capable of performing... Figure 2 The method involves an edge server and multiple client clients (e.g., M clients). Figure 4 An illustration is shown of a mobile edge computing system performing the training method of the present invention.

[0060] As shown in the figure, firstly, in step 1, a well-trained model is selected from the source domain. This model is then introduced into the target domain through transfer learning. The target domain can be implemented as a mobile edge computing system, including an edge server and M clients. Subsequently, in step 2, the edge server can distribute the modified current model to all M clients for initial training (the clients use their respective initial training datasets for this initial training). Reinforcement learning is then performed based on the initial training results and a threshold K (the edge server uses a validation dataset that does not involve client data for client combination selection). This process selects the clients that will ultimately undergo federated learning in step 3 (e.g., ...). Figure 4 The edge server identifies unreliable clients (those formally trained using their respective training datasets) and can also pinpoint unreliable clients based on a decrease in the predetermined accuracy of the model obtained from various client combinations during initial training. These unreliable clients can not only be excluded from federated learning but can also be investigated more thoroughly to determine their unreliability attributes. In some embodiments, the edge server identifies unreliable clients by selecting said client combinations and issuing warnings. For example, this could be done by a device repair technician inspecting a specific client.

[0061] Application Example

[0062] The following will combine Figure 3 and Figure 4 A detailed description of an application example of training an intrusion detection model according to the present invention is provided to facilitate a further understanding of the principles of the present invention.

[0063] First, consider a joint transfer learning (FTL) framework for training an intrusion detection (ID) model based on selected clients in mobile edge computing. This framework consists of an edge server and M clients. These M clients can be represented as C = {C1, C2, ..., C...}. M Network intrusion refers to critical events that could lead to the leakage of valuable network resources and jeopardize network security, such as technical malfunctions. The goal of intrusion detection is to identify data points, events, and / or observations that deviate from normal behavior. This requires training a high-precision ID model to detect intrusions in a timely manner. Training data is collected locally on the client side, denoted as D = {D1, D2, ..., D...}. M The training data has a range of characteristics (e.g., connection duration, protocol type, and destination network service) and different labels (normal and various attack types). The first step is as follows: Figure 4 The example shows a pre-trained model selected in the source domain and transmitted to an edge server in the target domain for FL model training. Therefore, the first problem is to design an efficient FTL framework for the system under consideration.

[0064] To avoid unreliable or low-quality ID model updates, it is necessary to select useful clients to participate in FL (Flexible Learning) model training, thereby improving learning performance and saving system costs. This raises the second problem of client selection. First, a budget constraint, denoted by K, can be set for the number of clients participating in FL. Therefore, the goal of the second problem is to design a RL-based scheme to select K clients from M clients to participate in model training, with the objective of maximizing accuracy (K≤M). Thus, the second problem can be formulated as follows:

[0065]

[0066] st,|C sel |≤K

[0067] Where C sel It is the set of selected clients, |C sel | is C sel The number of clients, A is the accuracy.

[0068] In deep learning networks, features progress from general to specific (from low to high layers), thus the transferability of features and classifiers decreases as cross-domain variability increases. Therefore, fine-tuning methods are widely used and proven effective; after model transfer, all layers except the last few fully connected (FC) layers are frozen, and only the last few FC layers are trained.

[0069] Based on the principles of FL and TL introduced above, an ID framework based on FTL can be proposed, such as... Figure 3 Figure 3 As shown. Specifically, the initial model has been well trained for the ID task in the source domain, and then the model is transferred to the target domain as the initial global model for FL training, i.e., the "current task model" mentioned above.

[0070] The training process of FTL is shown in Algorithm 1 below.

[0071] Algorithm 1 for ID model training in MEC:

[0072] 1. Select a trained intrusion detection deep learning model in the source domain, and then pass the trained parameters to the edge server in the target domain;

[0073] 2: In the target domain, replace the last FC layer with a new FC layer, aligned with the task's classification number, thus forming the initial global model;

[0074] 3: The server notifies all clients to participate;

[0075] 4: The server selects clients to participate in FL training (Algorithm 1 does not involve client selection, so all clients are selected for model training here);

[0076] 5: Repeat

[0077] 6: The selected client downloads the latest global model and its parameters from the server;

[0078] 7: The customer trains their model locally. Specifically, all layers except the last fully connected (FC) layer are frozen, and only the last FC layer is trained.

[0079] 8: The client uploads the model update to the server;

[0080] 9: The server aggregate model is updated, generating a new global model;

[0081] 10: Until convergence or after a certain number of rounds;

[0082] 11: Obtain the final global model.

[0083] Based on Algorithm 1 above, the training efficiency and accuracy of the model in this invention can be further improved by incorporating the step of selecting the client through reinforcement learning.

[0084] RL is a learning process for an agent that takes actions based on its environment to maximize its reward. Specifically, assume there is a set of states and a set of actions. At each time step t, the agent observes state s. t And execute action a t Then, the state changes from s t Transfer to s t+1 The agent received a reward. t State transitions follow Markov decision processes. The goal of RL is to learn the mapping from states and actions to long-term reward values, called the optimal value function. Here, the typical RL algorithm Q-learning (QL) can be used to select a subset of clients in the target domain for training.

[0085] Specifically, based on the principles of QL, a client selection algorithm is designed to select a subset of clients to achieve the highest accuracy within the budget. Specifically, the state, action, and reward function of QL can be defined as follows:

[0086] States: Each client has two sub-states, "selected" and "unselected," represented by "1" and "0" respectively. These sub-states are used to distinguish "states" in QL. Therefore, considering all clients, the Q-function table has a total of 2M states. The agent, together with the edge server, selects all clients with state "1" to participate in the FTL training for each QL episode.

[0087] Action: If each state transition is set as an action, the action space is 2. M To shorten the action space, we can design an action set with space M+1, namely {flip 1, flip 2, flip 3, ..., flip M, stay}. Here, "flip n" represents changing C. n The substate, then transition to change C. n The state following a substate, "stay" means to remain in the current state.

[0088] Rewards: The rewards observed at the end of each level can be set as follows:

[0089]

[0090] Where A t It is the accuracy of FTL verification based on the participating clients after level t, and γ>1 is a constant. Here, when When B = 0, When B = 1, where This is the set of clients selected by episodeodet. (Settings) The incentives for B are as follows: the first is to incentivize the agent to achieve the highest accuracy rate, and the second is to ensure that the number of selected clients is within budget. Specifically, an exponential function with a large base γ can be used to amplify the reward increment caused by accuracy (e.g., setting γ = 100). On the other hand, due to Since it is less than 1, B=1 can be set when the budget is met.

[0091] Based on the state, action, and reward functions designed for QL, a RL-based client selection algorithm is proposed for the FTL framework, as shown in Algorithm 2.

[0092] MEC uses RL-based client-selected algorithm 2 for ID model training:

[0093] 1: Steps 1-3 in Algorithm 1;

[0094] 2: Initialize the Q function table, that is, set all elements in Q(s,a) to 0, with the initial state s0 = [1,1...,1];

[0095] 3: Repeat

[0096] 4: Select and execute action a based on the ε-greedy policy t Following the probability of 1-ε, a t =Argmax a Q(s t The strategy is to select a random action with probability ε and update the state to s.t+1 ;

[0097] 5: Check s t+1 Has the verification accuracy been derived? If not, then based on the selected s t+1 The corresponding client performs FL training (following steps 4-10 of Algorithm 1) to obtain the validation accuracy;

[0098] 6: Measure reward r according to (8) t+1 ;

[0099] 7: Update the Q function table according to (7);

[0100] 8: Until the average reward converges or a specific level is reached;

[0101] 9: The final convergence state represents the final selected participating clients;

[0102] 10: Perform FL training based on the selected client to obtain the most optimized global model.

[0103] Algorithm 2 allows us to obtain the most optimized subset of clients within the budget, thus achieving the most optimized global model.

[0104] Alternatively, an approximate trained model can be selected from the source domain as the initial model. The training of this model in the source domain can be done under the FL protocol or not.

[0105] Based on the above application example, the present invention can be particularly implemented as a method for training an intrusion detection model in mobile edge computing, comprising: obtaining an initial intrusion detection model and its parameters trained from a source domain; replacing the fully connected layer of the initial model with a new fully connected layer that conforms to the current task classification to obtain a current task model; distributing the current task model and the initial parameters excluding the replaced layer to multiple clients for each client to perform initial training; obtaining the initial training results of the multiple clients; selecting a client combination with the highest prediction accuracy and not exceeding a threshold number based on reinforcement learning; and generating current task model parameters based on the training results of each client in the client combination for the new fully connected layer.

[0106] The training method for a deep learning model in mobile edge computing according to the present invention has been described in detail above with reference to the accompanying drawings. In the mobile edge computing of the present invention, joint transfer learning is performed by selecting clients to participate in model training. This scheme is particularly suitable for training intrusion detection models. Specifically, the present invention can utilize joint transfer learning to train an intrusion detection model to achieve efficient and secure mobile edge computing, and designs a reinforcement learning-based client selection scheme for joint transfer learning to select useful clients within a budget, thereby achieving the highest accuracy. The proposed scheme can successfully exclude malicious clients from model training. Thus, the training scheme of the present invention achieves mobile edge computing model training with higher accuracy, higher communication efficiency, and lower system cost.

[0107] Furthermore, the method according to the present invention can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing the steps defined in the above-described method of the present invention.

[0108] Alternatively, the present invention can also be implemented as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) storing executable code (or computer program, or computer instruction code) thereon, which, when executed by a processor of an electronic device (or computing device, server, etc.), causes the processor to perform the various steps of the method described above according to the present invention.

[0109] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both.

[0110] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems and methods according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0111] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for training a deep learning model in mobile edge computing, comprising: obtaining an initial trained model, modifying the initial model based on classification requirements of a current task to obtain a current task model; downloading the current task model and parameters of the initial model to a plurality of clients for initial training by the plurality of clients respectively; selecting a client combination with the highest prediction accuracy based on initial training results of the plurality of clients; and obtaining formal training results of each client in the client combination and generating parameters of the current task model. The selecting of the client combination with the highest prediction accuracy based on the initial training results of the plurality of clients comprises: using the prediction accuracy as a reward and whether to include a certain client as an action to select the client combination from the plurality of clients by self-learning, wherein the number of the client combination is not more than a threshold value. Further comprising: determining unreliable clients existing in certain client combinations based on degradation of the prediction accuracy in the certain client combinations. The initial training and the formal training are two independent training stages, and a proportion of an initial training data set in the initial training in all available data is less than a proportion of a formal training data set in the formal training, wherein the formal training data set in the formal training is all available data, and a training round of the initial training is less than a training round of the formal training. The modifying of the initial model to obtain the current task model comprises:

2. The method of claim 1, wherein, modifying a fully connected layer of the initial model based on classification requirements of a current task to obtain the current task model containing a new fully connected layer, and The downloading of the current task model and the parameters of the initial model to the plurality of clients comprises: providing the current task model and initial parameters except for modified layers to the plurality of clients, wherein the plurality of clients perform training of updating parameters of the new fully connected layer while keeping initial parameters of the unmodified layers unchanged. 3.The method of claim 1, wherein the selecting of the client combination with the highest prediction accuracy based on the initial training results of the plurality of clients comprises: setting a threshold number of client combinations to be selected; and setting the threshold number as a learning condition of the self-learning. 4.The method of claim 1, further comprising: downloading a trained current task model to clients other than the unreliable clients in the plurality of clients for prediction of the current task by the plurality of clients respectively. In a case where the plurality of clients include new clients and / or clients that collect new data, the updating selection of the client combination and the updating training of the current task model parameters are performed based on the current task model parameters. 6.A mobile edge computing system, comprising:

5. The method of claim 1, wherein, an edge server configured to perform the method of any one of claims 1-5, and the plurality of clients. ​ ​ The plurality of clients perform the initial training using respective initial training data sets, the edge server performs client combination selection using a validation data set that does not involve client data, and each client in the client combination performs formal training using a respective training data set.

7. The system of claim 6, wherein, The edge server identifies unreliable clients by selecting the client combination and performs a warning of client unreliability.

8. A mobile edge computing intrusion detection model training method, comprising: obtaining a trained initial intrusion detection model from a source domain and parameters thereof, replacing a fully connected layer of the initial intrusion detection model with a new fully connected layer conforming to a current task classification to obtain a current task model; downloading the current task model and the parameters of the initial intrusion detection model to a plurality of clients for initial training by the plurality of clients using respective initial training data obtained by the clients; obtaining initial training results of the plurality of clients, and selecting a client combination with the highest prediction accuracy and not exceeding a threshold number based on reinforcement learning; and obtaining a training result for the new fully connected layer based on formal training by each client in the client combination using training data obtained by the client, and generating current task model parameters; the obtaining of the initial training results of the plurality of clients and the selection of the client combination with the highest prediction accuracy and not exceeding a threshold number based on reinforcement learning comprises: determining unreliable clients existing in certain client combinations based on degradation of the prediction accuracy of the combinations. The method further comprises: The initial training and the formal training are two independent training stages, and the proportion of initial training data in the initial training in all available data is less than the proportion of formal training data in the formal training. The formal training data in the formal training is all available data, and the training rounds of the initial training are less than the training rounds of the formal training.

9. A computing device, comprising: a processor; and a memory having executable code stored thereon that, when executed by the processor, causes the processor to perform the method of any one of claims 1-5 or 8.

10. A non-transitory machine-readable storage medium having executable code stored thereon that, when executed by a processor of an electronic device, causes the processor to perform the method of any one of claims 1-5 or 8. ​

Images