A malicious traffic detection method and apparatus

Abstract

A method of detecting malicious traffic in a telecommunication network is disclosed. The method includes training a machine learning model to identify malicious traffic using a training dataset to obtain a trained model. The method also includes deploying the trained model as a deployed model to detect malicious traffic in an incoming traffic stream. The method further includes incrementally training the deployed model to identify new malicious traffic using new data by applying a learning without forgetting method. Data previously used for training the deployed model is not used in the incremental training and any layer of the deployed model may be changed. The method further includes deploying the incrementally trained model to detect malicious traffic in the incoming traffic stream. A server which implements and incrementally trains the deployed model is also disclosed.

Description

A MALICIOUS TRAFFIC DETECTION METHOD AND APPARATUSTECHNICAL FIELD

[0001] This invention relates to a malicious traffic detection method and apparatus. More particularly, this invention relates to a method and apparatus for detecting malicious traffic in a 5G telecommunication network.BACKGROUND

[0002] The following discussion of the background to the invention is intended to facilitate an understanding of the present invention only. It should be appreciated that the discussion is not an acknowledgement or admission that any of the material referred to was published, known or part of the common general knowledge of the person skilled in the art in any jurisdiction as at the priority date of the invention.

[0003] Network traffic detection models have been trained and used to detect malicious traffic patterns in 5G networks. However, these network traffic detection models may not be able to detect new malicious traffic that is emerging in rapidly evolving 5G network environments. It is therefore important that deployed models can achieve model generalization, i.e. the ability of a deployed model to perform well on data it has not seen during training. In the context of a 5G network, where network traffic data distributions are continuously changing, maintaining a high model generalization has been challenging. Moreover, the increasing complexity and variability of 5G network traffic can make detection systems vulnerable to false positives or false negatives.

[0004] Dealing with the large volumes of data generated by 5G networks is another challenge. Efficiently processing this data and achieving effective model training and updates are crucial for maintaining accurate and real-time detection capabilities. If an offline trained model is to be updated after it has been deployed to generate classification for newly emerging traffic data distributions, the model needs to be retrained offline again. However, the training set used for such retraining is largely the same as that used for initial training with only a small fraction of the data being new. Using such an approach for retraining of the model, the training set will also need to be continuously updated for retraining. This can result in a waste of resources.

[0005] To mitigate some of these problems, some have turned to incremental learning to continuously train a model. One such solution is disclosed in the article "Self-evolving Malware Detection for Cyber Security using Network Traffic and Incremental Learning" by Xu et. al, in the 9th International Conference on Dependable Systems and Their Applications (DSA). The solution mainly compares feature extraction and joint training methods of incremental learning. Feature extraction (FE) is one of the methods to realize incremental learning. Its main implementation principle is that after training the model for the first time, certain layers, such as the convolutional layers, can already identify the characteristics of the training dataset. Therefore, in each subsequent training using both old and new data, such network layers are frozen, and only the fully connected layers are trained to allow the network to incrementally classify and identify the new dataset. In this manner, the model is able to recognize the features of both the previous data and the new data. As mentioned above, training using both old and new data is a waste of resources.

[0006] Joint training is also one of the disclosed methods to realize incremental learning. The difference between joint training and features extraction is that joint training does not freeze any parameters on the network. During model training under joint training, the training set of joint training increases over time. At each training, the dataset includes the whole training dataset used in the previous training plus new data. In other words, the dataset for training grows over time. Since the datasets used each time for joint training includes all data previously used for training, the model can learn new malicious attacks without forgetting old malicious attacks. However, training with such datasets can be wasteful on resources.

[0007] There is therefore a need for a method and apparatus which addresses, at least in part, one or more of the forgoing problems.SUMMARY

[0008] According to an aspect of the present disclosure, there is provided a method of detecting malicious traffic in a telecommunication network. The method includes training a machine learning model to identify malicious traffic using a training dataset to obtain a trained model and deploying the trained model as a deployed model to detect malicious traffic in an incoming traffic stream. The method further includes incrementally training the deployed model to identify new malicious traffic using newdata by applying a learning without forgetting method, wherein data previously used for training the deployed model is not used in the training and any layer of the deployed model may be changed, and deploying the incrementally trained model to detect malicious traffic in the incoming traffic stream.

[0009] In some embodiments of the method, the method further includes terminating incremental training early without all the new data being used in the incremental training.

[0010] In some embodiments of the method, the new data has no overlap with the data previously used for training.

[0011] In some embodiments of the method, the new data has some overlap with the data previously used for training.

[0012] In some embodiments of the method, incrementally training the deployed model includes splitting the new data into a plurality of batches and incrementally training the deployed model batch by batch until all of the batches in the plurality of batches have been used.

[0013] In some embodiments of the method, the method further includes after training with a batch, determining a performance measure of the incrementally trained model, and wherein deploying the incrementally trained model includes deploying the incrementally trained model only if the performance measure meets performance requirements.

[0014] In some embodiments of the method, determining a performance measure comprises determining at least one of an ability of the incrementally trained model to identify old malicious traffic and an ability to identify the new malicious traffic.

[0015] In some embodiments of the method, the ability of the incrementally trained model to identify old malicious traffic is based on a difference between an accuracy of the trained model and an accuracy of the incrementally trained model in identifying the old malicious traffic.

[0016] In some embodiments of the method, the method further includes adjusting at least one hyperparameter of the deployed model; and repeating incrementally training the deployed model if it is determined that the performance measure does not meet the performance requirements.

[0017] In some embodiments of the method, the method further includes switching the deployed model to a different model if it is determined that the performancemeasure of the incrementally trained model does not meet the performance requirements after the at least one hyperparameter has been adjusted a predetermined number of times.

[0018] In some embodiments, the telecommunication network comprises a 5G telecommunication network.

[0019] According to another aspect of the present disclosure, there is provided a server having a processor configured to use a deployed model to detect malicious traffic in an incoming traffic stream, the deployed model having been trained using a training dataset to identify malicious traffic. The processor is configured to also incrementally train the deployed model to identify new malicious traffic using new data by applying a learning without forgetting method, wherein data previously used for training the deployed model is not used in the training and any layer of the deployed model may be changed. The processor is configured further to use the incrementally trained model to detect malicious traffic in the incoming traffic stream.

[0020] In some embodiments of the server, the processor is further configured to terminate incremental training early without all the new data being used in the incremental training.

[0021] In some embodiments of the server, incrementally training the deployed model includes splitting the new data into a plurality of batches, and incrementally training the deployed model batch by batch until all of the batches in the plurality of batches have been used.

[0022] In some embodiments of the server, the processor is further configured to, after training with a batch, determine a performance measure of the incrementally trained model, and wherein deploying the incrementally trained model comprises deploying the incrementally trained model only if the performance measure meets performance requirements.

[0023] In some embodiments of the server, determining a performance measure comprises determining at least one of an ability of the incrementally trained model to identify old malicious traffic and an ability to identify the new malicious traffic.

[0024] In some embodiments of the server, the ability of the incrementally trained model to identify old malicious traffic is based on a difference between an accuracy of the trained model and an accuracy of the incrementally trained model in identifying the old malicious traffic.

[0025] In some embodiments of the server, the processor is further configured to adjust at least one hyperparameter of the deployed model; and repeat incrementally training the deployed model if it is determined that the performance measure does not meet the performance requirements.

[0026] In some embodiments of the server, the processor is further configured to switch the deployed model to a different model if it is determined that the performance measure of the incrementally trained model does not meet the performance requirements after the at least one hyperparameter has been adjusted a predetermined number of times.

[0027] According to another aspect of the present disclosure, there is provided a program storage device readable by a computing device, tangibly embodying a program of instructions, executable by the computing device to perform the abovedescribed method.

[0028] Other aspects and advantages of the invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the invention.BRIEF DESCRIPTION OF DRAWINGS

[0029] The invention will be better understood with reference to the drawings, in which:

[0030] Figure 1 is a flowchart showing a sequence of steps for detecting malicious traffic in a telecommunication network according to one embodiment of the invention;

[0031] Figure 2 is a flowchart showing a sequence of steps for detecting malicious traffic in a telecommunication network according to another embodiment of the invention;

[0032] Figure 3 is a block diagram illustrating typical apparatus in the telecommunication network in Figures 1 and 2, one of which the sequence of steps in either Figure 1 or Figure 2 is implemented therein; and

[0033] Figure 4 is a block diagram illustrating typical elements of the apparatus in which the sequence of steps is implemented therein.DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] Throughout this document, unless otherwise indicated to the contrary, the terms “comprising”, “consisting of”, “having” and the like, are to be construed as non- exhaustive, or in other words, as meaning “including, but not limited to.”

[0035] Furthermore, throughout the specification, unless the context requires otherwise, the word “include” or variations such as “includes” or “including” will be understood to imply the inclusion of a stated integer or group of integers but not the exclusion of any other integer or group of integers.

[0036] Throughout the description, it is to be appreciated that the term ‘processor / controller’ and its plural form include microcontrollers, microprocessors, programmable integrated circuit chips such as application specific integrated circuit chip (ASIC), computer servers, electronic devices, and / or combination thereof capable of processing one or more input electronic signals to produce one or more output electronic signals. The controller includes one or more input modules and one or more output modules for processing of electronic signals.

[0037] Throughout the description, it is to be appreciated that the term ‘server’ and its plural form can include local, distributed servers, and combinations of both local and distributed servers.

[0038] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as is commonly understood by a skilled person to which the subject matter herein belongs.

[0039] As shown in the drawings for purposes of illustration, the invention may be embodied in an efficient method that can be continually trained for detecting the latest malicious traffic in a 5G telecommunication network. Existing methods tend to be overly forgetful, that is, after learning new malicious traffic, they forget some previously learned malicious traffic. Figure 1 shows a sequence 2 of steps for implementing a method of detecting malicious traffic in a 5G telecommunication network. The method includes training 4 a machine learning model to identify malicious traffic using a training dataset to obtain a trained model. The method further includes deploying 6 the trained model as a deployed model to detect malicious traffic in an incoming traffic stream. The method further includes incrementally training 8 the deployed model to identify new malicious traffic using new data by applying a learning without forgetting method. Data previously used for training the deployed model is not used in the incremental training, and any layer of the deployed model may be changed. Themethod yet further includes deploying 10 the incrementally trained model to detect malicious traffic in the incoming traffic stream.

[0040] Specifically, Figure 2 is a sequence 12 of steps for detecting malicious traffic in a 5G telecommunication network according to an embodiment of the invention. The sequence 12 starts in a TRAIN MODEL step 4, wherein two or more different machine learning models, hereinafter referred to simply as models or networks, are trained to identify malicious traffic using an offline training dataset to obtain respective trained models. For example, three different models may be trained in this TRAIN MODEL step. The three machine learning models may include, but are not limited to, a support vector machine model (model A), a logistic regression model (model B) and a perceptron model (model C). The training dataset may be any suitable network traffic data including, but not limited to, the 5G-NIDD dataset. This 5G-NIDD dataset is a labelled dataset, which has clearly stated which traffic session is benign or malicious. The 5G-NIDD dataset contains data extracted from a 5G testbed. The testbed is attached to the 5G Test Network in the University of Oulu, Finland. The malicious and benign data are collected from two base stations, where network traffic from an attacker node and several benign 5G users in the testbed pass through. The attack scenarios in the dataset include Denial-of-Service (DoS) attacks and port scans known to those skilled in the art. The DoS attacks include Internet Control Message Protocol (ICMP) Flood, User Datagram Protocol (UDP) Flood, SYN Flood, HTTP Flood, and Slowrate DoS attacks. The port scans include SYN Scan, TCP Connect Scan, and UDP Scan attacks. Those skilled in the art will readily recognize that training involves identifying and extracting protocol-agnostic features, i.e. time-series features, statistical numerical features, side-channel features, etc, from the labelled 5G cybersecurity network traffic in the dataset. These features from both the offline training dataset (base training dataset) and an offline test data are fed into the models for training and testing the models to enable them to identify existing malicious traffic.

[0041] After obtaining the three trained models that can identify existing malicious traffic, the sequence proceeds to a DEPLOY MODEL step 6, wherein one of the models, for example, the best performing model of the three trained models, is deployed as a model to detect malicious traffic in an incoming traffic stream. The sequence 12 next proceeds to NEW DATA AVAILABLE FOR TRAINING? decision step 14, wherein it is determined if new data is available for training the deployedmodel that was previously trained using the base training dataset. The new data may be the same as or different from the data previously used for training. That is, there may be no overlap between the new data and the base training dataset or there may be some overlap between them. This new data is typically smaller in size than the base training dataset.

[0042] If it is determined in this NEW DATA AVAILABLE FOR TRAINING? decision step 14 that no new data for training is available, the sequence 12 proceeds to a DETECT MALICIOUS TRAFFIC step 16, wherein the deployed model is used to detect malicious traffic in the incoming traffic stream. If, however, it is determined in the NEW DATA AVAILABLE FOR TRAINING? decision step 14 that new data is available for training, the sequence 12 proceeds to a SPLIT NEW DATA INTO BATCHES step 18, wherein selected features of the new incoming data are extracted and the new incoming data is split into smaller batches according to a given batch size. Each batch may include one or more malicious traffic data and other benign traffic.

[0043] The sequence 12 next proceeds to an ALL BATCHES PROCESSED? decision step 20, wherein it is determined if all the batches of new data are processed. If it is determined in this ALL BATCHES PROCESSED? decision step 20 that not all batches of data are processed, the sequence 12 proceeds to an INCREMENTALLY TRAIN MODEL step 8, wherein the deployed model is trained using a next available batch of data to train the deployed model. The deployed model is trained using only this new data; the base training dataset is not used. In other words, the new data is not added to data previously used for training to make a complete training dataset including both old and new malicious traffic. Incremental training in this step 8 involves training the deployed model to identify new malicious traffic in the batch of new data by applying a Learning without Forgetting and an Early Exit technique. Learning without Forgetting (LwF) and early stopping techniques allow the model to be trained with reduced catastrophic forgetting.

[0044] By using LwF, the deployed model can be trained so that it can perform reasonably well in identifying both old malicious traffic and new malicious traffic when only new malicious traffic data is used for training. The old malicious traffic data in the base training dataset is not used in this INCREMENTAL TRAINING step 8. LwF adds a new branch in the deployed model for the new malicious traffic and then trains all necessary parameters or layers in the model to optimize identification of both the oldmalicious traffic and the new malicious traffic. More specifically, weights of the deployed model are updated so that the new malicious traffic may be further identified in addition to the identification of old malicious traffic. However, this updating of the weights of the deployed model may affect its ability to identify the old malicious traffic. LwF is inspired by knowledge distillation. For LwF, a distillation weight (A) needs to be determined to achieve a trade-off between the new malicious traffic classification loss and the distillation loss. Distillation is the process of transferring knowledge from the previously trained model to a new model by making the new model mimic the behavior of the old model. The distillation weight (A) is a hyperparameter that controls the influence of the distillation loss during training. It determines how much the new model should try to mimic the old model’s output. If the distillation weight is too large, the model might over-emphasize the importance of the old malicious traffic and underperform on the new malicious traffic. If the weight is too small, the model might forget the old malicious traffic. Fine-tuning of this parameter is required to achieve a trade-off in LwF.

[0045] Early exiting in incremental learning involves monitoring the performance of the deployed model during incremental learning training. It will stop training when the trained deployed model reaches an acceptable trade-off between accuracy and model forgetting. Specifically, during the model training in the step 8, the performance of the deployed model on the old types of malicious traffic data (base test set) and the performance of the deployed model on the new incoming data test set are inversely proportional to each other. The incremental training is aimed at enhancing the performance of the deployed model on newly incoming types of malicious traffic data without compromising the model's ability to detect old types of malicious traffic data, or at least minimizing the speed at which such a compromise occurs. The role of early exiting is to monitor the improvement in the model's detection of new data and the potential decrease in its detection of old data at each round of training, ensuring it reaches an acceptable trade-off and to stop any further training. Once the trained deployed model is determined to have reached an acceptable performance level after training with some data in the batch, the training is stopped. In other words, not all data in the batch will be used in incremental training. Once training is stopped, the remaining data in the batch will be discarded. Such early stopping of training in the INCREMENTALLY TRAIN MODEL step 8 may be applied manually or automatically.A user can monitor the performance of the trained model. When the performance has reached an acceptable level, the user can manually end the training. Alternatively, the performance level can be automatically monitored during training. When the performance level is detected to be at a predetermined level, training will be terminated. The performance level may include a catastrophic forgetting rate (CFR) and a detection performance (DP), which will be described later.

[0046] With this incremental training by LwF and early exiting in the INCREMENTALLY TRAIN MODEL step 8, instead of retraining the latest model from scratch with a combination of both the old and the new data, the deployed model is further trained based only on the new data. The deployed model is thus continuously trained online using newly emerging data batch by batch. If the data size of each batch is one, such learning is similar to online learning. If the data size of each batch is larger than one, the learning is similar to online batch learning. The batch size may be selected based on a trade-off between computational efficiency and model performance. Typically for deep learning, a batch size of larger than one is chosen so that there is a good balance between computational efficiency and model performance. The batch size is typically determined based on specific tasks for learning and available computational resources, such as memory and processing capacity, the desired training speed, and the pursuit of generalization ability. Thus, an incremental learning system can keep up to date with new incoming traffic without the need to store and reprocess all past traffic. This can be much more efficient and scalable for large or fast-changing traffic datasets and addresses the high-processing and resource utilization challenges of 5G.

[0047] After the deployed model is trained with the batch of new data, the sequence 12 proceeds to an EVALUATE PERFORMANCE MEASURE step 30, wherein the incrementally trained model is tested using two different test data inputs. These test data inputs are the offline network test data and new incoming test data. In this step, at least one of two performance measures are evaluated based on the test data inputs. These performance measures include, but are not limited to, a catastrophic forgetting rate (CFR) and a detection performance (DP).

[0048] The CFR is based on the accuracy of identifying old malicious traffic using the offline test data for both the trained model and the incrementally trained model, and it is given by:(Original accuracy - Current accuracy) / Original accuracy whereinOriginal accuracy is the accuracy of identifying old malicious traffic by the trained model; andCurrent accuracy is the accuracy of identifying old malicious traffic by the incrementally trained model.

[0049] The DF is the measure of how accurately the incrementally trained model is able to identify new malicious traffic in the new incoming test data. The new incoming test data is fed into the incrementally trained model to see how it performs in identifying new malicious traffic.

[0050] The sequence 12 next proceeds to a PERFORMANCE MEASURE MEETS REQUIREMENT? decision step 32, wherein it is determined if the performance measure of the deployed model meet predetermined performance requirements. If it is determined that the performance measure meet the performance requirements, i.e., the CFR and DP exceeds respective thresholds, the sequence 12 proceeds to a DEPLOY INCREMENTALLY TRAINED MODEL step 34, wherein the incrementally trained model is deployed in place of the earlier deployed model for detecting malicious traffic in the incoming traffic stream. The sequence 12 then returns to the ALL BATCHES PROCESSED? decision step 20 to repeat training of the deployed model with a next available batch of new data.

[0051] If it is however determined that the performance measure, i.e., at least one of the CFR and the DP, does not meet the predetermined performance requirements, the sequence 12 proceeds to a NUMBER OF TIMES HYPERPARAMETERS ARE ADJUSTED? decision step 36, wherein it is determined if the number of times model hyperparameters are adjusted has reached a predetermined number. If it is determined in this step 36 that the predetermined number has not been reached, the sequence 12 proceeds to an ADJUST MODEL HYPERPARAMETERS step 38, wherein the hyperparameters of the deployed model are adjusted. The model hyperparameters may be adjusted, for example by a user, in an attempt to improve the model's performance. For example, to enhance detection performance, hyperparameters like a learning rate and the batch size may be adjusted. However, as the detection performance improves, the model's forgetting rate may increase. In thiscase, relevant hyperparameters, such as the distillation weight (A) of LwF, may be adjusted to control the rate of forgetting.

[0052] The sequence 12 then returns to the SPLIT NEW DATA INTO BATCHES step 18 to repeat the steps so as to incrementally train the hyperparameter adjusted model using the same batch data as before and to determine if its performance measure meets the performance requirements.

[0053] If it is determined in the NUMBER OF TIMES HYPERPARAMETERS ARE ADJUSTED? decision step 36 that the number of times the hyperparameters are adjusted has reached the predetermined number, the sequence 12 proceeds to a IS THERE ANOTHER MODEL? step 40, where it is determined if a there is yet another model to be evaluated. If it is determined that there is yet another model to be evaluated, the sequence 12 proceeds to a DEPLOY ANOTHER MODEL step 42, wherein another model is selected for incrementally training with the batch of new data. For example, if model A is first deployed in the DEPLOY MODEL step 6 but after incremental training, it is discovered that its performance measure does not meet the performance requirements, model B or model C may be next deployed in the DEPLOY ANOTHER MODEL step 42 and made to undergo incremental training as described above to see if it can perform better than model A. This is repeated until a model is found whose performance measure meets the performance requirements.

[0054] If it is determined in the IS THERE ANOTHER MODEL? decision step 40 that all models have been exhausted, i.e., none of the models has a performance measure that meets the performance requirements even after the hyperparameters have been adjusted for each model for the predetermined number of times, the sequence 12 proceeds to a DEPLOY BEST MODEL step 44, wherein the best performing model of the available models is deployed. After this step 44, the sequence 12 returns to the NEW DATA FOR TRAINING decision step 14.

[0055] In this manner, the above-described method allows continuous training of the deployed model using new data batches, evaluating its performance measure after each training session based on both incoming test data and old offline network test data, and adjusting the hyperparameters where necessary. This iterative process continues until a model is obtained that meets the performance requirements.

[0056] Experimental results have shown that the above-described method overcomes the problem of model forgetfulness and is able to identify both old and new malicious traffic.

[0057] Figure 3 is a block diagram showing apparatus in the 5G telecommunication network 50. The network includes a core network and a radio access network (RAN). The core network provides many of the key network functions for the network, including but not limited to, an Access and Mobility Management Function (AMF), an Authentication Server Function (AUSF), a Session Management Function (SMF), a User plane function (UPF), a Unified Data Management (UDM), amongst others. The RAN provides a connection between an individual device, e.g. a mobile device, and the data network through a radio link.

[0058] The above-described method can be implemented as a standalone malicious network traffic detection function (MNTDF) 52 within the Control Plane of the 5G core network or as a part of the the Network Data Analytics Function (NWDAF) (not shown) introduced in TS 29.520 of 3GPP (3GPP. 5G System; Network Data Analytics Services; Stage 3. Technical Specification (TS) 29.520, 3rd Generation Partnership Project (3GPP), 2023. Version 18.1.0.). The MNTDF 52 may be hosted in a dedicated server or a common server hosting the other network functions of the 5G core network.

[0059] Figure 4 is a block diagram illustrating typical elements of a computing system 60 that may be appropriately programmed to function as the above-described server. The elements include a programmable processor 62 connected to a system memory 64 via a system bus 66. The processor 62 accesses the system memory 64 as well as other input / output (I / O) channels 68 and peripheral devices 70. The computing system 60 further includes at least one program storage device 72, such as a CD-ROM, tape, magnetic media, EPROM, EEPROM, ROM or the like. The computing system 60 stores one or more computer programs that implement the method of detecting malicious traffic according to an embodiment of the present invention. The processor 62 reads and executes the one or more computer programs to perform the method. Each of the computer programs may be implemented in any desired computer programming language (including machine, assembly, high level procedural, or object oriented programming languages). In any case, the language may be a compiled or interpreted language.

[0060] Advantageously, incremental learning that is employed in the abovedescribed method allows a model to be updated gradually. The model learns new tasks without forgetting old tasks. Incremental learning leverages previously learned knowledge and the model only needs fine-tuning for new data. This requires less computational and storage resources and is therefore more resource efficient. Incremental learning also allows new incoming data to be learned continuously, thereby improving the model’s robustness. And since the model is trained only using new data, the risk of data leakage for the base training dataset is low.

[0061] Although the present invention is described as implemented in the above described embodiments], it is not to be construed to be limited as such. It is to appreciated that modifications and improvements may be made without departing from the scope of the present invention.

[0062] For example, although the method is described in the context of a 5G telecommunication network, the method may be used to detect any malicious traffic in any telecommunication network.

[0063] As another example, incremental training is described to include training using a LwF technique and to also exit training early once it is determined that the trained model’s performance is acceptable. However, it is also possible during incremental training that there be no early exit. In other words, there is no monitoring of any performance level of the model during incremental training. All data in the batch will be used during incremental training.

[0064] As another example, it is described that performance measure of the model is evaluated and compared with performance requirements. However, this is not to be construed to be limited as such. After incremental training, the trained model may be deployed straightaway without having to evaluate its performance measure.

[0065] With regard to performance measure, it is described that the CFR and DP are evaluated. It is possible that only one of these two is evaluated instead. In some other embodiments, performance measures other than these two may also be used.

[0066] As yet a further example, it is described that a number of hyperparameters are adjusted. It should be noted that in some embodiments, only one hyperparameter is adjusted.

[0067] As yet another example, learning without forgetting is described as retaining a network’s weights. However, those skilled in the art readily recognizes that learningwithout forgetting may involve two neural networks, an old network and a new network. The old network contains knowledge about the previous tasks. The new network is trained on the current task. The old network's knowledge is transferred to the new network. This can be carried out, instead of transferring the old network's weights, by using the old network as a teacher to provide pseudo-labels or soft targets for the new network during training.

[0068] It should be further appreciated by the person skilled in the art that one or more of the above modifications or improvements, not being mutually exclusive, may be further combined to form yet further embodiments of the present invention.

Claims

CLAIMS1. A method of detecting malicious traffic in a telecommunication network, the method comprising: training a machine learning model to identify malicious traffic using a training dataset to obtain a trained model; deploying the trained model as a deployed model to detect malicious traffic in an incoming traffic stream; incrementally training the deployed model to identify new malicious traffic using new data by applying a learning without forgetting method, wherein data previously used for training the deployed model is not used in the training and any layer of the deployed model may be changed; and deploying the incrementally trained model to detect malicious traffic in the incoming traffic stream.

2. The method according to Claim 1 , further comprising terminating incremental training early without all the new data being used in the incremental training.

3. The method according to Claim 1 or Claim 2, wherein the new data has no overlap with the data previously used for training.

4. The method according to Claim 1 or Claim 2, wherein the new data has some overlap with the data previously used for training.

5. The method according to any one of the preceding claims, wherein incrementally training the deployed model comprises: splitting the new data into a plurality of batches; and incrementally training the deployed model batch by batch until all of the batches in the plurality of batches have been used.

6. The method according to Claim 5, further comprising after training with a batch, determining a performance measure of the incrementally trained model, and whereindeploying the incrementally trained model comprises deploying the incrementally trained model only if the performance measure meets performance requirements.

7. The method according to Claim 6, wherein determining a performance measure comprises determining at least one of an ability of the incrementally trained model to identify old malicious traffic and an ability to identify the new malicious traffic.

8. The method according to Claim 7, wherein the ability of the incrementally trained model to identify old malicious traffic is based on a difference between an accuracy of the trained model and an accuracy of the incrementally trained model in identifying the old malicious traffic.

9. The method according to Claim 6, further comprising: adjusting at least one hyperparameter of the deployed model; and repeating incrementally training the deployed model if it is determined that the performance measure does not meet the performance requirements.

10. The method according to Claim 9, further comprising: switching the deployed model to a different model if it is determined that the performance measure of the incrementally trained model does not meet the performance requirements after the at least one hyperparameter has been adjusted a predetermined number of times.11 . The method according to any one of the preceding claims, wherein the telecommunication network comprises a 5G telecommunication network.

12. A server comprising a processor configured to: use a deployed model to detect malicious traffic in an incoming traffic stream, the deployed model having been trained using a training dataset to identify malicious traffic; incrementally train the deployed model to identify new malicious traffic using new data by applying a learning without forgetting method, wherein datapreviously used for training the deployed model is not used in the training and any layer of the deployed model may be changed; and use the incrementally trained model to detect malicious traffic in the incoming traffic stream.

13. The server according to Claim 12, wherein the processor is further configured to terminate incremental training early without all the new data being used in the incremental training.

14. The server according to Claim 12 or Claim 13, wherein incrementally training the deployed model comprises: splitting the new data into a plurality of batches; and incrementally training the deployed model batch by batch until all of the batches in the plurality of batches have been used.

15. The server according to Claim 14, wherein the processor is further configured to, after training with a batch, determine a performance measure of the incrementally trained model, and wherein deploying the incrementally trained model comprises deploying the incrementally trained model only if the performance measure meets performance requirements.

16. The server according to Claim 15, wherein determining a performance measure comprises determining at least one of an ability of the incrementally trained model to identify old malicious traffic and an ability to identify the new malicious traffic.

17. The server according to Claim 16, wherein the ability of the incrementally trained model to identify old malicious traffic is based on a difference between an accuracy of the trained model and an accuracy of the incrementally trained model in identifying the old malicious traffic.

18. The server according to Claim 15, wherein the processor is further configured to:adjust at least one hyperparameter of the deployed model; and repeat incrementally training the deployed model if it is determined that the performance measure does not meet the performance requirements.

19. The server according to Claim 18, wherein the processor is further configured to: switch the deployed model to a different model if it is determined that the performance measure of the incrementally trained model does not meet the performance requirements after the at least one hyperparameter has been adjusted a predetermined number of times.

20. A program storage device readable by a computing device, tangibly embodying a program of instructions, executable by the computing device to perform the method of detecting malicious traffic in a telecommunication network according to any one of Claims 1 -11.

Images