METHOD FOR DETECTING AND MITIGATING DISTRIBUTED DENIAL-OF-SERVICE ATTACKS BASED ON REAL-TIME DISTRIBUTED LEARNING

Description

16 METHOD FOR DETECTION AND MITIGATION OF DISTRIBUTED DENIAL OF SERVICE ATTACKS BASED ON TIME-DISTRIBUTED LEARNING REAL FIELD OF THE INVENTION

[001] This invention relates to the field of mobile communication networks and cybersecurity. Specifically, it relates to protecting the initial access procedure of user equipment (UEs) to the Radio Access Network (RAN) against Distributed Denial of Service (DDoS) attacks. This invention proposes a method using multi-agent reinforcement learning algorithms for identifying and mitigating DDoS attacks. Intelligent access network control mechanisms, based on real-time distributed learning, are used to establish adaptive and cooperative knowledge exchange mechanisms between RAN elements in order to create more comprehensive, faster, and more efficient policies. FUNDAMENTALS OF THE INVENTION AND STATE OF THE ART

[002] Several studies have recently been conducted on detecting DDoS (Distributed Denial of Service) attacks in the context of future generations of mobile communication networks. Many of these studies have focused on approaches based on Machine Learning (ML) and Artificial Intelligence (AI). In addition to conventional ML and AI algorithm training techniques, such as supervised and unsupervised learning, Federated Learning (FL) techniques are also widely applied, with the aim of promoting distributed and collaborative learning without compromising sensitive data from its isolated components.

[003] In [1] the authors propose TenaxDoS, a framework, or method, for detecting anomalies, especially DDoS attacks, in the context of Open RAN mobile communication networks, based on a continuous federated learning approach. The method works in three basic steps: Initialization, where the individual neural network models – each model hosted on a Near-RT RIC – receive the initial parameters of the global model from a central FL server hosted on the Non-RT RIC; local training, where, for each round of network communication, the models Petition 870240111297, dated 12 / 30 / 2024, page 6 / 57 / 16 individuals have a dataset containing legitimate and malicious network traffic data, which is used for local training. At the end of the local training, the parameters are sent to the FL server; finally, the Federated Aggregation step, which applies a federated average (FedAvg) of the individual parameters to obtain the global parameters.

[004] This approach, however, proposes the use of multilayer perceptron (MLP) networks, which are less effective at detecting complex patterns in heterogeneous data or data with strong temporal correlation. Furthermore, the use of FedAvg can be very sensitive to the very particular conditions of an agent, since it is sensitive to the individual parameters of each model, such as the number of layers and neurons.

[005] In [2] a framework for anomaly detection in Open RAN mobile networks is proposed, also based on FL, but with a peer-to-peer (P2P) approach, that is, agents exchange information directly with each other, dispensing with the need for a central server. This distributed approach is even capable of detecting DDoS attacks. In this approach, the authors propose that the training models and individual detectors be implemented in dedicated xApps and hosted on Near-RT RICs, distributed across edge servers, with cloud connections between them. The data for training the models are network data stored on these edge servers. Thus, when a detector identifies an anomaly in the network, such as a DDoS attack, an auxiliary xApp in the respective Near-RT RIC triggers control actions for the E2 nodes.The authors further propose 4 potential variations of this approach: (i) P2P communication between Near-RT RICs; (ii) P2P communication between groups of Near-RT RICs, which aims to mitigate effects due to unbalanced data between different network contexts; (iii) Hierarchical communication between groups of Near-RT RICs, in this approach each group has a Near-RT RIC responsible for collaboration between the groups; (iv) Encrypted P2P communication, this approach is identical to the first, including a fully secure homomorphic encryption (FHE) process, the main difference being the application of encryption in the communication between Near-RT RICs. Petition 870240111297, dated 12 / 30 / 2024, page 7 / 57 / 16

[006] Again, the proposed collaboration approach is based on FedAvg, which makes the solution sensitive to the model parameters. Furthermore, although approaches (ii) and (iii) are proposed to mitigate damage due to data heterogeneity, it is possible to see that applying FedAvg at the group level is also subject to the same problems. Finally, the authors do not present performance evaluations of the solution in the face of a DDoS attack, only a DoS attack, which does not explore the distributed nature of the application scenario.

[007] In addition to scientific articles, patents related to the topic are available. [3] patents a DDoS attack detection method based on Hidden Markov Cooperation (HMM) and Q-learning. HMM models are used to model the sequence of dynamic states, while Q-learning enables distributed cooperative detection that optimizes the response to the attack. To this end, we can identify two important steps in the invention: (i) detection step, where TCP states are monitored and subsequently used to identify normal patterns and anomalous behaviors associated with DDoS attacks.Thus, the HMM model is trained in order to accurately define the dynamic sequence of TCP connection states and identify anomalous network patterns; (ii) secondly, the distributed detection stage with Q-learning, where local nodes are trained to learn local patterns, and then cooperation is established where nodes exchange information to establish robust optimizations at a global level.

[008] However, this Q-learning-based approach can be easily limited in terms of scalability, since the number of states and actions grows exponentially, and is limited by the computational capacity of the network nodes.

[009] In [4] a network intrusion detection method based on federated learning is patented, which can be applied in the context of DDoS attacks. The general framework is very similar to that proposed in [1], where local agents perform their individual training and share their parameters and loss values ​​with the server, which applies a FedAvg to determine global parameters. A different point in this technique is the use of recurrent neural networks, such as LSTM, instead of an MLP, as proposed in [1]. Petition 870240111297, dated 12 / 30 / 2024, page 8 / 57 / 16

[010] It is noteworthy to realize, however, that the invention of [4] is subject to the same problems of applying a weighted average to the individual parameters described above, as it makes the overall model sensitive to unbalanced data.

[011] Thus, state-of-the-art cooperative solutions rely heavily on federated learning, a technique that excels in terms of data privacy and security. However, solutions based on federated learning have significant limitations when applied to the detection of DDoS attacks on Open RAN networks, such as: • Communication overhead: the training process requires multiple rounds of communication between network nodes and the central server to exchange model updates, introducing significant latency that can compromise real-time response to attacks. The proposed solution leverages the concept of real-time cooperative learning, significantly reducing the number of iterations between network nodes; • Synchronization: the need for synchronization between network nodes can create performance bottlenecks and delay the response to emerging threats. The proposed solution considers asynchronous solutions for mitigating attacks; • Aggregation complexity: the process of combining model updates from multiple network nodes presents technical challenges, especially when some nodes have more representative or relevant data than others. The proposed solution considers solutions based on decentralized models and cooperation between entities at the same hierarchical level; • Resource heterogeneity: the diversity of computational capabilities and data quality among network nodes can result in asymmetrical contributions to the overall model, potentially degrading its performance. The proposed solution leverages the Open RAN architecture's interface standardization features to establish appropriate contributions to the real-time learning process; • Limited scalability: increasing the number of participating nodes can result in communication overhead, since most state-of-the-art solutions... Petition 870240111297, dated 12 / 30 / 2024, page 9 / 57 / 16: The technique requires a large amount of data for training overly complex algorithms. The proposed solution explores real-time learning solutions with lower computational complexity. OBJECTIVES AND ADVANTAGES OF THE INVENTION

[012] This invention proposes a method for implementing multi-agent reinforcement learning algorithms for detecting and mitigating DDoS (Distributed Denial of Service) attacks during the initial access process of devices to the radio access network. The method exploits the Open RAN architecture, leveraging its flexibility and distributed components to establish adaptive mechanisms for cooperative control of the access network, enabling a coordinated and real-time response to potential security threats. The main objectives of this invention are: • Integration of a DDoS attack detection and mitigation method into the Open RAN architecture: the invention is designed to integrate with the Open RAN architecture, exploiting its principles of disaggregation, virtualization, and openness of the radio access network. This is achieved through the use of its interfaces to access metrics, messages, and procedures, allowing for flexible and scalable implementation. This approach ensures that the solution is compatible with a new generation of access networks, a significant limitation of existing solutions in the current state of the art; • Development of a multi-agent reinforcement learning-based method: the proposed invention is designed to explore cooperative reinforcement learning solutions for DDoS attack detection and mitigation. Through information exchange between the components of the access network, the proposed solution promotes an adaptive configuration of detection and mitigation algorithms at different points in the network, obtaining faster and more efficient solutions. Existing solutions in the current state of the art exploit cooperative learning between the units of the Open RAN architecture in a limited way, using approaches with little scalability or Petition 870240111297, dated 12 / 30 / 2024, page 10 / 57 / 16, not exploring the possibilities of communication between the components of the access network; • Development of a real-time detection and mitigation method: Unlike existing solutions that rely on time-consuming statistical analyses and / or require extensive training datasets, the proposed invention implements advanced adaptive learning techniques that operate in real time. The proposed method continuously processes network data as it is generated, allowing for the immediate identification of suspicious patterns and anomalous behaviors characteristic of DDoS attacks at different points in the network. This instantaneous analysis capability, combined with dynamic adaptation mechanisms, allows the system to respond quickly to emerging threats, keeping the network operational and protected even during attack attempts.Real-time performance is especially critical for preserving the availability and quality of service for legitimate UEs, or User Equipment, avoiding delays and vulnerability periods typical of systems based on retrospective analysis, as observed in the current state of the art.

[013] The proposed solution presents significant advantages over the state of the art, mainly due to its real-time response capability and lower dependence on centralized synchronization. While existing state-of-the-art solutions rely heavily on federated learning, which is more suitable for scenarios where data privacy is critical and latency is not a limiting factor, the proposed solution explores multi-agent reinforcement learning, which proves more appropriate for applications that demand fast and adaptive responses, such as protection against DDoS attacks. GENERAL DESCRIPTION OF THE INVENTION

[014] This invention proposes a method for implementing multi-agent reinforcement learning algorithms for the detection and mitigation of DDoS (Distributed Denial of Service) attacks during the initial access process. Petition 870240111297, dated 12 / 30 / 2024, page 11 / 57 / 16 of devices. The invention explores new metrics, signals, and procedures related to the interfaces and components of the Open RAN architecture to establish adaptive and cooperative mechanisms for knowledge exchange between RAN elements in order to create more comprehensive, faster, and more efficient mechanisms. In the proposed solution, network components act as reinforcement learning agents and thus undergo an adaptive learning process in which, through an iterative process, they establish the best strategy based on the objectives set by the network operator. Furthermore, they cooperate with each other to share available information and acquire knowledge to determine the most appropriate configuration to detect and mitigate DDoS attacks.These components are equipped with intelligence and establish information exchange strategies to accelerate their learning process; that is, they acquire knowledge from their interaction with other network components to proactively detect and mitigate attacks, thereby reducing signaling overload and network response latency. In this way, the proposed invention provides efficient signaling exchange between different points in the network, aiming for scalability and reliability with low latency in scenarios with multiple control elements and UEs (User Equipment), representing a significant contribution to the development of 6G mobile networks.

[015] The proposed invention is characterized by the innovative aspects highlighted below: • Integration of a DDoS attack detection and mitigation method into the Open RAN architecture: the invention proposes a machine learning method that integrates with the components of the Open RAN architecture (e.g., Near-RT RIC, O-RU, O-DU, O-CU) and uses standardized interfaces (e.g., E2, A1, O1) to incorporate metrics and messages into the detection and mitigation mechanisms. This approach ensures that the proposed solution is compatible with modern access networks, overcoming limitations of the prior art, which in many cases cannot adapt to new network architectures or require significant modifications; Petition 870240111297, dated 12 / 30 / 2024, page 12 / 57 / 16 • Development of a reinforcement learning-based method with multiple agents: the invention explores reinforcement learning to establish proactive solutions for detecting and mitigating DDoS attacks. Through the exchange of information between the components of the access network, an adaptive configuration of the decision-making network elements is promoted at different points in the network, obtaining faster and more efficient solutions. Existing solutions in the current state of the art exploit cooperative learning between the units of the Open RAN architecture in a limited way, using approaches with little scalability or not exploring the possibilities of communication between the components of the access network; • Development of a real-time detection and mitigation method: the invention incorporates real-time adaptive learning techniques into the detection and mitigation of DDoS attacks. The method continuously processes network data, identifying suspicious patterns and anomalous behaviors of DDoS attacks and sharing this knowledge with other network components to accelerate the network's ability to detect and respond to threats. By integrating dynamic adaptation mechanisms, the proposed solution allows its parameters to be adjusted dynamically without interrupting its operation. Furthermore, it allows the network to adjust its defense strategies based on the specific characteristics of the attack.

[016] Other possible uses for the proposed invention include: • Analyzing network access patterns and user equipment traffic to identify trends and anomalous behaviors in other contexts of mass access, in addition to DDoS attacks, such as large events, disasters, etc.; • Continuous monitoring and optimization of network performance and reinforcement learning policies.

[017] In summary, the invention utilizes a combination of multi-agent reinforcement learning integration into the Open RAN architecture; coordinated monitoring across multiple points in the network; real-time attack detection; proactive mitigation of Petition 870240111297, dated 12 / 30 / 2024, page 13 / 57 / 16 attacks, continuous learning and knowledge transfer in order to overcome security challenges in large-scale networks and achieve the protection objectives defined by the network operator. DESCRIPTION OF THE FIGURES

[018] The invention will be described in detail below, and for better understanding, reference will be made to the attached drawings, in which are represented: Figure 1: Illustrative diagram of the intelligent nodes of a mobile communications network, showing normal access on these nodes and showing an attack carried out on one of these nodes; Figure 2: Illustrative diagram of information sharing after detection of an attack on one of the intelligence-equipped nodes of a mobile communications network; Figure 3: Illustrative diagram of information sharing after an attack on one of the intelligence-equipped nodes of a mobile communications network, this sharing taking place using a central server; Figure 4: Flowchart showing the steps of the specific and main implementation mode of the proposed method; Figure 5: Sequence diagram showing the specific implementation of the method in an Open RAN network and using the components of this Open RAN network for the implementation of nodes with intelligence and data sharing. DETAILED DESCRIPTION OF THE INVENTION

[019] The METHOD FOR DETECTION AND MITIGATION OF DISTRIBUTED DENIAL OF SERVICE ATTACKS BASED ON REAL-TIME DISTRIBUTED LEARNING is implemented and used for the detection of attacks on networks, and more specifically, for the detection of distributed denial of service attacks on RAN networks, or Radio Access Networks, or even on Open RAN networks.

[020] The proposed invention incorporates multi-agent reinforcement learning algorithms in an Open RAN network to select the most appropriate configuration to detect and mitigate a distributed denial-of-service, or DDoS, attack. Petition 870240111297, dated 12 / 30 / 2024, page 14 / 57 / 16 Figure 1 illustrates a scenario of a distributed denial-of-service attack on a 5G mobile network, where multiple malicious devices attempt to overload network resources through simultaneous access requests.

[021] Multiple network nodes (NR) undergo an adaptive learning process to iteratively determine the most appropriate response to the different types of behavior observed in the initial access.

[022] The network nodes (NRs) responsible for intelligent control, also called intelligent nodes or agents, acquire their knowledge from interaction with the other components of the system. In addition to the direct collection of information from the node (NR) from the network, the proposed method incorporates knowledge transfer between nodes (NRs) to accelerate their decision-making process. This feature exploits characteristics of the Open RAN architecture, such as decentralization, to reduce signaling overhead and latency in the decision-making process.

[023] The designed solution offers great operational flexibility. That is, the learning strategy used can be defined in terms of one or more network parameters, such as initial access metrics, traffic metrics, quality of service metrics, etc. Through communication between network elements, each intelligent node (NR) can update its policy according to different information, such as network conditions directly shared with the node (NR), information shared from other intelligent nodes (NR), the network operator's objectives, etc. Figures 2 and 3 represent two possible implementations of the proposed solution.

[024] Thus, the proposed solution is a sophisticated project that aims to guarantee the adaptability and flexibility of the system. These objectives are achieved by employing techniques with reduced computational complexity.

[025] The method iteratively adjusts the RAN's response to initial access requests from user equipment, known as User Equipment or UEs, based on a cost function and allows immediate reaction from the network nodes responsible for intelligent control to abrupt variations in metrics. In case of identification of an attack at a given point in the network, the other network nodes (NR) responsible Petition 870240111297, dated 12 / 30 / 2024, page 15 / 57 / 16, regarding the response to DDoS attacks, receives information so that they are aware of the attack and can define an adjustment in their policies in a coordinated manner with the other objectives for the functioning of the network.

[026] In the proposed solution, an intelligent network node (NR) is an element of the RAN that undergoes an iterative decision-making process to determine the appropriate response to the initial access behavior of a given user device, being able to classify the accesses of these user devices as normal (AN) or legitimate accesses, or as attack accesses (AA). That is, the intelligent node (NR) is responsible for taking actions based on its policy, which is the mapping between actions, states, and rewards.

[027] In the proposed method, each intelligent node (NR) implements a set of actions in a sequential and coordinated manner. The first action is detection, that is, each intelligent node (NR) must determine whether the initial access request is legitimate or a DDoS attack. This action analyzes the metrics provided by the RAN, the knowledge acquired from previous decisions, and the information shared by other nodes to classify whether a DDoS attack is occurring or not. The second action is mitigation, to be executed if the access request must be rejected due to classification as an attack. From a specific set of measures that can be taken, the intelligent node (NR) chooses the most appropriate one, such as rejecting RRC connection attempts, proceeding with the RRC connection, changing the RRC connection configuration, etc. The third action is the transfer of knowledge to other intelligent nodes (NR).This information is sent selectively to reduce signaling overload, sharing only relevant information when necessary.

[028] After the set of actions is performed, two types of information are reported back to the intelligent node (NR). The first is called reward and is defined according to a specific cost function that the network wishes to optimize. It quantifies the quality of the actions performed according to the network's objectives. The second is called state, which indicates the values ​​of the network parameters after the implementation of the determined actions. The reward is always necessary, and the state Petition 870240111297, dated 12 / 30 / 2024, page 16 / 57 / 16 is important feedback information for updating the policy on the intelligence-equipped node (NR).

[029] The proposed method has a component called a central server (CS), a server that distributes the parameters of the method's agents under its coverage. In a network with a large number of intelligent nodes (NRs), the exchange of information between agents may take longer than is adequate for the detection process at other points in the network to be optimized before the attack reaches them. Thus, the central server (CS) helps update the agents' policies according to the environment data reported by each agent, observing the latency for this information exchange. To avoid signaling overload, the central server (CS) must be associated with intelligent nodes (NRs) within a specific region of interest, which must be limited in area and defined according to the network's objectives and parameter availability.

[030] The proposed solution defines reward as a cost function based on RAN metrics. As this is a method for detecting attacks on initial access in an Open RAN network, priority is given to metrics associated with Radio Resource Control (RRC) connection establishment procedures, such as RRC Setup, RRC Connection, RRC Reconfiguration, RRC Reestablishment, etc. Other metrics associated with network performance, such as latency, data transfer rate, or throughput, among others, can be considered in the cost function to quantify the effect of detection and mitigation decisions. The reward function can also include the impact of decisions made by other agents, the impact of the decision on overall network performance, etc. The information considered in the composition of the cost function is generically represented as {F_1, F_2, ..., F_N}, where N corresponds to the number of network parameters considered in the cost function.The cost function comprises a set of associated weights W = {W_1, W_2, ..., W_N}, which can be adjusted according to the network operator's objectives and routed to the nodes (NR) equipped with intelligence. These values ​​can... Petition 870240111297, dated 12 / 30 / 2024, page 17 / 57 / 16 to be modified to alter the behavior of the agents. One implementation using these cost functions and weights is as follows: Equation 1: R = W_{1} · F_{1} + WJ2} · F_{2} + ··· + W_{W} · F_{N} ;

[031] Based on the configuration of these method elements, the intelligent nodes (NR) initiate an iterative process of actions, calculation of associated rewards, and transmission of relevant information to other intelligent nodes (NR). The policy of each agent, that is, the interaction between the intelligent nodes (NR) and the network, is modeled according to the mapping between the actions that can be taken, the rewards calculated from the cost functions, and the states. The proposed method determines the most appropriate actions based on this mapping. Furthermore, a mechanism is proposed to accelerate the policy update process through the sharing of information between intelligent nodes (NR) at points in the network with anomalous behavior and intelligent nodes (NR) at points in the network still under normal conditions.When mapping advances to a reasonable level of knowledge defined according to the network operator's objectives, each intelligent node (NR) uses it to detect the DDoS attack, take mitigation action, and transmit the information to other intelligent nodes (NR) with great speed and efficiency. If changes in the scenario are observed, the method has the autonomy to update the policies and another solution is provided.

[032] Figure 4 shows the flowchart of the main steps in implementing the method, which includes node initialization (NR), continuous monitoring, attack detection, selection and execution of mitigation actions, and information sharing between agents. The steps performed more specifically in this specific and preferred implementation shown in Figure 4 are: • Stage 1 (1): Implementation of multiple reinforcement learning agents in nodes (NR) of the network, with the configuration parameters of the proposed method being defined according to the network objectives, with the total number of simultaneous connections and the total number of attempts being non-limiting examples. Petition 870240111297, dated 12 / 30 / 2024, page 18 / 57 / 16 of connection, cost function parameters, quality of service measurements, network performance indicators of interest to the operator; • Step 2 (2): Each intelligent node (NR) begins its mapping between actions, states, and rewards; the state is considered as the set of metrics associated with the RRC connection establishment procedures and other metrics associated with network performance; actions are understood as detection, mitigation, or information transmission procedures regarding the behavior of user equipment; cost functions calculate the reward from metrics of interest for the behavior strategy desired by the operator; and the initial mapping values ​​of each agent can be defined according to the network operator's objectives; • Step 3 (3): Each intelligent node (NR) selects a set of actions according to the observed state, note that no action can also be considered as an action; • Step 4 (4): Each intelligent node (NR) executes the set of actions selected in Step 3 (3); • Step 5 (5): Each intelligent node (NR) uses the associated metrics and weights to calculate the reward for the action implemented in Step 3 (3), such as, in a specific implementation, using Equation 1; • Step 5B (5B): Each intelligent node (NR) verifies whether it should report information to be shared with other nodes (NR), thus proceeding to Step (6A), or whether information sharing is not necessary, thus proceeding to Step 7 (7), if this information is not, for example, relevant to what other nodes have learned; the information includes, for example, decisions made, associated rewards, and measurements of the impact of decisions made on network performance; information sharing can be triggered by events or occurrences that indicate the need to update the network's response to anomalous behavior; and the number of nodes (NR) associated with a given node. Petition 870240111297, dated 12 / 30 / 2024, page 19 / 57 / 16: The central server (CS) must be defined according to the network objectives to avoid signaling overload and allow policy changes at the most appropriate time; • Step 6A (6A): Each intelligent node (NR) verifies whether it should report the information to be shared with the nodes (NR) to a central server (SC), proceeding to Step 6C (6C), or whether communication between the nodes (NR) is carried out directly, proceeding to Step 6B (6B); • Steps 6B (6B) and 6C (6C): Each intelligent node (NR) receives information about actions, values ​​calculated from cost functions, measurements of the impact of decisions taken on network performance, this information is shared by other intelligent nodes (NR) in a Step 6B (6B), or by the central server (CS) in a Step 6C (6C); • Step 7 (7): Each node (NR) equipped with intelligence updates its policy to improve its decision-making process in accordance with the network objectives; and • Step 8 (8): It is checked whether any stopping criterion of the proposed method has been met, and if so, the action of the proposed method is terminated; and if the stopping criterion has not been met, each node (NR) with intelligence determines the next action based on the updated policy, that is, the cycle returns to Step 3 (3).

[033] Figure 5 shows an illustrative and non-limiting example of applying the distributed version of the proposed method in an Open RAN network, using different components of the Open RAN network as elements used in the method. In this scenario, the Near-RT RIC acts as an intelligent node (NR). The O1 interface performs data sharing between the intelligent nodes (NR), while the E2 interface forwards communication information between the user equipment and the CU / DU, the Centralized and Distributed Units, of the site under the intelligent control of the Near-RT RIC. In the centralized version of the method, the Non-RT RIC operates as a central server (SC), distributing information through the other components of the SMO, or Service Management and Orchestration. Petition 870240111297, dated 12 / 30 / 2024, page 20 / 57 / 16

[034] REFERÊNCIAS BIBLIOGRÁFICAS • C. Benzai’d, F. M. Hossain, T Taleb, P. M. Gómez and M. Dieudonne, A Federated Continual Learning Framework for Sustainable Network Anomaly Detection in O-RAN, 2024 IEEE Wireless Communications and Networking Conference (WCNC), Dubai, United Arab Emirates, 2024, pp. 1-6, doi: 10.1109 / WCNC57260.2024.10570951. • D. Attanayaka, P. Porambage, M. Liyanage and M. Ylianttila, Peer-to-Peer Federated Learning Based Anomaly Detection for Open Radio Access Networks, ICC 2023 - IEEE International Conference on Communications, Rome, Italy, 2023, pp. 5464-5470, doi: 10.1109 / ICC45041.2023.10278993. • ANG, Zhongyun; HU, Haiyang. Network intrusion detection method based on federated learning. HANGZHOU DIANZI UNIVERSITY; SHANGYU SCIENCE AND ENGINEERING RESEARCH INSTITUTE CO. LTD. Patent application no. CN202310883184A, 18 Jul. 2023. Published as CN116708009A, 05 Sep. 2023. Classifications: IPC: G06N3 / 0442; G06N3 / 098; H04L9 / 40. CPC: G06N3 / 0442 (CN); G06N3 / 098 (CN); H04L63 / 1416 (CN); H04L63 / 1425 (CN); H04L9 / 40 (CN); Y02D30 / 50 (EP). • BAI, Lingling; NING, Zhenhu; XUE, Fei; YANG, Yongli; ZHANG, Shiqiang. DDoS attack detection based on hidden Markov and Q-learning cooperation. BEIJING UNIVERSITY OF TECHNOLOGY. Patent application no. CN201910118547A, February 16, 2019. Published as CN109639515A, April 16, 2019. Classifications: IPC: H04L12 / 24; H04L12 / 26; H04L29 / 06; H04L29 / 08. CPC: H04L41 / 0631 (CN); H04L41 / 142 (CN); H04L41 / 145 (CN); H04L41 / 147 (CN); H04L43 / 16 (CN); H04L63 / 1408 (CN); H04L63 / 1458 (CN); H04L67 / 14 (CN). Petition 870240111297, dated 12 / 30 / 2024, page 21 / 57

Claims

1 / 5 CLAIMS 1) METHOD FOR DETECTION AND MITIGATION OF DISTRIBUTED DENIAL OF SERVICE ATTACKS BASED ON REAL-TIME DISTRIBUTED LEARNING, this method for detection and mitigation of distributed denial of service attacks in mobile communication networks, is characterized by comprising: • Implementing multiple reinforcement learning agents in network nodes (NR), where each agent is a node (NR) endowed with intelligence responsible for making decisions based on a policy, the policy being the mapping between actions, states and rewards; • Initializing the policies of all nodes (NR) endowed with intelligence according to the objectives of the network operator; • Executing a sequential and coordinated set of actions comprising: o Detecting whether an initial access request is legitimate (AN) or a distributed denial of service attack (AA); o Selecting and executing an appropriate mitigation action if an attack (AA) is detected;and selectively transfer knowledge to other intelligent nodes (NRs); • Calculate a cost-based reward that considers initial access metrics, RAN metrics, and other parameters of the network of interest; • Iteratively update the policies of intelligent nodes (NRs) based on calculated rewards, observed states, and shared information. 2) METHOD FOR DETECTING AND MITIGATING DISTRIBUTED DENIAL OF SERVICE ATTACKS BASED ON REAL-TIME DISTRIBUTED LEARNING, according to claim 1, characterized by comprising a central server (CS) that connects to a number of intelligent nodes (NRs), such that: • The number of nodes (NRs) connected to a given central server (CS) is defined according to the network objectives; Petition 870240111297, dated 12 / 30 / 2024, page 22 / 57 2 / 5 • The central server (CS) distributes parameters of the intelligent nodes (NRs) under its coverage; • The central server (CS) updates the agents' policies based on the shared information. 3) METHOD FOR DETECTION AND MITIGATION OF DISTRIBUTED DENIAL OF SERVICE ATTACKS BASED ON REAL-TIME DISTRIBUTED LEARNING, according to claims 1 and 2, characterized in that the method comprises the following Steps: • Step 1 (1): Implementation of multiple reinforcement learning agents in network nodes (NR), with the configuration parameters of the proposed method being defined according to the network objectives; • Step 2 (2): Each intelligent node (NR) starts its mapping between actions, states and rewards; • Step 3 (3): Each intelligent node (NR) selects a set of actions according to the observed state; • Step 4 (4): Each intelligent node (NR) executes the set of actions selected in Step 3 (3); • Step 5 (5): Each intelligent node (NR) uses the associated metrics and weights to calculate the reward for the action implemented in Step 3 (3);• Step 5B (5B): Each intelligent node (NR) verifies whether it should report information to be shared with other nodes (NR), thus proceeding to Step (6A), or whether information sharing is not necessary, thus proceeding to Step 7 (7); • Step 6A (6A): Each intelligent node (NR) verifies whether it should report information to be shared with the nodes (NR) to a central server (SC), thus proceeding to Step 6C (6C), or whether communication between nodes (NR) is carried out directly, thus proceeding to Step 6B (6B); • Steps 6B (6B) and 6C (6C): Each intelligent node (NR) receives information about actions, values ​​calculated from cost functions, measurements of the impact of decisions taken on network performance, these Petition 870240111297, of 12 / 30 / 2024, p. 23 / 57 3 / 5 information shared by other intelligence-equipped nodes (NR) in a Step 6B (6B), or by the central server (SC) in a Step 6C (6C);• Step 7 (7): Each node (NR) equipped with intelligence updates its policy to improve its decision-making process in accordance with the network objectives; and • Step 8 (8): It is verified whether any stopping criterion of the proposed method has been met, and if so, the action of the proposed method is terminated; and if the stopping criterion has not been met, each node (NR) with intelligence determines the next action based on the updated policy, returning to Step 3 (3).; 4) A METHOD FOR DETECTING AND MITIGATING DISTRIBUTED DENIAL-OF-SERVICE ATTACKS BASED ON REAL-TIME DISTRIBUTED LEARNING, according to claims 1 and 3, characterized in that the reward calculation cost function comprises: • a set of parameters associated with network metrics; and • associated weights that are adjusted according to the network operator's objectives, with each weight associated with a parameter. 5) A METHOD FOR DETECTING AND MITIGATING DISTRIBUTED DENIAL-OF-SERVICE ATTACKS BASED ON REAL-TIME DISTRIBUTED LEARNING, according to claim 4, characterized in that the weights are updated according to the network objectives. 6) METHOD FOR DETECTING AND MITIGATING DISTRIBUTED DENIAL-OF-SERVICE ATTACKS BASED ON REAL-TIME DISTRIBUTED LEARNING, according to claims 1 and 3, characterized in that the RAN metrics used comprise: • Metrics associated with RRC connection establishment procedures, including RRC Setup, RRC Connection, RRC Reconfiguration, and RRC Reestablishment; • Network performance metrics, including latency and data transfer rate. Petition 870240111297, dated 12 / 30 / 2024, p. 24 / 57 4 / 5 7) METHOD FOR DETECTION AND MITIGATION OF DISTRIBUTED DENIAL OF SERVICE ATTACKS BASED ON REAL-TIME DISTRIBUTED LEARNING, according to claim 3, characterized in that, in Step 2 (2), the state is considered as the set of metrics associated with the RRC connection establishment procedures and metrics associated with network performance; actions are understood as detection, mitigation, or information transmission procedures relating to the behavior of user equipment; cost functions calculate the reward from metrics of interest for the behavior strategy desired by the operator; and the initial mapping values ​​of each agent are defined according to the objectives of the network operator. 8) METHOD FOR DETECTION AND MITIGATION OF DISTRIBUTED DENIAL OF SERVICE ATTACKS BASED ON REAL-TIME DISTRIBUTED LEARNING, according to claim 3, characterized by, in Step 1 (1), using as configuration parameters the total number of simultaneous connections, the total number of connection attempts, cost function parameters, quality of service measurements, and network performance indicators of interest to the operator. 9) METHOD FOR DETECTION AND MITIGATION OF DISTRIBUTED DENIAL OF SERVICE ATTACKS BASED ON REAL-TIME DISTRIBUTED LEARNING, according to claim 3, characterized in that, in Step 5B (5B), the information to be shared comprises decisions taken, associated rewards and measurements of the impact of the decisions taken on network performance; and the sharing of information being triggered by events or occurrences that indicate the need to update the network response to anomalous behavior. 10) METHOD FOR DETECTION AND MITIGATION OF DISTRIBUTED DENIAL OF SERVICE ATTACKS BASED ON REAL-TIME DISTRIBUTED LEARNING, according to claim 1, characterized by the method being implemented in an Open RAN network, the Near-RT RIC acting as an intelligent node (NR), the O1 interface performing data sharing between intelligent nodes (NR), and the E2 interface forwarding communication information between user equipment and the Centralized and Distributed Units of the site under the control of the Near-RT RIC. 11) METHOD FOR DETECTING AND MITIGATING DISTRIBUTED DENIAL-OF-SERVICE ATTACKS BASED ON REAL-TIME DISTRIBUTED LEARNING, according to claims 2 and 9, characterized in that the Non-RT RIC operates as a central server (CS), distributing information through the other components of the Service Management and Orchestration. Petition 870240111297, dated 12 / 30 / 2024, p. 26 / 57

Images