IMS network defense method and device

Abstract

The invention discloses an IMS (IP Multimedia Subsystem) network defense method and device, and the method comprises the steps: obtaining a current external network state and an internal network state of an IMS network according to user request information after receiving the user request information, and obtaining the user basic information of a current user; querying historical behaviors of the current user to obtain historical behavior information; constructing an external network state set S1 according to the external network state, constructing an internal network state set S2 according to the internal network state of the IMS network, and constructing a historical behavior state set S3 according to the historical behavior information, thereby obtaining an agent decision state set S, S = {S1, S2, S3}; converting the agent decision state set S into a corresponding agent decision state vector; the agent decision state vector is transmitted to a preset DRL model for decision making to obtain a decision vector, and the decision vector is analyzed to obtain an agent strategy; and performing corresponding operation on the current user according to the intelligent agent strategy. Compared with the prior art, the method is high in defense efficiency, low in false alarm rate and rapid in response.

Description

Technical Field

[0001] The present invention relates to the field of communications, and in particular to security defense of an IMS network. Background Art

[0002] As the pace of digital transformation accelerates, the role of IP Multimedia Subsystem (IMS) in network communications is becoming increasingly prominent. IMS not only supports traditional voice calls, but also carries a variety of multimedia services such as video conferencing and instant messaging, becoming an important part of the next-generation communication network. However, with the continuous development of network attack technology, the security issues of IMS have gradually surfaced, and traditional network defense strategies seem to be powerless in the face of these new threats.

[0003] Traditional network defense mechanisms mainly include firewalls, intrusion detection systems (IDS), intrusion prevention systems (IPS), and anti-virus software. These tools and technologies are designed to prevent unauthorized access and detect and respond to potential security threats. However, as network attack methods become increasingly complex and diverse, these traditional defense measures have gradually exposed their limitations. Although firewalls can filter data packets in and out of the network according to preset rules, their static rules are difficult to cope with the dynamically changing threat environment; although intrusion detection / prevention systems (IDS / IPS) can monitor abnormal behaviors in network traffic in real time and respond, they are less effective against unknown threats or advanced persistent threats (APT); anti-virus software is mainly used to prevent, detect, and remove malware, but when facing zero-day attacks or mutant viruses, its update speed often cannot keep up with the speed of virus mutation.

[0004] As a core part of the next generation communication network, IMS's security issues cannot be ignored. In recent years, IMS attacks have been common. For example, China Telecom once had an IMS network failure, which affected VoLTE services. Although the specific cause is unknown, similar problems show that IMS networks are vulnerable. In addition, in some cases, attackers use VoIP services to make harassing calls or conduct social engineering attacks. As one of the core technologies of VoIP, IMS also faces such risks. These cases not only pose a threat to personal privacy, but may also affect national security and social stability.

[0005] For IMS networks, although traditional network defense technologies have played a protective role to a certain extent, their own limitations are becoming increasingly obvious. As the first line of defense at the network boundary, the firewall's main function is to establish a barrier between networks, filter data packets entering and leaving the network, and prevent unauthorized access to the internal network. The firewall determines which data packets can be released and which should be intercepted or discarded based on the preset security rule set. These rules can be defined based on multiple dimensions such as source address, destination address, port number, protocol type, etc. However, firewalls rely on preset security rules, which are usually static and cannot respond to emerging threats in a timely manner. For example, for unknown zero-day vulnerabilities, the firewall may not be able to identify and block related attacks. In addition, attackers can bypass the firewall's protection in many ways, such as using encrypted channels to transmit malicious data and using legitimate ports to attack. Once attackers find a way to bypass the firewall, the security of the internal network will be seriously threatened. In addition, in order to give full play to the role of the firewall, the rule set needs to be carefully configured, which not only increases management costs, but also easily creates security vulnerabilities due to configuration errors.

[0006] Intrusion detection systems (IDS) are mainly used to monitor network traffic, detect abnormal behavior, and issue alarms when potential attack activities are found. According to different detection methods, IDS can be divided into signature-based IDS and anomaly-based IDS. The former identifies threats by matching known attack patterns, while the latter detects abnormal behavior by analyzing the statistical characteristics of network traffic. Although IDS has shown superior capabilities in real-time monitoring and alarm, its high false alarm rate is very prominent. Signature-based IDS relies on pre-defined attack patterns. If these patterns are too broad, they may cause a large number of false alarms. Anomaly-based IDS may trigger false alarms frequently in the early stage because it needs to establish a normal behavior baseline. In addition, IDS usually analyzes network traffic after the fact, which means that even if an attack is detected, it cannot prevent its ongoing damage. With the widespread application of encryption technology, more and more data transmissions are encrypted. However, the concealment of encrypted traffic makes it difficult for IDS to effectively detect it, thereby reducing its defense effect.

[0007] Therefore, there is an urgent need for an IMS network defense method and device that can solve the above problems. Summary of the invention

[0008] The purpose of the present invention is to provide an IMS network defense method with high defense efficiency, low false alarm rate and fast response.

[0009] In order to achieve the above-mentioned purpose, the present invention provides an IMS network defense method, comprising: after receiving user request information, obtaining the current external network state and the internal network state of the IMS network according to the user request information, and obtaining the basic user information of the current user; querying the historical behavior of the current user to obtain the historical behavior information; constructing an external network state set S1 according to the external network state, constructing an internal network state set S2 according to the internal network state of the IMS network, and constructing a historical behavior state set S3 according to the historical behavior information; constructing an intelligent agent decision state set S according to the external network state set S1, the internal network state set S2 and the historical behavior state set S3, S = {S1, S2, S3}; converting the intelligent agent decision state set S into a corresponding intelligent agent decision state vector; transmitting the intelligent agent decision state vector to a preset DRL model for decision-making to obtain a decision vector, parsing the decision vector to obtain an intelligent agent strategy; performing corresponding operations on the current user according to the intelligent agent strategy.

[0010] Preferably, before querying the historical behavior of the current user to obtain the historical behavior information, the process also includes: querying whether the current user is in a preset blacklist, and if so, refusing service; if not, querying the historical behavior of the current user to obtain the historical behavior information.

[0011] Preferably, converting the agent decision state set S into the corresponding agent decision state vector specifically includes: converting the agent decision state set S into a state vector, and normalizing and regularizing the state vector to obtain the corresponding agent decision state vector.

[0012] Preferably, the agent strategy A=[a 1 ,a 2 ] T , where a1 is the value of the agent strategy and a2 is the time to execute the agent strategy.

[0013] Specifically, the agent strategy includes a combination of two or more of requesting release, adding to a blacklist, rejecting this service, closing a port, restarting the IMS network, and disconnecting from the external network.

[0014] Preferably, the current external network status includes connectivity, latency, bandwidth usage, and whether there are any potential network attacks or abnormal traffic; the internal network status of the IMS network includes the network element status of each network element in the IMS core network, and the network element status includes whether the current network element is busy or abnormal; the basic user information includes user account information and registration information.

[0015] Preferably, the IMS network defense method also includes a DRL model training method: setting a DRL initial model, designing an overall environment for training a reinforcement learning network defense system, and training the DRL initial model in the overall environment; the overall environment for designing and training a reinforcement learning network defense system includes: setting simulated user parameters and setting network mutation parameters; training the DRL initial model specifically includes: initializing a simulated user according to the simulated user parameters and issuing corresponding simulated user request information according to the type of the simulated user, notifying the environment of the user type of the simulated user and the request type of the simulated user request information, simulating the current external network state and internal network state with simulated mutation parameters and notifying the environment of the simulated external network state and internal network state; accepting the simulated user request information, obtaining the agent decision state vector of the current overall environment after receiving the simulated user request information, and transmitting the agent decision state vector to the DRL model for decision-making to obtain a decision vector, parsing the decision vector to obtain an agent strategy, the environment itself scores the agent strategy to obtain a reward, and updating the agent parameters of the DRL model.

[0016] Specifically, the simulated user parameters include a mutation probability, a total number of simulated users, a malicious user ratio, and an attack intensity; initializing a simulated user according to the simulated user parameters and issuing corresponding simulated user request information according to the type of the simulated user specifically includes: setting a number of simulated users according to the total number of simulated users, setting a number of malicious users among the simulated users according to the malicious user ratio, the simulated users are divided into normal users and malicious users according to user types, and corresponding simulated user request information is continuously initiated to the IMS network at a preset frequency according to the user types of the simulated users, the malicious user randomly selects a malicious attack to the IMS network with a preset attack intensity, and before each sending of the simulated user request information, the malicious user and the normal user are converted to each other at the mutation probability.

[0017] Specifically, the network mutation parameters include external network fluctuation probability and internal network mutation probability. When the simulated user initiates a simulated user request, the current external network state and internal network state are simulated according to the external network fluctuation probability and the internal network probability.

[0018] Specifically, training the DRL initial model includes: accepting the simulated user request information, obtaining the agent decision state vector of the current overall environment based on the user request in the simulated user request information, and transmitting it to the DRL model for decision-making to obtain a decision vector, parsing the decision vector to obtain the agent strategy, the environment itself scores the agent strategy to obtain rewards, and updating the agent parameters of the DRL model.

[0019] Preferably, the environment itself scores the agent strategy to obtain rewards specifically by: querying the simulated user information to obtain the simulated user's real identity and supply type, querying the IMS network status and disaster situation to obtain the current external network status and the IMS network's internal network status, and giving rewards based on the simulated user information, the current external network status, the IMS network's internal network status and the agent's strategy.

[0020] The present invention also provides an IMS network defense device, including a processor and a memory, wherein the memory stores at least one executable instruction, and the at least one executable instruction enables the processor to cooperate with the signal transceiver unit to perform the operation of the IMS network defense method as described above.

[0021] Compared with existing technologies, this technology applies deep reinforcement learning (DRL) to the field of network security. DRL is an artificial intelligence technology that learns the best action strategy by letting the "agent" interact with the environment without explicit programming. This adaptive learning method enables DRL to perform well in the face of ever-changing network threats. Unlike traditional rule-based methods, the DRL model can automatically adjust the defense strategy according to changes in the network status and achieve rapid response to new threats. Through continuous trial and error learning, DRL is able to optimize its decision-making process and improve defense efficiency. Based on historical data and real-time information, DRL can predict potential attack behaviors and take preventive measures in advance, reducing the need for human intervention, speeding up threat response, and reducing false alarm rates. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flow chart of the IMS network defense method of the present invention.

[0023] Figure 2 It is a flow chart of the training method of the DRL model of the present invention. DETAILED DESCRIPTION

[0024] In order to explain the technical content, structural features, achieved objectives and effects of the present invention in detail, the following is a detailed description in conjunction with the implementation methods and the accompanying drawings.

[0025] refer to Figure 1 The present invention discloses an IMS network defense method, including steps S1 to S10.

[0026] S1, receiving user request information.

[0027] When an unknown user attempts to access a network service, the system first receives the request (user request information) and immediately parses the request category, including but not limited to Register, Message, Option, etc., while extracting basic user information from the request. This initial step provides the necessary context for subsequent decisions.

[0028] S2, after receiving the user request information, obtain the current external network status and the internal network status of the IMS network according to the user request information.

[0029] Specifically, before processing user request information, the system will also pre-check the status of the external network (current external network status). The system builds an external network status set S1 by monitoring the network's connectivity, latency, bandwidth usage, and whether there are any potential network attacks or abnormal traffic. This stage of work ensures that the defense system can fully grasp the information of the external environment and provide a basis for subsequent decision-making.

[0030] Next, the system actively queries the status of each network element in the IMS core network (internal network status of the IMS network) to determine whether any network element is busy or abnormal. Through this query, the system constructs the internal network status set S2. This step helps the system understand the health status of the internal network and make more reasonable security decisions.

[0031] S3, obtain the basic information of the current user.

[0032] Specifically, the system then obtains the user's basic information, which includes the user's account information, registration information, etc. This is used as a query database DB2 for the user's historical behavior, whether it is on the blacklist, etc. This step provides data support for subsequent user behavior analysis.

[0033] Among them, step S2 and step S3 can be performed simultaneously or sequentially, and the execution order thereof can be interchanged.

[0034] S4, query whether the current user is in a preset blacklist.

[0035] Specifically, the system queries the blacklist database DB1 to determine whether the user currently requesting the service (current user) has been marked as a malicious user. If the user requesting the service does exist in the blacklist database, the system will immediately refuse to provide the service to him and exit the processing flow. This mechanism effectively prevents known malicious users from further infringing on the network.

[0036] S5: If the current user is in the preset blacklist, the service is denied.

[0037] S6: If the current user is not in the preset blacklist, query the historical behavior of the current user to obtain historical behavior information.

[0038] S7, construct an external network state set S1 according to the external network state, construct an internal network state set S2 according to the internal network state of the IMS network, and construct a historical behavior state set S3 according to the historical behavior information, so as to obtain an intelligent agent decision state set S, S = {S1, S2, S3}.

[0039] Specifically, for users who are not in the blacklist, the system continues to query the user's historical behavior data table DB2 to construct the user's historical behavior state set S3. For users who access the IMS network for the first time, the system will create a new data item for them and record this behavior as their historical behavior to construct the user's historical behavior state set S3. In this way, even new users can be included in the system's monitoring scope.

[0040] After obtaining S1, S2 and S3, the system constructs the agent decision state set S based on the external network state set S1, internal network state set S2 and historical behavior state set S3. This state set integrates all relevant information and lays the foundation for subsequent intelligent decision-making.

[0041] Among them, the current external network status includes connectivity, latency, bandwidth usage, and whether there are any potential network attacks or abnormal traffic; the internal network status of the IMS network includes the network element status of each network element in the IMS core network, and the network element status includes whether the current network element is busy or abnormal; the user basic information includes user account information and registration information.

[0042] S8, convert the agent decision state set S into the corresponding agent decision state vector.

[0043] Among them, converting the intelligent agent decision state set S into the corresponding intelligent agent decision state vector specifically includes: converting the intelligent agent decision state set S into a state vector, and normalizing and regularizing the state vector to obtain the corresponding intelligent agent decision state vector.

[0044] S9, transmitting the agent decision state vector to a preset DRL model for decision-making to obtain a decision vector, and parsing the decision vector to obtain the agent strategy.

[0045] Wherein, the agent strategy A=[a 1 ,a 2 ] T, where a1 is the value of the agent strategy and a2 is the time to execute the agent strategy. After obtaining a1 and a2, they are rounded down and mapped to the strategy number selected by the agent. a2 is the strategy execution time. If the agent selects a long-term strategy such as joining the blacklist, a2 is the time to execute the strategy.

[0046] In this embodiment, a comparison table of decision vectors and meanings is shown below:

[0047]

[0048] S10, performing corresponding operations on the current user according to the agent strategy.

[0049] The execution operation includes a combination of two or more of requesting release, adding to a blacklist, rejecting this service, closing a port, restarting the IMS network, and disconnecting from an external network.

[0050] Among them, the IMS network defense method also includes a DRL model training method: setting a DRL initial model, designing an overall environment for training a reinforcement learning network defense system, and training the DRL initial model in the overall environment.

[0051] The overall environment includes setting simulated user parameters and setting network mutation parameters. The simulated user parameters include mutation probability ε, total number of simulated users n, malicious user ratio σ and attack intensity λ. Among them, a number of simulated users are set according to the total number of simulated users n, and a number of malicious users are set among the simulated users according to the malicious user ratio σ. The simulated users are divided into normal users and malicious users according to user types. During training, the simulated user is used to continuously initiate simulated user request information to the IMS network at a preset frequency, and before each sending of the simulated user request information, the malicious user and the normal user are converted to each other at the mutation probability ε. The simulated user request information includes user request, user identity and supply type. The malicious user randomly selects a malicious attack to the IMS with a preset attack intensity, and the IMS network performs corresponding mutations with the network mutation parameters.

[0052] refer to Figure 2 Training the DRL initial model in the overall environment specifically includes steps S31 to S36 and steps S21 to S25.

[0053] Training begins.

[0054] S31, setting simulation user parameters: mutation probability ε, total number of simulated users n, malicious user ratio σ and attack intensity λ.

[0055] Among them, network variation parameters can also be set. Network variation parameters include fluctuation parameters of external networks and random variation parameters of internal networks. Specifically, in the process of simulating user-initiated requests, network request transmission may randomly encounter fluctuations. These fluctuations include increased latency, reduced bandwidth, or disconnected connections, with the aim of simulating the instability of networks in the real world. In this way, the system allows the agent to learn as much as possible about how to make correct decisions in various abnormal situations. In addition to fluctuations in network transmission, the IMS network itself also randomly experiences anomalies with a small probability, such as service interruptions and performance degradation. These anomalies require timely detection and processing by the agent to ensure the continuity and quality of network services. In this way, the trained agent has the ability to maintain the internal IMS network, enabling it to respond quickly to emergencies.

[0056] S32, initializing a simulated user.

[0057] Specifically, an initial state (normal or malicious) is assigned to each user, and the malicious users are determined according to the set malicious user ratio (σ).

[0058] S33, generate a random number p.

[0059] Before each user initiates a request, the system generates a random number p. This random number will be used to determine whether the user status has changed, that is, whether it changes from a normal user to a malicious user, or from a malicious user to a normal user.

[0060] S34, determine whether the random number p is less than the mutation probability ε, if so, mutate the current user (convert the user type) in S34a, if the current user is a normal user, convert it to a malicious user, if the current user is a malicious user, convert it to a normal user. If the random number p is not less than the mutation probability ε, the current user works according to its user type.

[0061] This scheme makes it possible for users to become malicious users even if they are initially set as normal users, and vice versa. This mechanism simulates the uncertainty and dynamic changes of user behavior in the real world.

[0062] S35: If the current user is a normal user, a normal request is sent, and the user type and request type are notified to the environment.

[0063] Specifically, for those users who remain in normal status, they will continue to operate in the preset manner, such as performing registration, sending messages, or configuring options and other network activities that comply with regulations.

[0064] S36, if the current user is a malicious user, a malicious attack request is randomly selected according to the attack intensity, and the user type and attack method are notified to the environment.

[0065] Specifically, the malicious user randomly selects to send malicious attacks according to the set attack intensity. The attack intensity determines the frequency and intensity of the malicious user's attack.

[0066] Once a simulated user initiates a request, regardless of its nature, the simulated user system will inform other components in the simulated environment of its current state (normal or malicious) and the specific request type or attack method. This feedback mechanism helps the central control system evaluate the performance of the agent and adjust the training strategy accordingly. In this way, the agent can learn how to better respond to future threats, thereby improving the overall security of the network.

[0067] S21, receiving the simulated user request information. The simulated user system simulates a user (may be a normal user or a malicious user) to initiate a network request. These requests simulate the activities of users in a real network environment, including legitimate service requests and potential malicious attacks.

[0068] Specifically, in the process of simulating user-initiated requests, network request transmission may randomly encounter fluctuations. These fluctuations include increased latency, reduced bandwidth, or disconnected connections, with the goal of simulating the instability of networks in the real world. In this way, the system allows the agent to learn as much as possible about how to make correct decisions in various abnormal situations. In addition to fluctuations in network transmission, the IMS network itself also randomly experiences anomalies with a small probability, such as service interruptions and performance degradation. These anomalies require timely detection and processing by the agent to ensure the continuity and quality of network services. In this way, the trained agent has the ability to maintain the internal IMS network, enabling it to respond quickly to emergencies.

[0069] S22, after receiving the simulated user request information, obtaining the agent decision state vector of the current overall environment. Specifically, the step of obtaining the agent decision state vector of the current overall environment is shown in steps S2 to S8.

[0070] S23, the agent decision state vector is transmitted to the DRL model for decision-making to obtain a decision vector, and the decision vector is parsed to obtain the agent strategy. The goal of the agent is to protect the network from attacks while minimizing the impact on normal services.

[0071] Specifically, step S23 includes steps S231 to S235.

[0072] S231, generate random numbers: use a random number seed to generate a random number seed between 0 and 1 as the simulation agent exploration probability;

[0073] S232, obtain exploration probability: obtain the probability of agent exploration under the current number of steps from the function f(k). This probability should be consistent with reality, that is, the function f(x) is a monotonically non-increasing function with a value range of 0-1. The reference function is given:

[0074] f(x)=1-e k-episode

[0075] S233, random exploration: If the random number p is less than the current exploration probability, the agent randomly moves to a 1 ∈[0, 6], a 2 ∈[T min , T max ] to generate a decision vector A, where T min is the minimum duration of the strategy, T max is the maximum duration of the strategy. Otherwise, the process goes to S234.

[0076] S234, preprocessing the state set and sending it to the agent for decision: The data preprocessing unit converts the agent decision state set S into a state vector, and performs preprocessing operations such as normalization and regularization on the state vector S to obtain the preprocessed state vector S′. This step ensures that all input features are on the same scale to avoid some features dominating the decision process due to a large numerical range. The agent makes a decision based on the preprocessed state vector S′ and obtains the decision vector A.

[0077] S235, Analyze Agent Strategy: The system analyzes the decision vector output by the agent.

[0078] A=[a 1 , a 2 ] T

[0079] where a 1 For strategy, a 1 The value range is [0, 6], and after rounding down, it is mapped to the strategy number selected by the agent. 2 is the strategy execution time. If the agent chooses a long-term strategy such as joining the blacklist, a 2 The time to execute the strategy.

[0080] S24, the environment itself scores the agent's strategy and obtains rewards.

[0081] S25, updating the agent parameters of the DRL model.

[0082] Specifically, step S25 includes: the agent uses the DDPG algorithm to update the neural network parameters and learn the experience gained from this exploration:

[0083] TD algorithm updates the value network:

[0084]

[0085] The gradient ascent algorithm calculates the policy network gradient:

[0086]

[0087] Update network parameters:

[0088]

[0089] in, θ is the policy network and value network parameters, and α is the learning rate. The value network scores the current action selected by the agent in the current state, and the policy network is the neural network ultimately used for agent decision-making. Its input is the current environment state, and its output is the best action.

[0090] Among them, the environment itself scores the agent strategy to obtain rewards as follows: query the simulated user information to obtain the real identity and supply type of the simulated user, query the IMS network status and disaster situation to obtain the current external network status and the internal network status of the IMS network, and give rewards based on the simulated user information, the current external network status, the internal network status of the IMS network and the agent strategy.

[0091] Among them, query the real identity and attack type of the simulated user: the intelligent agent training center queries whether the user is a malicious user, and the type of attack they launched (such as DDoS attack, SQL injection, etc.). And give a score based on the response strategy given by the intelligent agent. Query the IMS network status and disaster situation: the intelligent agent training center queries whether the user is a malicious user, and the type of attack they launched (such as DDoS attack, SQL injection, etc.). This information is used to evaluate whether the response strategy given by the intelligent agent is appropriate, and give a corresponding score based on the response strategy of the intelligent agent. Give the intelligent agent a comprehensive reward r for this decision: the intelligent agent training center gives the intelligent agent a comprehensive reward r for this decision based on the above evaluation results. The reward mechanism can motivate the intelligent agent to continuously optimize its decision-making strategy and improve the overall performance of the simulated environment. The reward mechanism is designed to promote the intelligent agent to effectively identify and resist various forms of network attacks while ensuring the normal operation of network services.

[0092] The present invention also discloses an IMS network defense device, including a processor and a memory, wherein the memory stores at least one executable instruction, and the at least one executable instruction enables the processor to cooperate with the signal transceiver unit to perform the operation of the IMS network defense method as described above.

[0093] The present invention applies deep reinforcement learning (DRL) to the field of network security. DRL is an artificial intelligence technology that allows an "agent" to interact with the environment to learn the best action strategy without explicit programming. This adaptive learning method enables DRL to perform well in the face of ever-changing network threats. Unlike traditional rule-based methods, the DRL model can automatically adjust the defense strategy according to changes in the network status and achieve rapid response to new threats. Through continuous trial and error learning, DRL can optimize its decision-making process and improve defense efficiency. Based on historical data and real-time information, DRL can predict potential attack behaviors and take preventive measures in advance. In addition, reducing the need for human intervention, speeding up threat response, and reducing false alarm rates are also major advantages of DRL.

[0094] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. Therefore, equivalent changes made according to the scope of the patent application of the present invention are still within the scope covered by the present invention.

Claims

1. An IMS network defense method, characterized in that: include: After receiving the user request information, obtain the current external network status and the internal network status of the IMS network according to the user request information, and obtain the basic user information of the current user; Query the historical behavior of the current user to obtain historical behavior information; According to the external network state, an external network state set S1 is constructed, according to the internal network state of the IMS network, an internal network state set S2 is constructed, and according to the historical behavior information, a historical behavior state set S3 is constructed; according to the external network state set S1, the internal network state set S2 and the historical behavior state set S3, an intelligent agent decision state set S is constructed, S = {S1, S2, S3}; Convert the agent decision state set S into the corresponding agent decision state vector; The agent decision state vector is transmitted to a preset DRL model for decision-making to obtain a decision vector, and the decision vector is analyzed to obtain an agent strategy; Perform corresponding operations on the current user according to the agent strategy.

2. The IMS network defense method according to claim 1, characterized in that: Before querying the historical behavior of the current user to obtain the historical behavior information, the method further includes: querying whether the current user is in a preset blacklist, and if so, refusing service; if not, querying the historical behavior of the current user to obtain the historical behavior information.

3. The IMS network defense method according to claim 1, characterized in that: The transformation of the agent decision state set S into the corresponding agent decision state vector specifically includes: The agent decision state set S is converted into a state vector, and the state vector is normalized and regularized to obtain the corresponding agent decision state vector.

4. The IMS network defense method according to claim 1, characterized in that: The agent strategy A = [a1, a2] T , where a1 is the value of the agent strategy and a2 is the time to execute the agent strategy.

5. The IMS network defense method according to claim 4, characterized in that: The agent strategy includes a combination of two or more of requesting release, adding to a blacklist, rejecting this service, closing ports, restarting the IMS network, and external network disconnection; the current external network status includes connectivity, latency, bandwidth usage, and whether there are any potential network attacks or abnormal traffic; the internal network status of the IMS network includes the network element status of each network element in the IMS core network, and the network element status includes whether the current network element is busy or abnormal; the user basic information includes user account information and registration information.

6. The IMS network defense method according to claim 1, characterized in that: The method also includes a DRL model training method: setting a DRL initial model, designing an overall environment for training a reinforcement learning network defense system, and training the DRL initial model in the overall environment; The overall environment of designing and training the reinforcement learning network defense system includes: setting simulation user parameters, setting network abnormality parameters; Training the DRL initial model specifically includes: initializing a simulated user according to the simulated user parameters and issuing corresponding simulated user request information according to the type of the simulated user, notifying the environment of the user type of the simulated user and the request type of the simulated user request information, simulating the current external network state and internal network state with simulated variation parameters and notifying the environment of the simulated external network state and internal network state; accepting the simulated user request information, obtaining the agent decision state vector of the current overall environment after receiving the simulated user request information, and transmitting the agent decision state vector to the DRL model for decision-making to obtain a decision vector, parsing the decision vector to obtain the agent strategy, the environment itself scores the agent strategy to obtain a reward, and updating the agent parameters of the DRL model.

7. The IMS network defense method according to claim 6, characterized in that: The simulated user parameters include mutation probability, total number of simulated users, malicious user ratio and attack strength; initializing the simulated user according to the simulated user parameters and issuing corresponding simulated user request information according to the type of the simulated user specifically includes: setting a number of simulated users according to the total number of simulated users, setting a number of malicious users among the simulated users according to the malicious user ratio, the simulated users are divided into normal users and malicious users according to user types, and corresponding simulated user request information is continuously initiated to the IMS network at a preset frequency according to the user types of the simulated users, the malicious users randomly select a malicious attack to the IMS network with a preset attack strength, and before each sending of the simulated user request information, the malicious users and normal users are converted to each other at the mutation probability.

8. The IMS network defense method according to claim 6, characterized in that: The network mutation parameters include external network fluctuation probability and internal network mutation probability. When the simulated user initiates a simulated user request, the current external network state and internal network state are simulated according to the external network fluctuation probability and the internal network probability.

9. The IMS network defense method according to claim 6, characterized in that: The environment itself scores the agent's strategy to obtain rewards as follows: query the simulated user information to obtain the simulated user's real identity and supply type, query the IMS network status and disaster situation to obtain the current external network status and the IMS network's internal network status, and give rewards based on the simulated user information, the current external network status, the IMS network's internal network status and the agent's strategy.

10. An IMS network defense device, characterized in that: It includes a processor and a memory, wherein the memory stores at least one executable instruction, and the at least one executable instruction enables the processor to cooperate with the signal transceiver unit to perform the operation of the IMS network defense method according to any one of claims 1 to 9.

Images