Defense method and device for network environment, equipment, medium and program product
By adopting a network defense model with reinforcement learning mechanism in network defense, the operating state of the network environment is processed and the reward function is used to optimize the defense strategy, the problems of response lag and false alarms in the existing technology are solved, and efficient and accurate Cyberattack interception and rapid response.
Patent Information
- Application Number
- CN202411525269.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-30
- Publication Date
- 2024-12-03
AI Technical Summary
Existing network security protection methods lag in response to new and complex network attacks, which are prone to false alarms and missed reports, resulting in data loss or tampering, and require manual intervention, resulting in long response time of the system and it is difficult to quickly deal with rapid evolution. attack.
A network defense model based on reinforcement learning mechanism is adopted to generate defense actions by processing the operating state of the network environment, and the reward function is used to update the model to improve defense effect and response speed. The model processes the running state based on the current network defense model, generates initial defense actions, optimizes defense strategies through election mechanisms and reward functions, and dynamically adjusts defense strategies to deal with unknown attacks.
It improves the interception accuracy and response speed of network defense, reduces false alarms and missed reports, reduces the cost of human resources participation, realizes automated network security decisions and optimization, and improves the operation efficiency of the network environment.
Smart Images

Figure CN119071087A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of network security, and more specifically to a defense method, device, equipment, medium and program product for a network environment. Background Art
[0002] With the increasing popularity and dependence on the Internet, network security has become particularly important. During network data communication, it is easy to be invaded by unauthorized attack traffic, which can cause data loss or tampering.
[0003] Current network security protection methods mostly rely on rule matching and signature detection. This method often lags in response to new and complex network attacks. Moreover, rule-based security protection may generate a large number of false positives, wasting normal traffic processing, and may also cause attack traffic to be missed, resulting in data loss or tampering. In addition, the above methods require manual intervention, resulting in a long system response time and easy to miss rapidly evolving attacks. Summary of the invention
[0004] In view of the above problems, the present disclosure provides a defense method, apparatus, device, medium and program product for a network environment.
[0005] According to a first aspect of the present disclosure, a defense method for a network environment is provided, including: using a current network defense model to process an operating state corresponding to the network environment to obtain a defense action, wherein the defense action is used to defend against attack traffic in the network environment to obtain a new operating state, and the network defense model is constructed based on a reinforcement learning mechanism; processing a defense result obtained by executing the defense action based on a reward function to obtain reward information, wherein the reward information is used to characterize the attack traffic intercepted based on the defense action, and to characterize the degree of influence of executing the defense action on the operating state; updating the current network defense model based on the reward information to obtain a target network defense model; and processing a target operating state based on the target network defense model to obtain a target defense action, wherein the generation time of the target operating state is later than the generation time of the operating state.
[0006] According to an embodiment of the present disclosure, the current network defense model is used to process the operating status corresponding to the network environment to obtain a defense action, including: using the network defense model to process the network status of the current time period to obtain an initial defense action for the current time period; using an election mechanism to determine the defense action for the current time period from historical defense actions in historical time periods and the initial defense action for the current time period, wherein the moment of determining the historical defense action is earlier than the moment of determining the initial defense action.
[0007] According to an embodiment of the present disclosure, an election mechanism is utilized to determine a defense action for a current period from among historical defense actions in historical periods and initial defense actions in a current period, including: determining a first weight corresponding to the historical defense action based on defense attributes of the historical defense action; determining a second weight corresponding to the initial defense action based on defense attributes of the initial defense action; comparing the first weight with the second weight to obtain a first comparison result; and determining, based on the first comparison result, a defense action for the current period from among among the historical defense actions in historical periods and the initial defense action in the current period.
[0008] According to an embodiment of the present disclosure, the operating status and defense action are processed based on a reward function to obtain reward information, including: processing defense success information and defense failure information based on a reward function to obtain reward information, wherein the defense success information represents that the defense action successfully intercepts the attack traffic, and the defense failure information represents that the defense action fails to intercept the attack traffic, and the defense result includes the defense success information and the defense failure information.
[0009] According to an embodiment of the present disclosure, the defense success information includes at least one of the following: the amount of intercepted attack traffic, and the importance level corresponding to the intercepted attack traffic.
[0010] According to an embodiment of the present disclosure, processing defense success information and defense failure information based on a reward function includes: processing defense success information, defense failure information and a reference operating state based on a reward function, wherein the reference operating state is generated based on the successful interception of attack traffic by a defense action.
[0011] According to an embodiment of the present disclosure, a reward function is constructed based on an operating state reward sub-function; processing defense success information, defense failure information and a reference operating state based on the reward function includes: using an operating state reward sub-function to process a reference operating state and a preset state to obtain operating state sub-reward information, wherein the reward information is determined based on the operating state sub-reward information.
[0012] According to an embodiment of the present disclosure, the operating status includes at least one of the following: information flow receiving traffic and information flow sending traffic in network data transmission; the proportion of data packet loss in network data transmission; and bandwidth usage in network data transmission.
[0013] According to an embodiment of the present disclosure, the reward information is obtained by processing the operating state and the defense action based on the reward function, including: using the reward function to process the i-th defense action in the i-th time period and the defense result corresponding to the i-th defense action, to obtain the reward information of the i-th time period; wherein the reward information of the i-th time period is used to update the network defense model of the i-th time period to obtain the updated network defense model, and the updated network defense model is used to generate the target defense action for the i+1-th time period.
[0014] A second aspect of the present disclosure provides a method for training a network defense model, including: constructing a simulated network environment; using the network defense model to process and simulate the operating status of the network environment to obtain defense actions; processing the defense results obtained by executing the defense actions based on a reward function to obtain reward information; and training the network defense model based on the reward information to obtain a trained network defense model.
[0015] The third aspect of the present disclosure provides a defense device for a network environment, including: a first processing module, used to use the current network defense model to process the operating state corresponding to the network environment to obtain a defense action, wherein the defense action is used to defend against attack traffic in the network environment to obtain a new operating state, and the network defense model is constructed based on a reinforcement learning mechanism; a second processing module, used to process the defense result obtained by executing the defense action based on a reward function to obtain reward information, wherein the reward information is used to characterize the attack traffic intercepted based on the defense action, and to characterize the degree of influence of executing the defense action on the operating state; an update module, used to update the current network defense model based on the reward information to obtain a target network defense model; and a third processing module, used to process the target operating state based on the target network defense model to obtain a target defense action, wherein the generation time of the target operating state is later than the generation time of the operating state.
[0016] A fourth aspect of the present disclosure provides an electronic device, comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.
[0017] The fifth aspect of the present disclosure further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.
[0018] The sixth aspect of the present disclosure further provides a computer program product, including a computer program or instructions, which implement the steps of the above method when the above computer program or instructions are executed by a processor.
[0019] According to the embodiment of the present disclosure, the current network defense model is used to process the running state corresponding to the network environment to obtain a defense action; the defense result obtained by executing the defense action is processed based on the reward function to obtain the reward information. Since the reward information is used to characterize the attack traffic intercepted based on the defense action, and to characterize the degree of influence of executing the defense action on the running state, the current network defense model is updated based on the reward information, so that the obtained target network defense model can understand the interception effect of the defense action on the attack traffic based on the reward information, and can understand the degree of influence of the defense action on the running state of the network environment through the reward information. In this way, the target running state detected later is processed based on the target network defense model obtained after training, so that the obtained target defense action can intercept the supply traffic more accurately and efficiently, and reduce the degree of negative impact of the target defense action on the running state of the network environment, so as to at least partially solve the problems of low interception accuracy of defense technical means in the network environment and slow operation response time of the network environment, and then realize the automatic optimization of the network defense model according to multiple defense actions, reduce the human resource cost generated by human resources participating in network security decision-making, and achieve the technical effect of improving the defense response speed and accuracy, and improving the operation efficiency of the network environment. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The above contents and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:
[0021] Figure 1 The application scenario diagram of the network environment defense method, apparatus, device, medium and program product according to the embodiment of the present disclosure is schematically shown;
[0022] Figure 2 A flowchart of a defense method for a network environment according to an embodiment of the present disclosure is schematically shown;
[0023] Figure 3 A schematic diagram schematically shows a defense method for a network environment according to an embodiment of the present disclosure;
[0024] Figure 4 A flowchart of a method for training a network defense model according to an embodiment of the present disclosure is schematically shown;
[0025] Figure 5 A schematic diagram schematically illustrates the training and application of a network defense model according to an embodiment of the present disclosure;
[0026] Figure 6 A structural block diagram of a defense device for a network environment according to an embodiment of the present disclosure is schematically shown; and
[0027] Figure 7A block diagram of an electronic device suitable for implementing a defense method for a network environment according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION
[0028] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.
[0029] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0030] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
[0031] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0032] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.
[0033] In the technical solution of the present disclosure, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0034] In the scenario of using personal information to perform automated actions, the methods, devices, and systems provided by the embodiments of the present disclosure provide users with corresponding operation portals for users to choose to agree or reject the results of the automated actions; if the user chooses to reject, the expert action process is entered. The expression "automated action" here refers to the activity of automatically analyzing and evaluating a person's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through a computer program, and performing actions. The expression "expert action" here refers to the activity of people who specialize in a certain field of work, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.
[0035] An embodiment of the present disclosure provides a defense method for a network environment, which utilizes a current network defense model to process an operating state corresponding to the network environment to obtain a defense action, wherein the defense action is used to defend against attack traffic in the network environment to obtain a new operating state, and the network defense model is constructed based on a reinforcement learning mechanism; a defense result obtained by executing a defense action is processed based on a reward function to obtain reward information, wherein the reward information is used to characterize the attack traffic intercepted based on the defense action, and to characterize the degree of influence of executing the defense action on the operating state; the current network defense model is updated based on the reward information to obtain a target network defense model; and a target operating state is processed based on the target network defense model to obtain a target defense action, wherein the generation time of the target operating state is later than the generation time of the operating state.
[0036] Figure 1 The application scenario diagram of the defense method, apparatus, device, medium and program product for a network environment according to an embodiment of the present disclosure is schematically shown.
[0037] like Figure 1As shown, the application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.
[0038] The user can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).
[0039] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.
[0040] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.
[0041] It should be noted that the defense method for the network environment provided by the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the defense device for the network environment provided by the embodiment of the present disclosure can generally be set in the server 105. The defense method for the network environment provided by the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the defense device for the network environment provided by the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.
[0042] It should be understood that Figure 1The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0043] The following will be based on Figure 1 The scene described by Figure 2~Figure 6 The defense method for the network environment of the disclosed embodiment is described in detail.
[0044] Figure 2 The flowchart of the defense method for a network environment according to an embodiment of the present disclosure is schematically shown.
[0045] like Figure 2 As shown, the defense method for the network environment of this embodiment includes operations S210 to S240.
[0046] In operation S210, the current network defense model is used to process the operating state corresponding to the network environment to obtain a defense action, wherein the defense action is used to defend against attack traffic in the network environment to obtain a new operating state, and the network defense model is constructed based on a reinforcement learning mechanism.
[0047] According to an embodiment of the present disclosure, a network defense model can be constructed based on a reinforcement learning mechanism, for example, using reinforcement learning algorithms such as DQN (Deep Q-Network) or A3C (Asynchronous Advantage Actor-Critic) to learn the optimal defense strategy. The network defense model is highly scalable and can flexibly integrate new data sources and defense strategies to cope with possible future attacks.
[0048] According to an embodiment of the present disclosure, the operating status corresponding to the network environment may be the traffic statistics during data transmission under the current network environment, as well as the current device status of the system and the security event alarms received by the system, etc., which together constitute a state vector as the operating status corresponding to the network environment.
[0049] According to the embodiments of the present disclosure, the current network defense model issues a defense decision to the attack traffic, and the action issued based on the defense decision is used as a defense action to achieve defense against the attack traffic. The defense action may be to adjust the firewall rules, block specific IPs, isolate suspicious nodes, upgrade the alarm level, etc.
[0050] According to an embodiment of the present disclosure, the current operating state is affected by the defense action, resulting in a change in the current operating state, thereby obtaining a new operating state. For example, the new operating state receives less network traffic than before the defense action is performed.
[0051] According to the embodiments of the present disclosure, by utilizing the current network defense model to process the operating status corresponding to the network environment, defense actions are obtained, and network threats can be detected and responded to in real time to realize an automated defense process, reduce the burden of manual management, and effectively utilize limited network and system resources, improve overall performance, and reduce the system burden.
[0052] In operation S220, a defense result obtained by executing the defense action is processed based on the reward function to obtain reward information, wherein the reward information is used to characterize the attack traffic intercepted based on the defense action and the degree of influence of executing the defense action on the running state.
[0053] According to an embodiment of the present disclosure, the reward function can be an optimization objective for defining a network defense model to achieve a balance between defending attack traffic and maintaining normal network traffic, for example, minimizing the number of successful attacks, maximizing the normal traffic pass rate, etc.
[0054] According to an embodiment of the present disclosure, the reward information may be expressed in the form of a numerical value, and the reward information may be assigned a value based on the interception result of the attack traffic of the defense action or the running state after the defense action is executed.
[0055] According to an embodiment of the present disclosure, the attack traffic is recorded by the system log after being intercepted by the defense action.
[0056] According to an embodiment of the present disclosure, the degree of influence of executing a defense action on the running state may be a change in the running state based on the defense action, for example, a change in the amount of network traffic, a loss ratio of data packets, and the like.
[0057] In operation S230, the current network defense model is updated based on the reward information to obtain a target network defense model.
[0058] According to an embodiment of the present disclosure, the reward information is used to adjust the parameters of the current network defense model to dynamically adjust the defense strategy according to the real-time changes in the network environment and attack traffic, so that the target network defense model obtained enhances the adaptability to unknown attacks.
[0059] According to the embodiments of the present disclosure, the network defense model can be updated in a simulated network environment or in an actual network environment. By updating in a simulated network environment, the network defense model is fully tested before deployment, which enhances the stability and reliability of the network defense model in an actual environment; further updating in an actual network environment makes the target network defense model more flexible.
[0060] According to the embodiments of the present disclosure, the network defense model is continuously trained and updated to adapt to evolving attack patterns and handle unprecedented attacks and complex security scenarios without relying on fixed defense rules, so as to adaptively adjust the defense strategy of the network defense model and dynamically defend the network environment and attack patterns.
[0061] In operation S240, the target operating state is processed based on the target network defense model to obtain a target defense action, wherein the generation time of the target operating state is later than the generation time of the operating state.
[0062] According to an embodiment of the present disclosure, the target operating state may be an operating state generated by executing a defense action, or may be an operating state in a next period of time corresponding to the current operating state.
[0063] According to the embodiments of the present disclosure, the target network defense model can effectively reduce false positives and false negatives and improve the accuracy and robustness of threat identification compared to the network defense model.
[0064] According to the embodiments of the present disclosure, by utilizing the target network defense model to process the target operating state, it is possible to automatically adapt to changes in the network environment and the evolution of attack patterns, and obtain target defense actions without frequent manual adjustments to the rules.
[0065] According to the embodiments of the present disclosure, when the network defense model is running in an actual network environment, it can receive real-time feedback, continuously optimize strategies, and improve the real-time response capability of the system.
[0066] According to the embodiment of the present disclosure, the current network defense model is used to process the running state corresponding to the network environment to obtain a defense action; the defense result obtained by executing the defense action is processed based on the reward function to obtain the reward information. Since the reward information is used to characterize the attack traffic intercepted based on the defense action, and to characterize the degree of influence of executing the defense action on the running state, the current network defense model is updated based on the reward information, so that the obtained target network defense model can understand the interception effect of the defense action on the attack traffic based on the reward information, and can understand the degree of influence of the defense action on the running state of the network environment through the reward information. In this way, the target running state detected later is processed based on the target network defense model obtained after training, so that the obtained target defense action can intercept the supply traffic more accurately and efficiently, and reduce the degree of negative impact of the target defense action on the running state of the network environment, so as to at least partially solve the problems of low interception accuracy of defense technical means in the network environment and slow operation response time of the network environment, and then realize the automatic optimization of the network defense model according to multiple defense actions, reduce the human resource cost generated by human resources participating in network security decision-making, and achieve the technical effect of improving the defense response speed and accuracy, and improving the operation efficiency of the network environment.
[0067] According to an embodiment of the present disclosure, the operating status includes at least one of the following: information flow receiving traffic and information flow sending traffic in network data transmission; the proportion of data packet loss in network data transmission; and bandwidth usage in network data transmission.
[0068] According to an embodiment of the present disclosure, the network defense model performs streaming analysis on the operating status in network data transmission to detect anomalies in real time in the data stream and quickly respond to threats.
[0069] According to an embodiment of the present disclosure, the operating status may also include IP (Internet Protocol) behavior information corresponding to each flow, and the IP behavior information includes the time and geographic location of sending or receiving the flow. The network defense model judges the IP information and the preset normal information, for example, determining whether the time of sending or receiving the flow is a reasonable time. By detecting behaviors that deviate from the norm, potential attacks can be identified, so that the network defense model can learn and identify the boundaries between normal and abnormal behaviors.
[0070] According to the embodiments of the present disclosure, a trust model for IP behavior information can also be established to dynamically evaluate the trust level of IP behavior information to determine IP behavior information that can be judged as normal, so as to achieve a rapid response to attack behavior.
[0071] According to an embodiment of the present disclosure, the current network defense model is used to process the operating status corresponding to the network environment to obtain a defense action, including: using the network defense model to process the network status of the current time period to obtain an initial defense action for the current time period; using an election mechanism to determine the defense action for the current time period from historical defense actions in historical time periods and the initial defense action for the current time period, wherein the moment of determining the historical defense action is earlier than the moment of determining the initial defense action.
[0072] According to the embodiments of the present disclosure, the network defense model can detect the network status of the current period in real time to obtain the initial defense action to deal with the attack traffic of the current period.
[0073] According to an embodiment of the present disclosure, the network defense model counts the operating status of the historical time period and the defense actions performed to defend against historical traffic attacks in the historical time period to predict potential abnormal behaviors in the current time period, such as changes in the network status of the current time period and the attack traffic that may be received, and obtains historical defense actions for the historical time period based on the prediction results.
[0074] According to an embodiment of the present disclosure, when the historical defense actions of the historical period and the initial defense actions of the current period are inconsistent, an election mechanism is utilized to determine a defense action with better defense effect as the defense action of the current period between the historical defense actions of the historical period and the initial defense actions of the current period.
[0075] According to the embodiments of the present disclosure, by predicting potential abnormal behaviors in the current period, the accuracy and robustness of attack traffic identification can be improved, and the false alarm and missed alarm rates can be reduced. In addition, an election is conducted between the initial defense actions obtained in real time to determine the best defense action, and the defense strategy can be adjusted dynamically, which enhances the adaptability to unknown threats, so that the defense strategy of the network defense model can be learned and updated from the final defense action, reducing false alarms and missed alarms, and improving the response speed and defense accuracy. In addition, the use of network defense models to realize automated defense processes reduces the burden of manual management, allowing the security management team to focus on higher-level strategic planning and the handling of complex events.
[0076] According to an embodiment of the present disclosure, an election mechanism is utilized to determine a defense action for a current period from among historical defense actions in historical periods and initial defense actions in a current period, including: determining a first weight corresponding to the historical defense action based on defense attributes of the historical defense action; determining a second weight corresponding to the initial defense action based on defense attributes of the initial defense action; comparing the first weight with the second weight to obtain a first comparison result; and determining, based on the first comparison result, a defense action for the current period from among among the historical defense actions in historical periods and the initial defense action in the current period.
[0077] According to an embodiment of the present disclosure, the defense attribute may be the type of defense action, such as a firewall configuration action, an access control action, a data encryption action or a traffic interception action, etc., wherein the defense action may include multiple actions corresponding to multiple types to form a multi-level defense system, thereby increasing the difficulty for attackers to break through the defense line.
[0078] According to an embodiment of the present disclosure, the defense attribute may also be the priority of the defense action, etc. Different types of defense actions have corresponding priorities, which are expressed in numerical form. For example, the priority of the traffic interception action is 3, and the priority of the firewall configuration action is 1. The priority of the traffic interception action is greater than that of the firewall configuration action, etc.
[0079] According to an embodiment of the present disclosure, the first weight corresponding to the historical defense action may include the type and priority level of the historical defense action; and the second weight corresponding to the initial defense action may include the type and priority level of the initial defense action.
[0080] According to an embodiment of the present disclosure, the first weight is compared with the second weight, and the sum of the numerical values corresponding to the priority levels of the two is preferentially compared. If the first comparison result indicates that the sum of the numerical values corresponding to the priority level corresponding to the first weight is greater than the second weight, the historical defense action is used as the defense action for the current time period.
[0081] If the first comparison result indicates that the sum of the values corresponding to the priority levels corresponding to the first weight is equal to the second weight, the type of defense action is further judged to determine the degree of influence of different types of defense actions on the network operation status, and the first comparison result is updated. If the updated first comparison result indicates that the type of defense action corresponding to the first weight has a smaller influence on the network operation status than the type of defense action corresponding to the second weight, the defense action corresponding to the first weight is used as the defense action for the current time period.
[0082] According to an embodiment of the present disclosure, the defense action for the current period is determined based on the defense attributes of the historical defense actions and the defense attributes of the initial defense actions, so that the defense action for the current period can better cope with the attack traffic while avoiding affecting the input of normal traffic as much as possible.
[0083] According to an embodiment of the present disclosure, the operating status and defense action are processed based on a reward function to obtain reward information, including: processing defense success information and defense failure information based on a reward function to obtain reward information, wherein the defense success information represents that the defense action successfully intercepts the attack traffic, and the defense failure information represents that the defense action fails to intercept the attack traffic, and the defense result includes the defense success information and the defense failure information.
[0084] According to an embodiment of the present disclosure, the defense result is processed based on the reward function. If the defense action successfully intercepts the traffic attack, the reward information is assigned a positive reward; if the defense action fails to intercept the traffic attack, the reward information is assigned a negative reward.
[0085] According to an embodiment of the present disclosure, the defense success information includes at least one of the following: the amount of intercepted attack traffic, and the importance level corresponding to the intercepted attack traffic.
[0086] According to the embodiments of the present disclosure, when the defense action successfully intercepts the traffic attack, the attack traffic is captured and recorded, and the number of intercepted attack traffic is counted by the system log. In addition, different types of attack traffic have different attack degrees on system data, so the interception importance of each type of attack traffic is different, that is, the intercepted attack traffic has different corresponding importance levels.
[0087] According to the embodiments of the present disclosure, when the defense action fails to intercept the traffic attack, it may be that the attack traffic is not intercepted, or only part of the attack traffic is intercepted, for example, only the attack traffic with a low importance level is successfully intercepted, and the attack traffic with a high importance level is missed, causing damage to the system data or operating status, that is, invalid interception, and the defense action fails to intercept the traffic attack.
[0088] According to an embodiment of the present disclosure, processing defense success information and defense failure information based on a reward function includes: processing defense success information, defense failure information and a reference operating state based on a reward function, wherein the reference operating state is generated based on the successful interception of attack traffic by a defense action.
[0089] According to an embodiment of the present disclosure, a reward function is constructed based on an operating state reward sub-function; processing defense success information, defense failure information and a reference operating state based on the reward function includes: using an operating state reward sub-function to process a reference operating state and a preset state to obtain operating state sub-reward information, wherein the reward information is determined based on the operating state sub-reward information.
[0090] According to an embodiment of the present disclosure, the reward function may be a statistical reward information corresponding to the defense results within a period of time, wherein the reward function may include a defense success reward sub-function, a defense failure reward sub-function and an operation status reward sub-function.
[0091] According to an embodiment of the present disclosure, the defense success reward sub-function is used to count the defense success information within a period of time to obtain defense success sub-reward information.
[0092] According to an embodiment of the present disclosure, the defense failure reward sub-function is used to count the defense failure information within a period of time to obtain defense failure sub-reward information.
[0093] According to an embodiment of the present disclosure, the reference operating state is obtained after the defense action successfully intercepts the attack traffic and affects the current network state.
[0094] According to an embodiment of the present disclosure, the preset state may be a reasonable operating state generated when the system does not receive attack traffic or when all attack traffic is intercepted, for example, normal traffic is not intercepted.
[0095] According to an embodiment of the present disclosure, the operating status reward sub-function is used to count the reference operating status within a period of time, and perform similarity calculations on the reference operating status and the preset status to determine the distance value between the two, that is, the difference information between the two. When the distance value does not meet the preset distance value, the operating status sub-reward information is assigned a negative reward.
[0096] According to an embodiment of the present disclosure, statistics of defense success sub-reward information, defense failure sub-reward information, and operation status sub-reward information are assigned to determine reward information.
[0097] According to the embodiments of the present disclosure, by utilizing a reward function to process the reference operating state, the network defense model can optimize the rationality of the defense decision, so that the generated defense actions can ensure that the accuracy and response speed are improved while ensuring that the defense actions do not affect normal traffic.
[0098] According to an embodiment of the present disclosure, the reward information is obtained by processing the operating state and the defense action based on the reward function, including: using the reward function to process the i-th defense action in the i-th time period and the defense result corresponding to the i-th defense action, to obtain the reward information of the i-th time period; wherein the reward information of the i-th time period is used to update the network defense model of the i-th time period to obtain the updated network defense model, and the updated network defense model is used to generate the target defense action for the i+1-th time period.
[0099] According to an embodiment of the present disclosure, the reward function can process the defense results obtained by executing defense actions within a certain time period according to a preset cycle, and can also process the defense results obtained by executing defense actions in each time period to achieve real-time updating of the network defense model.
[0100] According to an embodiment of the present disclosure, the defense method for the network environment can also monitor the performance indicators of the network defense model, such as the defense success rate, false alarm rate and missed alarm rate, and adjust the parameters or reward function of the network defense model based on the performance indicators to achieve optimization of the network defense model.
[0101] According to an embodiment of the present disclosure, a bait module may also be provided in the defense process to attract and consume resources of attack traffic and collect attack information of the attack traffic, so as to provide insight into the attack traffic for the network defense model and serve as a basis for optimizing the defense strategy of the network defense model.
[0102] According to the embodiments of the present disclosure, artificial intelligence technology is used to conduct threat assessment and recommendations on attack traffic to assist network defense models and security managers in analysis and decision-making, thereby forming a human-machine collaborative defense.
[0103] Figure 3 A schematic diagram of a defense method for a network environment according to an embodiment of the present disclosure is schematically shown.
[0104] According to the embodiments of the present disclosure, Figure 3As shown, the defense method for the network environment includes operations S310 to S360. In operation 310, the initial defense action of the current period is determined. In operation S320, the historical defense actions of the historical period are determined. According to the defense attributes of the historical defense actions and the defense attributes of the initial defense actions, in operation S330, the defense action of the current period is determined in the initial defense action and the historical defense action. In operation S340, the defense action is executed to determine the defense success information, the defense failure information and the reference operating state. In operation S350, the defense success information, the defense failure information and the reference operating state are processed based on the reward function to obtain the reward information. In operation S360, the current network defense model is updated based on the reward information to obtain the target network defense model, so as to process the target operating state using the target network defense model.
[0105] According to the embodiments of the present disclosure, the defense method for the network environment of the embodiments of the present disclosure overcomes the limitations of traditional rule reliance, and can dynamically adjust the defense strategy according to the real-time changes of the network environment and attack mode, thereby enhancing the adaptability to unknown threats and realizing the adaptive defense strategy of the network defense model. In addition, since the strategy is learned and updated from the actual defense action, the response speed and defense accuracy of the defense action are improved, and false positives and false negatives are reduced to a certain extent.
[0106] A second aspect of the present disclosure provides a method for training a network defense model, the method comprising operations S410 to S440.
[0107] In operation S410, a simulated network environment is constructed.
[0108] In operation S420, the network defense model is used to process and simulate the operation status of the network environment to obtain a defense action.
[0109] In operation S430, a defense result obtained by performing the defense action is processed based on the reward function to obtain reward information.
[0110] In operation S440, the network defense model is trained based on the reward information to obtain a trained network defense model.
[0111] According to an embodiment of the present disclosure, an abstract model of a simulated network environment is defined, including network topology, system device properties, normal traffic patterns and attack models, and a simulated network environment is constructed to simulate network traffic, attack behaviors and defense actions.
[0112] According to an embodiment of the present disclosure, in a simulated network environment, a network defense model learns the best defense strategy by interacting with the simulated network environment, and reward information is generated each time or multiple actions are taken to update the network defense model.
[0113] According to the embodiments of the present disclosure, during the training of the network defense model, an experience replay buffer (Experience Replay) can also be set. The experience can be the operating state, defense action and reward information generated by the interaction between the network defense model and the simulated network environment. These experiences can be randomly extracted for the training of the network defense model, which helps to break the time correlation between experience samples and make the learning process more stable. In addition, these experiences are stored in a "first in, first out" manner. When the buffer reaches the upper limit of capacity, the earliest experience will be replaced by the new experience.
[0114] According to the embodiments of the present disclosure, in an actual network environment, the network defense model can still use the above-mentioned experience playback buffer to update the network defense model.
[0115] Figure 5 A schematic diagram schematically illustrates the training and application of a network defense model according to an embodiment of the present disclosure.
[0116] According to the embodiments of the present disclosure, Figure 5 As shown, in operation S510, a simulated network environment is constructed. In operation S520, the operating state of the simulated network environment is obtained. In operation S530, the operating state of the simulated network environment is processed using a network defense model, a defense action suitable for the operating state is obtained, the defense action is executed, and a defense result is obtained. In operation S540, the defense result is processed using a reward function to obtain reward information. In operation S550, the network defense model is trained based on the obtained reward information. In operation S560, based on the above operations S520 to S550, the network defense model is trained multiple times in the simulated network environment to obtain a trained network defense model. In operation S570, the trained network defense model is put into the actual network environment, and the network defense model is updated based on the operating state of the real-time network environment. In operation S580, based on the above operation S570, the network defense model is continuously updated in the actual network environment so that the network defense model adapts to the actual network environment.
[0117] Based on the above-mentioned defense method for network environment, the present disclosure also provides a defense device for network environment. Figure 6 The device is described in detail.
[0118] Figure 6 The structural block diagram of a defense device for a network environment according to an embodiment of the present disclosure is schematically shown.
[0119] like Figure 6 As shown, the defense device 600 for a network environment of this embodiment includes a first processing module 610 , a second processing module 620 , an update module 630 and a third processing module 640 .
[0120] The first processing module 610 is used to process the operating state corresponding to the network environment using the current network defense model to obtain a defense action, wherein the defense action is used to defend against attack traffic in the network environment to obtain a new operating state, and the network defense model is constructed based on a reinforcement learning mechanism. In one embodiment, the first processing module 610 can be used to perform the operation S210 described above, which will not be repeated here.
[0121] The second processing module 620 is used to process the defense result obtained by executing the defense action based on the reward function to obtain reward information, wherein the reward information is used to characterize the attack traffic intercepted based on the defense action and the degree of influence of executing the defense action on the running state. In one embodiment, the second processing module 620 can be used to perform the operation S220 described above, which will not be repeated here.
[0122] The updating module 630 is used to update the current network defense model based on the reward information to obtain the target network defense model. In one embodiment, the updating module 630 can be used to perform the operation S230 described above, which will not be described in detail here.
[0123] The third processing module 640 is used to process the target operating state based on the target network defense model to obtain a target defense action, wherein the target operating state is generated later than the operating state. In one embodiment, the third processing module 640 can be used to perform the operation S240 described above, which will not be repeated here.
[0124] According to the embodiment of the present disclosure, the current network defense model is used to process the running state corresponding to the network environment to obtain a defense action; the defense result obtained by executing the defense action is processed based on the reward function to obtain the reward information. Since the reward information is used to characterize the attack traffic intercepted based on the defense action, and to characterize the degree of influence of executing the defense action on the running state, the current network defense model is updated based on the reward information, so that the obtained target network defense model can understand the interception effect of the defense action on the attack traffic based on the reward information, and can understand the degree of influence of the defense action on the running state of the network environment through the reward information. In this way, the target running state detected later is processed based on the target network defense model obtained after training, so that the obtained target defense action can intercept the supply traffic more accurately and efficiently, and reduce the degree of negative impact of the target defense action on the running state of the network environment, so as to at least partially solve the problems of low interception accuracy of defense technical means in the network environment and slow operation response time of the network environment, and then realize the automatic optimization of the network defense model according to multiple defense actions, reduce the human resource cost generated by human resources participating in network security decision-making, and achieve the technical effect of improving the defense response speed and accuracy, and improving the operation efficiency of the network environment.
[0125] According to an embodiment of the present disclosure, any multiple modules of the first processing module 610, the second processing module 620, the update module 630, and the third processing module 640 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the first processing module 610, the second processing module 620, the update module 630, and the third processing module 640 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, at least one of the first processing module 610 , the second processing module 620 , the updating module 630 , and the third processing module 640 may be at least partially implemented as a computer program module, and when the computer program module is executed, a corresponding function may be performed.
[0126] According to an embodiment of the present disclosure, the first processing module 610 includes a first processing submodule and a first determining submodule.
[0127] The first processing submodule is used to process the network status of the current period by using the network defense model to obtain the initial defense action of the current period.
[0128] The first determination submodule is used to determine the defense action of the current period from the historical defense actions of the historical periods and the initial defense action of the current period by using an election mechanism, wherein the time of determining the historical defense action is earlier than the time of determining the initial defense action.
[0129] According to an embodiment of the present disclosure, the first determining submodule includes a first determining unit, a second determining unit, a first comparing unit, and a third determining unit.
[0130] The first determining unit is used to determine a first weight corresponding to the historical defense action according to the defense attribute of the historical defense action.
[0131] The second determining unit is used to determine a second weight corresponding to the initial defense action according to the defense attribute of the initial defense action.
[0132] The first comparison unit is used to compare the first weight with the second weight to obtain a first comparison result.
[0133] The third determining unit is used to determine the defense action of the current period from the historical defense actions of the historical period and the initial defense action of the current period according to the first comparison result.
[0134] According to an embodiment of the present disclosure, the second processing module 620 includes a second processing sub-module.
[0135] The second processing submodule is used to process the defense success information and the defense failure information based on the reward function to obtain the reward information, wherein the defense success information represents that the defense action successfully intercepts the attack traffic, and the defense failure information represents that the defense action fails to intercept the attack traffic, and the defense result includes the defense success information and the defense failure information.
[0136] According to an embodiment of the present disclosure, the defense success information includes at least one of the following: the amount of intercepted attack traffic, and the importance level corresponding to the intercepted attack traffic.
[0137] According to an embodiment of the present disclosure, the second processing submodule includes a first processing unit.
[0138] The first processing unit is used to process defense success information, defense failure information and a reference operating state based on a reward function, wherein the reference operating state is generated based on the successful interception of attack traffic by the defense action.
[0139] According to an embodiment of the present disclosure, the first processing unit includes a first processing sub-unit.
[0140] The first processing sub-unit is used to process the reference running state and the preset state by using the running state reward sub-function to obtain running state sub-reward information, wherein the reward information is determined based on the running state sub-reward information.
[0141] According to an embodiment of the present disclosure, the operating status includes at least one of the following: information flow receiving traffic and information flow sending traffic in network data transmission; the proportion of data packet loss in network data transmission; and bandwidth usage in network data transmission.
[0142] According to an embodiment of the present disclosure, the second processing module 620 includes a third processing submodule.
[0143] The third processing sub-module is used to use the reward function to process the i-th defense action in the i-th time period and the defense result corresponding to the i-th defense action to obtain the reward information of the i-th time period; wherein the reward information of the i-th time period is used to update the network defense model of the i-th time period to obtain the updated network defense model, and the updated network defense model is used to generate the target defense action in the i+1-th time period.
[0144] Figure 7 A block diagram of an electronic device suitable for implementing a defense method for a network environment according to an embodiment of the present disclosure is schematically shown.
[0145] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.
[0146] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. Processor 701 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 702 and / or RAM 703. It should be noted that the program can also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.
[0147] According to an embodiment of the present disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.
[0148] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.
[0149] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 702 and / or RAM 703 described above and / or one or more memories other than ROM 702 and RAM 703.
[0150] The embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the above method provided by the embodiment of the present disclosure.
[0151] The above functions defined in the system / device of the embodiment of the present disclosure are performed when the computer program is executed by the processor 701. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.
[0152] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 709, and / or installed from the removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.
[0153] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above functions defined in the system of the embodiment of the present disclosure are performed. According to the embodiment of the present disclosure, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.
[0154] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).
[0155] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0156] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.
[0157] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.
Claims
1. A defense method for a network environment, characterized in that: The method comprises: Using the current network defense model to process the operating state corresponding to the network environment, obtaining a defense action, wherein the defense action is used to defend against attack traffic in the network environment, and obtaining a new operating state, wherein the network defense model is constructed based on a reinforcement learning mechanism; Processing a defense result obtained by executing the defense action based on a reward function to obtain reward information, wherein the reward information is used to characterize the attack traffic intercepted based on the defense action and to characterize the degree of influence of executing the defense action on the running state; Based on the reward information, the current network defense model is updated to obtain a target network defense model; and A target operating state is processed based on the target network defense model to obtain a target defense action, wherein a generation time of the target operating state is later than a generation time of the operating state.
2. The method according to claim 1, characterized in that The method of using the current network defense model to process the operating state corresponding to the network environment to obtain a defense action includes: Use the network defense model to process the network status of the current period and obtain the initial defense action of the current period; The defense action of the current period is determined by using an election mechanism from historical defense actions of historical periods and an initial defense action of the current period, wherein the time of determining the historical defense action is earlier than the time of determining the initial defense action.
3. The method according to claim 2, characterized in that The method of utilizing an election mechanism to determine the defense action of the current period from the historical defense actions of the historical period and the initial defense action of the current period includes: determining, according to the defense attribute of the historical defense action, a first weight corresponding to the historical defense action; determining, according to the defense attribute of the initial defense action, a second weight corresponding to the initial defense action; Compare the first weight with the second weight to obtain a first comparison result; According to the first comparison result, a defense action for the current period is determined from the historical defense actions for the historical period and the initial defense action for the current period.
4. The method according to claim 1, characterized in that: The step of processing the running state and the defense action based on a reward function to obtain reward information includes: The defense success information and the defense failure information are processed based on a reward function to obtain the reward information, wherein the defense success information represents that the defense action successfully intercepts the attack traffic, and the defense failure information represents that the defense action fails to intercept the attack traffic, and the defense result includes the defense success information and the defense failure information.
5. The method according to claim 4, characterized in that The defense success information includes at least one of the following: The amount of intercepted attack traffic and the importance level corresponding to the intercepted attack traffic.
6. The method according to claim 4, characterized in that The processing of the defense success information and the defense failure information based on the reward function includes: The defense success information, the defense failure information and the reference operating state are processed based on the reward function, wherein the reference operating state is generated based on the defense action successfully intercepting the attack traffic.
7. The method according to claim 6, characterized in that The reward function is constructed based on the running state reward sub-function; Processing defense success information, defense failure information, and reference operation status based on the reward function includes: The reference operating state and the preset state are processed by using the operating state reward sub-function to obtain operating state sub-reward information, wherein the reward information is determined based on the operating state sub-reward information.
8. The method according to claim 1, characterized in that The operating status includes at least one of the following: The information flow receiving traffic and information flow sending traffic in network data transmission; The percentage of data packets lost during network data transmission; Bandwidth usage in network data transfer.
9. The method according to claim 1, characterized in that: Processing the running state and the defense action based on the reward function to obtain reward information includes: Processing the i-th defense action in the i-th time period and the defense result corresponding to the i-th defense action by using a reward function to obtain reward information for the i-th time period; The reward information of the i-th period is used to update the network defense model of the i-th period to obtain an updated network defense model, and the updated network defense model is used to generate a target defense action of the i+1-th period.
10. A method for training a network defense model, characterized in that: The method comprises: Build a simulated network environment; Using the network defense model according to any one of claims 1 to 9 to process the operating status of the simulated network environment, to obtain a defense action; Processing a defense result obtained by executing the defense action based on a reward function to obtain reward information; Based on the reward information, the network defense model is trained to obtain a trained network defense model.
11. A defense device for a network environment, characterized in that: The device comprises: A first processing module is used to process the operating state corresponding to the network environment using the current network defense model to obtain a defense action, wherein the defense action is used to defend against attack traffic in the network environment to obtain a new operating state, and the network defense model is constructed based on a reinforcement learning mechanism; A second processing module is used to process the defense result obtained by executing the defense action based on the reward function to obtain reward information, wherein the reward information is used to characterize the attack traffic intercepted based on the defense action and the degree of influence of executing the defense action on the running state; an updating module, configured to update the current network defense model based on the reward information to obtain a target network defense model; and The third processing module is used to process the target operating state based on the target network defense model to obtain a target defense action, wherein the generation time of the target operating state is later than the generation time of the operating state.
12. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 9.
13. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.
14. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.
Citation Information
Cited By
LFA passive defense method and device, electronic equipment and storage medium
CN119945728A
