Learning method for automating responses against security events and computing device using the same

Abstract

According to the present invention, a method for learning to automate the response to security events comprises: (a) when learning context information including learning Z-t operational state information and learning Z-t security state information is obtained from at least some of at least one security monitoring asset, a computing device calculates learning Z-t operational cost information and learning Z-t security benefit information by referring to the learning Z-t context information, and calculates each of a plurality of learning Z-t reality reward values ​​for each of a plurality of learning Z-t response actions that can be performed by the security monitoring asset by referring to the learning Z-t operational cost information and the learning Z-t security benefit information; (b) the computing device inputs the learning Z-t context information into a Q-Network as learning Z-t state information, so that the Q-Network outputs each of a plurality of learning Z-t state-action value function predictions that match each of the plurality of learning Z-t response actions when each of the plurality of learning Z-t response actions is performed by the security monitoring asset according to the learning Z-t state information;and (c) the computing device determines a specific learning response action to be performed by the security control asset among each of the plurality of specific learning response actions, and obtains a specific learning reality reward value determined according to the specific learning response action—the specific learning reality reward value is any one of the plurality of specific learning reality reward values—and learning state information (t+1), and inputs the learning state information (t+1) to the Q-Network so that when each of the plurality of specific learning response actions (t+1) is performed by the security control asset in the learning state information (t+1), the Q-Network outputs each of the multiple learning state-action value function prediction values ​​that match each of the plurality of learning response actions (t+1), and the largest learning state-action value function prediction value among the specific learning reward value, the learning rate, and each of the multiple learning state-action value function prediction values. A method and a computing device using the same are disclosed, comprising the step of generating a learning jet state-action value function GT by referencing a discount rate therefor, and learning the parameters of the Q-Network using a loss generated by referencing the learning jet specific state-action value function prediction value corresponding to the learning jet specific corresponding action among each of the plurality of learning jet state-action value function prediction values ​​and the learning jet state-action value function GT.

Description

Technology Field

[0001] The present invention relates to a method for learning to automate the response to security events and a computing device using the same, and to a method for learning to effectively neutralize security-related attack threats while minimizing operational costs that may arise from business operation interruptions, and a computing device using the same. Background Technology

[0002] In the modern cybersecurity environment, cyber attacks are constantly becoming more sophisticated and larger in scale, with various forms of web-based attacks, such as SQL Injection, XSS, and phishing domains, occurring in multiple dimensions. Traditionally, defenses were based on known attack patterns or static rules, but this type of defense had the problem that it was difficult to prevent attackers from scouting systems and analyzing vulnerabilities to launch attacks through Advanced Persistent Threats (APTs).

[0003] Therefore, to address the aforementioned problem, an MTD system based on Moving Target Defense (MTD) has emerged. This is an active defense technology capable of neutralizing attacker reconnaissance and attacks by dynamically changing vulnerable system elements such as IP addresses, port numbers, operating system information, and data schemas. At this time, the MTD system can change IPs, port numbers, etc., in three ways, which are explained as follows.

[0004] For example, if the MTD system can change IPs, port numbers, etc. at preset time intervals (e.g., 10 minutes, 1 hour, etc.), it may still be vulnerable to attacks if an attacker knows the change cycle, and there is a problem that the service may be unintentionally interrupted if the timing of the change of IPs, port numbers, etc. corresponds to peak business hours or does not correspond to a predetermined maintenance time.

[0005] As another example, if the MTD system can change IPs, port numbers, etc. at randomly determined intervals within a set time range (e.g., between 5 and 20 minutes), it becomes difficult for attackers to identify the change cycle, making it easier to respond to attacks. However, there is a problem in that this method may not take business situations into account, as the time when it is decided to randomly change IPs, port numbers, etc. may overlap with the time when security monitoring assets such as SOAR (Security Orchestration, Automation, and Response) are executing a playbook for ransomware response.

[0006] As another example, if the MTD system can change the IP, port number, etc., when a specific security-related event is detected more than a preset number of times in a security monitoring asset such as SIEM (Security Information and Event Management), it is an easy method to respond to a single specific security-related event, but there are problems such as, for instance, if the IP of the SIEM is changed when a scan event is detected 10 times in the SIEM, the large volume of log sources collected by the SIEM cannot be collected, and if the IP of the SOAR is changed while the SOAR is interacting with the firewall through a playbook, the firewall may be interrupted.

[0007] In other words, as a result of focusing on the change mechanism regarding how and what to change when responding to security attacks using MTD technology, it can affect the operation of the service, leading to a situation where the defender, rather than the attacker, denies the service (Self-inflicted DoS).

[0008] Therefore, when automating the response to security events by applying MTD technology, there is a need for a solution to simultaneously achieve the conflicting goals of enhancing security and ensuring business continuity for uninterrupted 24-hour monitoring services. The problem to be solved

[0009] The present invention aims to solve all of the aforementioned problems.

[0010] In addition, the present invention has another objective of training a Q-Network to enable decision-making while effectively neutralizing security-related attacks and minimizing cost losses associated with business operations.

[0011] In addition, the present invention has another objective of calculating security benefits and operating costs for a security monitoring asset through context information obtained from at least some of at least one security monitoring asset, and generating a target policy for learning a Q-Network by referring to a real reward value for each response action calculated by referring to the security benefits and operating costs, respectively, and a future profit. means of solving the problem

[0012] The characteristic configuration of the present invention for achieving the objectives of the present invention as described above and realizing the characteristic effects of the present invention described below is as follows.

[0013] According to one aspect of the present invention, a method for learning to automate the response to security events comprises: (a) when learning context information including learning Z-t operational state information and learning Z-t security state information is obtained from at least some of at least one security control asset, a computing device calculates learning Z-t operational cost information and learning Z-t security benefit information by referring to the learning Z-t context information, and calculates each of a plurality of learning Z-t reality reward values ​​for each of a plurality of learning Z-t response actions that can be performed by the security control asset by referring to the learning Z-t operational cost information and the learning Z-t security benefit information; (b) the computing device inputs the learning Z-t context information into a Q-Network as learning Z-t state information, so that the Q-Network outputs each of a plurality of learning Z-t state-action value function predictions that match each of the plurality of learning Z-t response actions when each of the plurality of learning Z-t response actions is performed by the security control asset according to the learning Z-t state information;and (c) the computing device determines a specific learning response action to be performed by the security control asset among each of the plurality of specific learning response actions, and obtains a specific learning reality reward value determined according to the specific learning response action—the specific learning reality reward value is any one of the plurality of specific learning reality reward values—and learning state information (t+1), and inputs the learning state information (t+1) to the Q-Network so that when each of the plurality of specific learning response actions (t+1) is performed by the security control asset in the learning state information (t+1), the Q-Network outputs each of the multiple learning state-action value function prediction values ​​that match each of the plurality of learning response actions (t+1), and the largest learning state-action value function prediction value among the specific learning reward value, the learning rate, and each of the multiple learning state-action value function prediction values. A method is disclosed comprising the step of generating a learning jet state-action value function GT by referring to a discount rate thereon, and learning parameters of the Q-Network using a loss generated by referring to a learning jet specific state-action value function prediction value corresponding to the learning jet specific corresponding action among each of the plurality of learning jet state-action value function prediction values ​​and the learning jet state-action value function GT.

[0014] As an example, a method is disclosed in which, in step (c) above, the computing device, by an ε-greedy policy, randomly determines the learning-specific corresponding action among each of the plurality of learning-specific corresponding actions when it corresponds to a first action decision mode, determines the largest value among each of the plurality of learning-specific state-action value function prediction values ​​as the learning-specific state-action value function prediction value, and determines the corresponding action corresponding to the learning-specific state-action value function prediction value as the learning-specific corresponding action.

[0015] As an example, a method is disclosed that further comprises the step of (d) after optimizing the parameters of the Q-Network by repeating steps (a) through (c) above, when test context information including test operation status information and test security status information is obtained from the security monitoring asset, the computing device inputs the test context information as test state information to the Q-Network, thereby causing the Q-Network to output each of a plurality of test state-action value function prediction values ​​that match each of the plurality of test state-action value function prediction values ​​when each of the plurality of test state-action value function prediction values ​​is performed by the security monitoring asset according to the test state information, determines a test specific response action corresponding to the largest test specific state-action value function prediction value among the plurality of test specific response actions, and causes the security monitoring asset to perform a response to the test context information according to the test specific response action.

[0016] As an example, the method further comprises the step of: (e) the computing device calculating test operation cost information and test security benefit information by referring to test context information; obtaining a specific reality reward value corresponding to a specific response action among a plurality of test response actions that can be performed by the security control asset by referring to test operation cost information and test security benefit information; obtaining test (m+1) state information determined according to the specific response action; storing a test data set including test context information, test specific response action, test specific reality reward value, and test (m+1) state information in a buffer memory; and when a preset number or more of test data sets are stored in the buffer memory, performing fine-tuning of the Q-network by performing steps (a) through (c) using the test data set. It begins.

[0017] A method is disclosed in which, in step (a) above, the computing device calculates the learning jet operation cost information including at least some of asset class information determined according to the degree of business impact when the operation of the security monitoring asset is interrupted, service dependency information determined according to the connection relationship of the security monitoring asset, active session information, and time-based weighting information by referring to the learning jet context information, calculates the learning jet security benefit information including at least some of risk information, attack phase information, and asset vulnerability information for the security monitoring asset, and calculates each of the plurality of learning jet reality reward values ​​for each of the plurality of learning jet response actions by referring to the difference between the value obtained by multiplying the learning jet security benefit information and the corresponding learning jet benefit weight and the value obtained by multiplying the learning jet operation cost information and the corresponding learning jet cost weight.

[0018] As an example, a method is disclosed in which, in step (a) above, the computing device calculates each of the plurality of learning reality reward values ​​by setting the initial value of the learning cost weight to be greater than the initial value of the learning profit weight.

[0019] As an example, a method is disclosed in which, in step (a) above, the computing device determines the asset class information by additionally referring to at least some of the number of other security control assets connected to the security control asset and the importance of the connection relationship of the security control asset.

[0020] As an example, a method is disclosed in which, in step (a) above, each of the plurality of learning response actions includes at least some of the following: a first learning response action for changing the IP address of the security monitoring asset, a second learning response action for changing the port of the security monitoring asset, a third learning response action for creating and distributing a decoy corresponding to the security monitoring asset, a fourth learning response action for controlling a network path using a Software Defined Networking (SDN) controller, and a fifth learning response action for continuing monitoring.

[0021] As an example, a method is disclosed in which, in step (b) above, the computing device performs normalization on the learning jet context information, converts it into a vector value, and then inputs it into the Q-network.

[0022] As an example, a method is disclosed in which, in step (a) above, the computing device obtains the learning jet operational state information from at least some of a service dependency information generating device, an asset management database, a switch, a load balancer, and a SIEM (Security Information and Event Management), and obtains the learning jet security state information from at least some of a SOAR (Security Orchestration, Automation, and Response), an IDS (Intrusion Detection System), an IPS (Intrusion Prevention System), a firewall, an EDR (Endpoint Detection and Response), a CTI (Cyber ​​Threat Intelligence), and the SIEM.

[0023] According to another aspect of the present invention, a computing device for learning to automate the response to security events comprises: at least one memory for storing instructions; and at least one processor configured to execute said instructions; wherein the processor comprises: (I) a process in which, when learning Z-t context information including learning Z-t operational state information and learning Z-t security state information is obtained from at least some of at least one security control asset, the processor calculates learning Z-t operational cost information and learning Z-t security benefit information by referring to said learning Z-t context information, and calculates each of a plurality of learning Z-t reality reward values ​​for each of a plurality of learning Z-t response actions that the security control asset can perform by referring to said learning Z-t operational cost information and said learning Z-t security benefit information; (II) A process of inputting the above-mentioned learning jet context information into a Q-Network as learning jet state information, so that when each of the above-mentioned learning jet response actions is performed by the above-mentioned security control asset according to the above-mentioned learning jet state information, the Q-Network outputs each of the above-mentioned learning jet state-action value function predictions that match each of the above-mentioned learning jet response actions;and (III) determine a specific learning response action to be performed by the security control asset among each of the plurality of specific learning response actions, and obtain a specific learning reality reward value determined according to the specific learning response action—the specific learning reality reward value is any one of the plurality of specific learning reality reward values—and learning state information (t+1), and input the learning state information (t+1) into the Q-Network so that when each of the plurality of learning response actions (t+1) is performed by the security control asset in the learning state information, the Q-Network outputs each of the multiple learning state-action value function predictions that match each of the plurality of learning response actions (t+1), and the largest learning state-action value function prediction among the specific learning reward value, the learning rate, and the discount rate for the largest learning state-action value function prediction among the multiple learning state-action value function predictions. A computing device is disclosed that performs a process of generating a learning jet state-action value function GT by reference, and learning parameters of the Q-Network using a loss generated by referencing the learning jet specific state-action value function prediction value corresponding to the learning jet specific corresponding action among each of the plurality of learning jet state-action value function prediction values ​​and the learning jet state-action value function GT.

[0024] As an example, a computing device is disclosed in which the processor, in the process (III), randomly determines the learning-specific corresponding action among each of the plurality of learning-specific corresponding actions when the first action decision mode is in accordance with the ε-greedy policy, and when the second action decision mode is in accordance with the second action decision mode, determines the largest value among each of the plurality of learning-specific state-action value function prediction values ​​as the learning-specific state-action value function prediction value, and determines the corresponding action corresponding to the learning-specific state-action value function prediction value as the learning-specific corresponding action.

[0025] As an example, a computing device is disclosed that further performs a process in which the processor, after optimizing the parameters of the Q-Network by repeating the process (I) through the process (III), inputs the test context information including test operation status information and test security status information from the security monitoring asset into the Q-Network as test state information, thereby causing the Q-Network to output each of a plurality of test state-action value function prediction values ​​that match each of the plurality of test state-action value function prediction values ​​when each of the plurality of test state-action value function prediction values ​​is performed by the security monitoring asset according to the test state information, determines a test specific response action corresponding to the largest test specific state-action value function prediction value among the plurality of test specific response actions, and causes the security monitoring asset to perform a response to the test context information according to the test specific response action.

[0026] As an example, the computing device further performs a process in which the processor (V) calculates test operation cost information and test security benefit information by referring to test context information, obtains a test specific reality reward value corresponding to test specific response action among a plurality of test response actions that can be performed by the security monitoring asset by referring to test operation cost information and test security benefit information, obtains test (m+1) state information determined according to test specific response action, stores a test data set including test context information, test specific response action, test specific reality reward value, and test (m+1) state information in a buffer memory, and performs fine-tuning on the Q-network by performing steps (a) through (c) using the test data set when a preset number or more of the test data set is stored in the buffer memory. It begins.

[0027] As an example, a computing device is disclosed in which the processor, in step (I), refers to the learning jet context information to calculate the learning jet operation cost information including at least some of asset class information determined according to the degree of business impact when the operation of the security monitoring asset is interrupted, service dependency information determined according to the connection relationship of the security monitoring asset, active session information, and time-based weight information, and calculates the learning jet security benefit information including at least some of risk information, attack stage information, and asset vulnerability information for the security monitoring asset, and calculates each of the plurality of learning jet reality reward values ​​for each of the plurality of learning jet response actions by referring to the difference between the value obtained by multiplying the learning jet security benefit information and the corresponding learning jet profit weight and the value obtained by multiplying the learning jet operation cost information and the corresponding learning jet cost weight.

[0028] As an example, a computing device is disclosed in which the processor calculates each of the plurality of learning reality reward values ​​by setting the initial value of the learning cost weight in the process (I) such that it is greater than the initial value of the learning profit weight.

[0029] As an example, a computing device is disclosed in which the processor determines the asset class information by additionally referring to at least some of the number of other security control assets connected to the security control asset and the importance of the connection relationship of the security control asset in the process (I).

[0030] As an example, a computing device is disclosed in which, in the above process (I), each of the plurality of learning response actions includes at least some of the following: a first learning response action for changing the IP address of the security monitoring asset, a second learning response action for changing the port of the security monitoring asset, a third learning response action for creating and distributing a decoy corresponding to the security monitoring asset, a fourth learning response action for controlling a network path using a Software Defined Networking (SDN) controller, and a fifth learning response action for continuing monitoring.

[0031] As an example, a computing device is disclosed in which the processor, in the process (II), performs normalization on the learning jet context information, converts it into a vector value, and then inputs it into the Q-network.

[0032] As an example, a computing device is disclosed in which the processor, in the process (I), obtains the learning jet operation state information from at least some of a service dependency information generating device, an asset management database, a switch, a load balancer, and a SIEM (Security Information and Event Management), and obtains the learning jet security state information from at least some of a SOAR (Security Orchestration, Automation, and Response), an IDS (Intrusion Detection System), an IPS (Intrusion Prevention System), a firewall, an EDR (Endpoint Detection and Response), a CTI (Cyber ​​Threat Intelligence), and the SIEM. Effects of the invention

[0033] The present invention has the effect of training a Q-Network to enable decision-making while effectively neutralizing security-related attacks and minimizing cost losses associated with business operations.

[0034] In addition, the present invention has the effect of calculating security benefits and operating costs for a security monitoring asset through context information obtained from at least some of at least one security monitoring asset, and generating a target policy for learning a Q-Network by referring to a real reward value for each response action calculated by referring to the security benefits and operating costs, respectively, and a future profit. Brief explanation of the drawing

[0035] The drawings attached below for use in describing embodiments of the present invention are merely some of the embodiments of the present invention, and other drawings can be obtained based on these drawings without inventive work by a person skilled in the art to which the present invention pertains (hereinafter "person skilled in the art"). FIG. 1 schematically illustrates a computing device that learns to automate the response to security events according to an embodiment of the present invention, and FIG. 2 schematically illustrates a process flowchart for learning to automate the response to security events according to an embodiment of the present invention, and FIG. 3 illustrates in detail a process for learning to automate the response to security events according to an embodiment of the present invention, and FIG. 4 schematically illustrates a process for generating a GT to train a Q-Network according to an embodiment of the present invention, and FIG. 5 schematically illustrates a process for testing to automate the response to security events according to one embodiment of the present invention. Specific details for implementing the invention

[0036] The following detailed description of the present invention refers to the accompanying drawings, which illustrate specific embodiments in which the present invention can be practiced in order to clarify the objects, technical solutions, and advantages of the present invention. These embodiments are described in sufficient detail to enable a person skilled in the art to practice the present invention.

[0037] Furthermore, throughout the detailed description and claims of the invention, the word “comprising” and its variations are not intended to exclude other technical features, additions, components, or steps. Other objects, advantages, and characteristics of the invention will become apparent to a person skilled in the art, in part from this description and in part from the practice of the invention. The following examples and drawings are provided by way of example and are not intended to limit the invention.

[0038] Furthermore, the present invention encompasses all possible combinations of the embodiments set forth in this specification. It should be understood that various embodiments of the present invention are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in one embodiment without departing from the spirit and scope of the invention. It should also be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not intended to be limiting, and the scope of the invention is limited only by the appended claims, including all equivalents thereof, provided appropriately described. Similar reference numerals in the drawings refer to the same or similar functions across various aspects.

[0039] The title or summary of the invention provided herein is provided merely for convenience and is not intended to limit or interpret the scope or meaning of these embodiments.

[0040] For reference, even if each component below is listed in the singular, it does not exclude the possibility of multiple possibilities.

[0041] Hereinafter, in order to enable a person skilled in the art to easily practice the present invention, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

[0042] FIG. 1 schematically illustrates a computing device that learns to automate the response to security events according to one embodiment of the present invention.

[0043] As illustrated in FIG. 1, the computing device (1000) may include a memory (1100) for storing instructions for learning to automate the response to security events and a processor (1200) for learning to automate the response to security events in response to the instructions stored in the memory (1100). At this time, the computing device (1000) may include various computing devices such as a server, a PC (Personal Computer), a laptop, a workstation, a tablet, a mobile computer, a PDA / EDA, a mobile phone, a smartphone, an IoT device, etc.

[0044] Specifically, the computing device (1000) may achieve desired system performance by utilizing a combination of a computing device (e.g., a device that may include components of a computer processor, memory, storage, input device and output device, and other conventional computing devices; an electronic communication device such as a router, switch, etc.; an electronic information storage system such as a Network Attached Storage (NAS) and a Storage Area Network (SAN)) and computer software (i.e., instructions that enable the computing device to perform in a specific way), but is not limited thereto.

[0045] Additionally, the processor (1200) of the computing device (1000) may include hardware configurations such as a Micro Processing Unit (MPU) or a Central Processing Unit (CPU), a cache memory, and a data bus. Additionally, the computing device may further include software configurations such as an operating system and an application for a specific purpose.

[0046] However, this does not exclude the case where the computing device (1000) includes an integrated processor in which a medium processor and memory for implementing the present invention are integrated.

[0047] Meanwhile, the computing device (1000) may further include a context collection module (1300), a utility output module (1400), a DRL decision module (1500), and an MTD execution module (1600) in addition to the memory (1100) and processor (1200), and this is described in detail as follows.

[0048] First, the context collection module (1300) may be configured to obtain operational status information from an operational asset (110) that includes at least some of a service dependency information generating device (111), an asset management database (112), a switch (113), a load balancer (114), and SEIM (131, Security Information and Event Management) among the security monitoring assets (100) linked with the computing device (1000), and to obtain security status information from a security-related asset (120) that includes at least some of SIEM (131), SOAR (121, Security Orchestration, Automation, and Response), IDS (122, Intrusion Detection System), IPS (123, Intrusion Prevention System), a firewall (124), EDR (125, Endpoint Detection and Response), and CTI (126, Cyber ​​Threat Intelligence) among the security monitoring assets (100) linked with the computing device (1000). At this time, SIEM (131) may be applicable to both operational assets (110) and security assets (120), and examples included in operational assets (110) and security assets (120) are not limited thereto.

[0049] For example, a service dependency information generating device (111) generates and manages service dependency information that allows the data transmission flow between security monitoring assets (100) to be verified by displaying the connection relationships between security monitoring assets (100) in the form of a map, such as nodes, edges, and directions, and an asset management database (112) determines and manages asset grades according to the business importance of each security monitoring asset (100), and a switch (113) and a load balancer (114) can manage L4 layer information that controls the load by including information on the number of people or machines using the security monitoring assets (100), IP addresses, TCP and UDP port information, etc., and L7 layer information that controls the load by further referencing information such as URI, payload, HTTP header, and cookies to the L4 layer information. Additionally, the SIEM (131) may acquire and manage operational status information generated from a dependency information generating device (111), an asset management database (112), a switch (113), and a load balancer (114).

[0050] As another example, SOAR (121) may manage information regarding how deeply an attacker has penetrated and the response performed based on a playbook to a security attack by an attacker, IDS (122) and IPS (123) may manage information regarding the results of blocking performed according to a security attack by an attacker by monitoring network traffic, firewall (124) and EDR (125) may manage attack severity scores based on CVSS (Common Vulnerability Scoring System), CTI (126) may manage information regarding threat credibility due to a security attack by an attacker and risk classification information regarding the attacker, and SIEM (131) may manage information regarding threat credibility due to a security attack by an attacker and information regarding how deeply an attacker has penetrated.

[0051] Next, the utility calculation module (1400) is configured to obtain context information regarding a security monitoring asset (100), including operational status information and security status information, from the context collection module (1300), and to calculate a real reward value for each response action that the security monitoring asset (100) can perform by referring to the context information regarding the security monitoring asset (100). At this time, the real reward value calculated by the utility calculation module (1400) can be calculated by the following equation (1), and its meaning is explained as follows.

[0052] Equation (1)

[0053] Here, R represents the reality reward value for each corresponding action, and represents security benefit information, which is the degree to which security for a security monitoring asset (100) is strengthened when a specific response action is performed, and represents the profit weight corresponding to the security profit information, and represents operational cost information, such as the extent to which a business using a security monitoring asset (100) suffers damage when a specific response action is performed. This may refer to a cost weight corresponding to operational cost information. At this time, the utility calculation module (1400) may calculate a realistic reward value by setting the initial value of the cost weight to be greater than the initial value of the profit weight, and by appropriately tuning these weight values ​​while repeating the learning of the Q-Network to be explained later, it may find an optimal value that can prevent operational interruptions caused by business (i.e., minimize operational costs) while strengthening security (i.e., increase security benefits) when responding to security events generated from the security monitoring asset (100).

[0054] Additionally, the utility calculation module (1400) can calculate operational cost information by including at least some of the following: asset class information determined according to the degree of business impact when the operation of the security monitoring asset (100) is interrupted; service dependency information determined according to the connection relationship of the security monitoring asset (100); active session information including the number of active sessions and traffic information; and time-based weighting information. It can also calculate security benefit information by including at least some of the following: risk information, attack stage information, and asset vulnerability information for the security monitoring asset (100).

[0055] At this time, the utility calculation module (1400) can determine the asset grade as “Grade C” in cases where (i) the security monitoring asset (100) is implemented as a system isolated from the outside so that it does not affect business even if the operation of the security monitoring asset (100) is interrupted, such as a developer test server, a general bulletin board, or idle equipment; (ii) the asset grade can be determined as “Grade B” in cases where there is a substitute means available to minimize business impact even if the operation of the security monitoring asset (100) is interrupted, such as an internal messenger or a file sharing server; (iii) the asset grade can be determined as “Grade A” in cases where business is paralyzed and recovery may take some time because there is no substitute means, such as a web portal for security analysts, groupware / ERP, or an email server, when the operation of the security monitoring asset (100) is interrupted; and (iv) the asset grade can be determined as “Grade S” in cases where immediate loss of revenue may occur if the security monitoring asset (100) is interrupted, such as a SIEM for log collection, a customer information database, an authentication / login server, or a payment approval database.

[0056] Additionally, the utility calculation module (1400) may determine asset grade information by additionally referring to at least some of the number of other security monitoring assets connected to the security monitoring asset (100) and the importance of the connection relationship of the security monitoring asset (100). For example, if there are 50 other servers connected to the first server corresponding to the security monitoring asset (100), the asset grade may be determined as "S grade," and if there is 1 other server connected to the second server corresponding to the security monitoring asset (100), the asset grade may be determined as "C grade," and even if the number of other servers connected to the third server corresponding to the security monitoring asset (100) is small, if all traffic must pass through it, the asset grade may be determined as "A grade," but is not limited thereto.

[0057] Meanwhile, the DRL decision module (1500) can acquire context information obtained from the context collection module (1300) as state information and learn a Q-Network based on reinforcement learning to determine the optimal specific response action that can enhance security while minimizing operating costs from the state information.

[0058] Specifically, the DRL decision module (1500) inputs learning context information obtained from the context collection module (1300) into the Q-Network as learning current state information, causing the Q-Network to output each predicted value of the learning state-action value function for each learning response action when each learning response action that the security monitoring asset (100) can perform in the learning current state is performed, and when a specific learning response action to be performed by the security monitoring asset (100) is determined, the Q-Network may learn a policy to maximize the specific real-world reward value corresponding to the specific response action and the future return among the multiple real-world reward values ​​calculated by the utility calculation module (1400).

[0059] And, the DRL decision module (1500), when the Q-Network has completed its learning, inputs the test context information obtained from the context collection module (1300) into the Q-Network as test current state information, causing the Q-Network to output each test state-action value function prediction value for each test response action when each test response action that the security monitoring asset (100) can perform is performed in the test current state information, and can determine the test specific response action corresponding to the test specific state-action value function prediction value that corresponds to the largest value among each test state-action value function prediction value as the response action to be performed by the security monitoring asset (100).

[0060] Finally, the MTD execution module (1600) may support the security monitoring asset (100) in performing the response action determined through the DRL decision module (1500).

[0061] FIG. 2 schematically illustrates a process flowchart for learning to automate the response to security events according to one embodiment of the present invention.

[0062] First, when learning context information including learning operation state information and learning security state information is obtained from at least some of at least one security monitoring asset, the computing device (1000) can calculate learning operation cost information and learning security benefit information by referring to the learning context information, and calculate each of a plurality of learning reality reward values ​​for each of a plurality of learning response actions that the security monitoring asset can perform by referring to the learning operation cost information and learning security benefit information (S210).

[0063] At this point, regarding the S210 process, I will explain it with reference to Fig. 3 for ease of understanding.

[0064] FIG. 3 illustrates in detail a process for learning to automate the response to security events according to one embodiment of the present invention.

[0065] Referring to FIG. 3, a computing device (1000) can obtain learning context information including learning operational status information and learning security status information by obtaining learning context information from at least some of the service dependency information generating device (111), asset management database (112), switch (113), load balancer (114) and SIEM (131) among the security monitoring assets (100) as described in FIG. 1, and obtaining learning security status information from at least some of the SOAR (121), IDS (122), IPS (123), firewall (124), EDR (125), CTI (126) and SIEM (131).

[0066] Additionally, the computing device (1000) may calculate learning Jet operation cost information including at least some of asset grade information determined according to the degree of business impact when the operation of the security monitoring asset (100) is suspended, service dependency information determined according to the connection relationship of the security monitoring asset (100), active session information, and time-based weight information by referring to learning Jet operation status information among learning Jet context information, and may calculate learning Jet security benefit information including at least some of risk information, attack stage information, and asset vulnerability information for the security monitoring asset (100) by referring to learning Jet security status information among learning Jet context information. At this time, the computing device (1000) may determine asset grade information by additionally referring to at least some of the number of other security monitoring assets connected to the security monitoring asset (1000) and the importance of the connection relationship of the security monitoring asset (100). For reference, specific details regarding the operating cost information and security benefit information for the learning jet have been explained in FIG. 1, so a repetitive explanation will be omitted. However, while the operating cost information and security benefit information for the learning jet may be obtained directly from the security control asset (100), the operating cost information and security benefit information for the learning jet may also be obtained through a virtual simulation environment built for the convenience of data acquisition during initial learning, but is not limited thereto.

[0067] And, the computing device (1000) can calculate the learning_t_1 real reward value to the learning_t_j real reward value as each of the learning_t real reward values ​​for each of the learning_t response actions that the security control asset (100) can perform, by referring to the learning_t operation cost information and learning_t security benefit information, in order to generate a policy for the Q-Network (200) to perform reinforcement learning (RL), according to the equation (1) described above.

[0068] That is, the computing device (1000) can calculate the learning Z_1 real reward value to the learning Z_j real reward value by referring to the difference between the value obtained by multiplying the learning Z_t security benefit information and the corresponding learning Z_t benefit weight and the value obtained by multiplying the learning Z_t operation cost information and the corresponding learning Z_t cost weight. It can also calculate each of the multiple learning Z_t real reward values ​​by setting the initial value of the learning Z_t cost weight to be greater than the initial value of the learning Z_t benefit weight. Subsequently, as the learning of the Q-Network (200) described later is repeated, the computing device (1000) can find the optimal value that minimizes the operation cost of the security monitoring asset while strengthening security by tuning the learning Z_t cost weight and the learning Z_t benefit weight.

[0069] For example, when a computing device (1000) refers to learning Jet context information and confirms that the learning Jet operation cost information corresponds to an S-class asset with a very large number of users, and that the learning Jet security benefit information confirms that the S-class asset is under attack, it can be seen that the learning Jet_1 real reward value or the learning Jet_j real reward value differs depending on the response action that the security monitoring asset (100) can perform as follows.

[0070] For example, if the IP address is changed as a learning t_1 response action, the computing device (1000) can immediately defend against attacks, so the value obtained by multiplying the learning t_1 security benefit information and the learning t_1 benefit weight is calculated as 100, and the value obtained by multiplying the learning t_1 operation cost information and the learning t_1 cost weight is calculated as 1000 because the risk related to operation costs is very high due to the blocking of connections for a very large number of users, then the learning t_1 real reward value for changing the IP address can be calculated as -900.

[0071] As another example, if monitoring is continued as a corresponding action for learning t_2, the computing device (1000) calculates the value obtained by multiplying the learning t_2 security benefit information and the learning t_2 benefit weight as 0 because it merely maintains the current state as it is, and similarly calculates the value obtained by multiplying the learning t_2 operation cost information and the learning t_2 cost weight as 0 because no change occurs for the user, then the learning t_2 real reward value for continuing monitoring can be calculated as 0.

[0072] As another example, if a decoy distribution is performed to lure an attacker as a response action for learning t_3, the computing device (1000) can strengthen security by securing time to prepare while the attacker is lured to the generated decoy (i.e., fake server) and attacks, so the value obtained by multiplying the learning t_3 security benefit information and the learning t_3 benefit weight is calculated as 100, and assuming that the decoy distribution does not touch the server connected to the actual user and requires relatively low operating costs, the value obtained by multiplying the learning t_3 operating cost information and the learning t_3 cost weight is calculated as 10, the learning t_3 real reward value for decoy distribution can be calculated as 90.

[0073] However, the computing device (1000) does not perform only the three corresponding actions described in the above example, but each of the multiple corresponding actions for learning may include at least some of the following: a first corresponding action for learning that changes the IP address of the security monitoring asset (100), a second corresponding action for learning that changes the port of the security monitoring asset (100), a third corresponding action for learning that creates and distributes a decoy corresponding to the security monitoring asset (100), a fourth corresponding action for learning that controls a network path using an SDN (Software Defined Networking) controller, and a fifth corresponding action for learning that continues monitoring, but is not limited thereto.

[0074] And, referring to FIG. 2, a computing device (1000) inputs learning jet context information into a Q-Network as learning jet state information, so that when each of a plurality of learning jet response actions is performed by a security control asset according to the learning jet state information, the Q-Network outputs each of a plurality of learning jet state-action value function prediction values ​​that match each of a plurality of learning jet response actions (S220).

[0075] At this point, regarding the S220 process, I will explain it with reference to Fig. 3 for ease of understanding.

[0076] Referring to FIG. 3, a computing device (1000) may input the learning context information obtained in the previous S210 process into the Q-Network (200) as current state information at the learning time point (i.e., learning state information). For reference, the computing device (1000) may not input the learning context information directly into the Q-Network (200), but may instead perform normalization on the learning context information to have a value between 0 and 1, convert it into a vector value having a predetermined dimension, make it into a form that the Q-Network (200) can understand, and then input the converted vector value into the Q-Network (200).

[0077] Additionally, the computing device (1000) can cause the Q-Network (200) to output the predicted values ​​of

[0078] In summary, the Q-Network (200) is a deep learning-based model that receives learning context information (i.e., learning state information) as input and predicts the value of the learning response action (t_1) or learning response action (t_j) determined by referring to the reward immediately obtained when the learning response action (t_1) or learning response action (t_j) is performed in the learning state information and the expected value of the profit estimated to be obtained in the future. That is, the Q-Network (200) is configured to quantify and predict the value of the learning response action (t_1) or learning response action (t_j) in the learning state information, and reinforcement learning may be performed through a process in which the Q-Network (200) is optimized by learning the Q-Network (200) so that it can determine the optimal specific response action to be performed by the security monitoring asset among the output data of the Q-Network (200).

[0079] And, referring to FIG. 2, a computing device (1000) determines a specific learning response action to be performed by a security control asset among a plurality of specific learning response actions, obtains a specific learning reality reward value (the specific learning reality reward value is any one of a plurality of specific learning reality reward values) determined according to the specific learning response action, and obtains information on a (t+1) state for learning, and inputs the information on a (t+1) state for learning into a Q-Network so that when each of a plurality of specific learning response actions is performed by a security control asset in the information on a (t+1) state for learning, the Q-Network outputs each of a plurality of predicted values ​​for a (t+1) state-action value function that matches each of a plurality of specific learning response actions, and among the specific learning reward value, the learning rate, and each of the multiple predicted values ​​for a (t+1) state-action value function, the largest predicted value for a specific learning state-action value function and the corresponding A learning state-action value function GT is generated by referencing a discount rate, and the parameters of the Q-Network can be learned (S230) using a loss generated by referencing the learning specific state-action value function prediction corresponding to the learning specific corresponding action among the multiple learning state-action value function predictions and the learning state-action value function GT.

[0080] At this point, regarding the S230 process, I will explain it with reference to Fig. 3 for ease of understanding.

[0081] Referring to FIG. 3, a computing device (1000) can determine which of the learning-specific response actions to be performed by a security control asset among the learning-specific response actions to learning-specific response actions to each of the learning-specific response actions, from learning-specific response action to learning-specific response action, as exemplarily in FIG. 3, it is assumed that the learning-specific response action to learning-specific response action to learning is determined, and it should be understood that the learning-specific reward value and the learning-specific state-action value function prediction value to learning-specific response action are marked separately.

[0082] Specifically, the computing device (1000), when it is determined by an ε-greedy policy that it corresponds to a first action determination mode based on a first probability (e.g., which may be ε, and which has a value between 0 and 1) determined by referencing the ε value in the learning state information, randomly determines one of the learning state_1 corresponding action to the learning state_j corresponding action as a learning state specific corresponding action as each of the plurality of learning state-action corresponding actions, and when it corresponds to a second action determination mode based on a second probability determined by referencing the ε value in the learning state information (the sum of the first probability and the second probability corresponds to 1, and may be 1-ε), determines the largest value among the learning state_1 state-action value function prediction value to the learning state_j state-action value function prediction value (in the example of FIG. 3, it corresponds to the learning state_1 state-action value function prediction value) as each of the plurality of learning state-action value function prediction values ​​as the learning state specific It is determined as a state-action value function prediction, and the corresponding action corresponding to the learning specific state-action value function prediction (i.e., the learning specific action t_1) can be determined as the learning specific action t_1.

[0083] This is because when the agent performing the action (i.e., the security monitoring asset) performs only the learning specific response action corresponding to the learning specific state-action value function prediction value where the Q-value is maximized, it may miss the opportunity to find a better Q-value. Therefore, to prevent this, if it is determined that it corresponds to the first action decision mode by the first probability (ε) determined by referencing the ε value, the learning specific response action may be randomly determined regardless of the output of the Q-Network (200). However, the ε value is determined as a predetermined value between 0 and 1. In the early stages of learning, the ε value may have a value close to 1, and as learning is repeated, the ε value may converge to 0, so that the learning specific response action is randomly determined mainly through the first action decision mode in the early stages of learning.

[0084] In addition, as learning is repeated, the second probability (1-ε) determined by referring to the ε value converges to 1, so the possibility of corresponding to the second action decision mode may gradually increase due to the second probability. At this time, if it is determined that it corresponds to the second action decision mode, the learning t_1 corresponding action corresponding to the largest learning t_1 specific state-action value function prediction among the learning t_1 state-action value function predictions and learning t_j state-action value function predictions as multiple learning t_1 state-action value function predictions output through the Q-Network (200) (in the example of FIG. 3, it corresponds to the learning t_1 state-action value function prediction) can be determined as the learning t_1 specific corresponding action.

[0085] Additionally, a specific reality reward value corresponding to a specific response action for learning can also be determined. However, if the computing device (1000) uses the specific reality reward value itself as a target policy for learning the Q-Network (200), the Q-Network (200) will be trained to follow short-term results, so a problem may arise in that it is difficult to determine which response action can be the optimal response action from a long-term perspective.

[0086] For example, it can be seen that when a computing device (1000) determines, by referring to learning context information, that an attacker is attempting an Advanced Persistent Threat (APT) attack to infiltrate a security monitoring asset, short-term and long-term results may differ depending on the following response action selection.

[0087] For example, if a computing device (1000) decides to change its IP address as a specific response behavior for learning, (i) in the short term, it can immediately defend against attacks, so it can obtain +10 as a real reward value for learning, and (ii) in the long term, it can detect that the IP address has been changed after 5 minutes and attempt a stronger attack, and because the connection of connected users is frequently interrupted due to frequent IP address changes, complaints about this increase and operating costs increase, so it can obtain -100 as future revenue.

[0088] As another example, if the computing device (1000) decides to deploy a decoy as a specific response action for learning, (i) in the short term, the attack is not immediately defended, but a small resource cost is required, so +5 can be obtained as a learning reality reward value, and (ii) in the long term, while the hacker attacks the decoy server, security for the actual server can be strengthened, so a response is possible without interruption of service, and thus +100 can be obtained as future profit.

[0089] To summarize, from a short-term perspective, changing the IP address can be seen as the optimal response, but from a long-term perspective, deploying decoys can be seen as the optimal response. Therefore, it is necessary to generate a learning state-action value function GT as a goal policy that supports Q-Network (200) in determining a specific response action for learning that is beneficial to the security monitoring asset from a long-term perspective, and this will be explained with reference to FIG. 4.

[0090] FIG. 4 schematically illustrates a process for generating a GT to train a Q-Network according to one embodiment of the present invention.

[0091] Referring to FIG. 4, when a learning specific response action is determined, a computing device (1000) can obtain a learning specific reality reward value and learning (t+1) state information determined according to the learning specific response action. At this time, the learning specific reality reward value corresponds to any one of a plurality of learning specific reality reward values, and the learning (t+1) state information may correspond to state information that appears at the next point in time as a result of the learning specific response action being performed in the learning state information.

[0092] Additionally, the computing device (1000) inputs learning (t+1) state information into the Q-Network (200) so that the Q-Network (200) outputs each of the multiple learning (t+1) state-action value function predictions that match each of the multiple learning (t+1) response actions when each of the multiple learning (t+1) response actions is performed by the security control asset in the learning (t+1) state information. At this time, each of the multiple learning (t+1) response actions may be identical to each of the multiple learning (t+1) response actions, but is not limited thereto.

[0093] And, the computing device (1000) can generate a learning (t+1) state-behavior value function GT by referring to the largest learning (t+1) state-behavior value function prediction among a learning (t+1) state-behavior value function prediction, a learning rate, and a discount rate for each of the multiple learning (t+1) state-behavior value function predictions. For example, in FIG. 4, among the learning (t+1)_1 state-behavior value function predictions and the learning (t+1)_j state-behavior value function predictions, the learning (t+1)_j state-behavior value function prediction may be determined as the learning (t+1) specific state-behavior value function prediction.

[0094] For reference, the learning state-action value function GT as a target policy for learning the Q-Network (200) can be expressed as Equation (2) below to approximate the optimal Q-value calculated by the Bellman optimality equation.

[0095] (2)

[0096] In the above formula, represents the training state-action value function GT, and represents the predicted value of a specific state-action value function for training, and represents the learning rate for the Q-Network (200), and represents the discount rate that determines how much of future earnings will be reflected, and 0.9 is generally used, and t refers to a specific reality reward value for training, can represent the predicted value of the specific state-action value function for training at (t+1). In this case, if the learning rate is low, the previous Q-value may be updated slightly, and if the learning rate is high, the previous Q-value may be updated by significantly reflecting the learning result of the Q-Network (200). As an example, if the learning rate is 1, it can be modified as shown in Equation (3) below.

[0097] (3)

[0098] To summarize, the computing device (1000) inputs the learning (t+1) state information into the Q-Network (200) to calculate the optimal Q-value in the learning (t+1) state information according to the above Bellman optimal equation, and the Q-Network (200) may calculate the learning (t+1)_1 state-action value function prediction or the learning (t+1)_j state-action value function prediction as the multiple learning (t+1) state-action value function prediction when each of the multiple learning (t+1) corresponding actions is performed in the learning (t+1) state information. And, since the computing device (1000) can determine the largest learning (t+1)_j state-behavior value function prediction among the learning (t+1)_1 state-behavior value function prediction and the learning (t+1)_j state-behavior value function prediction as the learning (t+1) specific state-behavior value function prediction, the learning (t+1) specific state-behavior value function prediction, the learning (t+1) specific reality reward value, the discount rate, and the learning rate are applied to Equation (2) and Equation (3) to calculate This makes it possible to generate the training state-action value function GT.

[0099] And, referring to FIG. 3, a computing device (1000) inputs a learning specific state-action value function prediction (e.g., learning specific state-action value function prediction) and a learning specific state-action value function GT corresponding to a learning specific corresponding action (e.g., learning specific state-action corresponding action) among a plurality of learning specific state-action value function predictions, to a loss layer (300), and causes the loss layer (300) to generate a loss by referencing the learning specific state-action value function prediction and the learning specific state-action value function GT, and can learn the parameters of the Q-Network (200) using the generated loss.

[0100] In this way, when the Q-Network (200) is optimized by repeatedly performing learning on the Q-Network (200), the computing device (1000) can support determining and performing the optimal response action that a security monitoring asset can perform using the Q-Network (200) through a test process, which will be explained with reference to FIG. 5.

[0101] FIG. 5 illustrates in detail a process for testing to automate the response to security events according to one embodiment of the present invention.

[0102] Referring to FIG. 5, the parameters of the Q-network (200) may be optimized as the process described in FIG. 2 to FIG. 4 is repeated.

[0103] Subsequently, when test element context information including test element operational state information and test element security state information is obtained from a security monitoring asset, the computing device (1000) inputs the test element context information as test element state information to the Q-Network (200), thereby causing the Q-Network (200) to output each of the multiple test element state-action value function prediction values ​​that match each of the multiple test element response actions when each of the multiple test element response actions is performed by the security monitoring asset (100) according to the test element state information. For reference, the process of obtaining test element operational state information and test element security state information is identical / similar to that described above in FIG. 1, so a description thereof will be omitted.

[0104] Additionally, the computing device (100) can determine a specific corresponding action for the test that corresponds to the largest specific state-action value function prediction among each of the multiple predicted state-action value functions for the test m. For example, FIG. 5 corresponds to an example where the predicted state-action value function for the test m_j state-action value function among the predicted state-action value function for the test m_1 state-action value function to the predicted state-action value function for the test m_j state-action value function is determined as the specific state-action value function prediction for the test m. Accordingly, the computing device (1000) can determine the corresponding action for the test m_j state-action value function that matches the predicted state-action value function for the test m_j state-action value function as the specific corresponding action for the test m.

[0105] And, when a specific response action for testing is determined, the computing device (1000) can cause the security monitoring asset (100) to perform a response to the context information for testing according to the specific response action for testing. That is, in automating the response to security events, by using reinforcement learning with the Q-Network (200), the security monitoring asset (100) can determine an optimized response action that strengthens security while minimizing necessary operating costs and respond to context data, thereby inducing the correct response to be performed.

[0106] Meanwhile, the computing device (1000) may support the Q-Network (200) in determining a more optimized response action by storing data generated or acquired in a buffer memory (400) while performing a test process using the Q-Network (200), and by using this data for additional learning.

[0107] For example, a computing device (1000) may calculate test operation cost information and test security benefit information by referring to test context information, and obtain a test specific reality reward value corresponding to a test specific response action among a plurality of test response actions that a security monitoring asset (100) can perform by referring to test operation cost information and test security benefit information. However, since the process of obtaining the test specific reality reward value is identical / similar to the process of obtaining the learning specific reality reward value described in FIG. 3, the description thereof will be omitted.

[0108] Additionally, the computing device (1000) may acquire test state information (m+1) determined according to test m specific response behavior determined based on the output data of the Q-Network (200), and may store a test m data set including test m context information, test m specific response behavior, test m specific reality reward value, and test m (m+1) state information in the buffer memory (400). Furthermore, if the computing device (1000) determines that more than a preset number of test m data sets are stored in the buffer memory (400), it may perform fine-tuning of the Q-network (200) by performing retraining on the Q-Network (200) using the test m data set according to the process described in FIGS. 2 to 4 after terminating the response to the security monitoring asset (100) through the test process.

[0109] Furthermore, the embodiments according to the present invention described above may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specifically designed and configured for the present invention, or they may be those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.

[0110] Although the present invention has been described above with specific details such as specific components, limited embodiments, and drawings, this is provided only to aid in a more comprehensive understanding of the invention, and the invention is not limited to the above embodiments, and a person skilled in the art to which the invention belongs can make various modifications and variations from this description.

[0111] Accordingly, the scope of the present invention should not be limited to the embodiments described above, and all modifications equivalent to or equivalent to the claims set forth below, as well as the claims described below, shall be considered to fall within the scope of the concept of the present invention.

Claims

Claim 1 A method for learning to automate the response to security events, comprising: (a) when learning context information including learning Z-t operational state information—wherein t is an integer greater than or equal to 1—and learning Z-t security state information is obtained from at least some of at least one security monitoring asset, a computing device calculates learning Z-t operational cost information and learning Z-t security benefit information by referring to the learning Z-t context information, and calculates each of a plurality of learning Z-t reality reward values ​​for each of a plurality of learning Z-t response actions that the security monitoring asset can perform by referring to the learning Z-t operational cost information and the learning Z-t security benefit information; (b) the computing device inputs the learning Z-t context information into a Q-Network as learning Z-t state information, thereby causing the Q-Network to output each of a plurality of learning Z-t state-action value function predictions that match each of the plurality of learning Z-t response actions when each of the plurality of learning Z-t response actions is performed by the security monitoring asset according to the learning Z-t state information;and (c) the computing device determines a specific learning response action to be performed by the security control asset among each of the plurality of specific learning response actions, and obtains a specific learning reality reward value determined according to the specific learning response action—the specific learning reality reward value is any one of the plurality of specific learning reality reward values—and learning state information (t+1), and inputs the learning state information (t+1) to the Q-Network so that when each of the plurality of specific learning response actions (t+1) is performed by the security control asset in the learning state information (t+1), the Q-Network outputs each of the multiple learning state-action value function prediction values ​​that match each of the plurality of learning response actions (t+1), and the largest learning state-action value function prediction value among the specific learning reward value, the learning rate, and each of the multiple learning state-action value function prediction values. A method comprising the step of generating a learning jet state-action value function GT by referencing a discount rate therefor, and learning the parameters of the Q-Network using a loss generated by referencing the learning jet specific state-action value function prediction value corresponding to the learning jet specific corresponding action among each of the plurality of learning jet state-action value function prediction values ​​and the learning jet state-action value function GT. Claim 2 A method according to claim 1, wherein in step (c), the computing device, by an ε-greedy policy, randomly determines the learning-specific corresponding action among each of the plurality of learning-specific corresponding actions when corresponding to a first action decision mode, and when corresponding to a second action decision mode, determines the largest value among each of the plurality of learning-specific state-action value function prediction values ​​as the learning-specific state-action value function prediction value, and determines the corresponding action corresponding to the learning-specific state-action value function prediction value as the learning-specific corresponding action. Claim 3 In claim 1, the method further comprises the step of: (d) after optimizing the parameters of the Q-Network by repeating steps (a) through (c), when test context information including test m operational state information - where m is an integer greater than or equal to 1 - and test m security state information is obtained from the security monitoring asset, the computing device inputs the test context information as test m state information into the Q-Network, thereby causing the Q-Network to output each of a plurality of test m state-action value function prediction values ​​that match each of a plurality of test m response actions when each of a plurality of test m response actions is performed by the security monitoring asset according to the test m state information, determining a test m specific response action corresponding to the largest test m specific state-action value function prediction value among the plurality of test m state-action value function prediction values, and causing the security monitoring asset to perform a response to the test context information according to the test m specific response action. method. Claim 4 In paragraph 3, the method further comprises the step of: (e) the computing device calculating test operation cost information and test security benefit information by referring to test context information; obtaining a specific reality reward value corresponding to a specific response action among a plurality of test response actions that can be performed by the security control asset by referring to test operation cost information and test security benefit information; obtaining test (m+1) state information determined according to the specific response action; storing a test data set including test context information, test specific response action, test specific reality reward value, and test (m+1) state information in a buffer memory; and when a preset number or more of test data sets are stored in the buffer memory, performing fine-tuning of the Q-network by performing steps (a) through (c) using the test data set. Claim 5 A method according to claim 1, wherein in step (a), the computing device calculates the learning jet operation cost information including at least some of asset class information determined according to the degree of business impact when the operation of the security monitoring asset is interrupted, service dependency information determined according to the connection relationship of the security monitoring asset, active session information, and time-based weighting information by referring to the learning jet context information, calculates the learning jet security benefit information including at least some of risk information, attack phase information, and asset vulnerability information for the security monitoring asset, and calculates each of the plurality of learning jet reality reward values ​​for each of the plurality of learning jet response actions by referring to the difference between the value obtained by multiplying the learning jet security benefit information and the corresponding learning jet profit weight and the value obtained by multiplying the learning jet operation cost information and the corresponding learning jet cost weight. Claim 6 A method according to claim 5, wherein in step (a) above, the computing device calculates each of the plurality of learning reality reward values ​​by setting the initial value of the learning cost weight to be greater than the initial value of the learning profit weight. Claim 7 In claim 5, the method is characterized in that, in step (a), the computing device determines the asset class information by additionally referring to at least some of the number of other security control assets connected to the security control asset and the importance of the connection relationship of the security control asset. Claim 8 A method according to claim 1, wherein in step (a), each of the plurality of learning response actions includes at least some of the following: a learning first response action for changing the IP address of the security monitoring asset; a learning second response action for changing the port of the security monitoring asset; a learning third response action for creating and distributing a decoy corresponding to the security monitoring asset; a learning fourth response action for controlling a network path using an SDN (Software Defined Networking) controller; and a learning fifth response action for continuing monitoring. Claim 9 A method according to claim 1, wherein in step (b), the computing device performs normalization on the learning jet context information, converts it into a vector value, and then inputs it into the Q-network. Claim 10 A method according to claim 1, wherein, in step (a), the computing device obtains the learning jet operational status information from at least some of a service dependency information generating device, an asset management database, a switch, a load balancer, and a SIEM (Security Information and Event Management), and obtains the learning jet security status information from at least some of a SOAR (Security Orchestration, Automation, and Response), an IDS (Intrusion Detection System), an IPS (Intrusion Prevention System), a firewall, an EDR (Endpoint Detection and Response), a CTI (Cyber ​​Threat Intelligence), and the SIEM. Claim 11 A computing device for learning to automate the response to security events comprises: at least one memory for storing instructions; and at least one processor configured to execute said instructions; wherein the processor comprises: (I) when learning Z-t context information including learning Z-t operational state information, wherein t is an integer greater than or equal to 1, and learning Z-t security state information is obtained from at least some of at least one security monitoring asset, the processor calculates learning Z-t operational cost information and learning Z-t security benefit information by referring to said learning Z-t context information, and calculates each of a plurality of learning Z-t reality reward values ​​for each of a plurality of learning Z-t response actions that the security monitoring asset can perform by referring to said learning Z-t operational cost information and said learning Z-t security benefit information; (II) A process of inputting the above-mentioned learning jet context information into a Q-Network as learning jet state information, so that when each of the above-mentioned learning jet response actions is performed by the above-mentioned security control asset according to the above-mentioned learning jet state information, the Q-Network outputs each of the above-mentioned learning jet state-action value function predictions that match each of the above-mentioned learning jet response actions;and (III) determine a specific learning response action to be performed by the security control asset among each of the plurality of specific learning response actions, and obtain a specific learning reality reward value determined according to the specific learning response action—the specific learning reality reward value is any one of the plurality of specific learning reality reward values—and learning state information (t+1), and input the learning state information (t+1) into the Q-Network so that when each of the plurality of learning response actions (t+1) is performed by the security control asset in the learning state information, the Q-Network outputs each of the multiple learning state-action value function predictions that match each of the plurality of learning response actions (t+1), and the largest learning state-action value function prediction among the specific learning reward value, the learning rate, and the discount rate for the largest learning state-action value function prediction among the multiple learning state-action value function predictions. A computing device that performs a process of generating a learning jet state-action value function GT by reference, and learning parameters of the Q-Network using a loss generated by referencing the learning jet specific state-action value function prediction value corresponding to the learning jet specific corresponding action among each of the plurality of learning jet state-action value function prediction values, and the learning jet state-action value function GT. Claim 12 A computing device according to claim 11, wherein the processor, in the process (III), randomly determines the learning-specific corresponding action among each of the plurality of learning-specific corresponding actions when the first action decision mode is in accordance with the ε-greedy policy, and when the second action decision mode is in accordance with the second action decision mode, determines the largest value among each of the plurality of learning-specific state-action value function prediction values ​​as the learning-specific state-action value function prediction value, and determines the corresponding action corresponding to the learning-specific state-action value function prediction value as the learning-specific corresponding action. Claim 13 In claim 11, the processor further performs a process in which, after optimizing the parameters of the Q-Network by repeating the processes (I) through (III), if test context information including test m operational state information - where m is an integer greater than or equal to 1 - and test m security state information is obtained from the security monitoring asset, the processor inputs the test context information as test m state information to the Q-Network, thereby causing the Q-Network to output each of a plurality of test m state-action value function prediction values ​​that match each of a plurality of test m response actions when each of a plurality of test m response actions is performed by the security monitoring asset according to the test m state information, determines a test m specific response action corresponding to the largest test m specific state-action value function prediction value among the plurality of test m state-action value function prediction values, and causes the security monitoring asset to perform a response to the test m context information according to the test m specific response action. Computing device. Claim 14 In claim 13, the processor further performs the process of: (V) calculating test operation cost information and test security benefit information by referring to test context information; obtaining a test specific reality reward value corresponding to test specific response action among a plurality of test response actions that can be performed by the security monitoring asset by referring to test operation cost information and test security benefit information; obtaining test (m+1) state information determined according to test specific response action; storing a test data set including test context information, test specific response action, test specific reality reward value, and test (m+1) state information in a buffer memory; and when a preset number or more of test data sets are stored in the buffer memory, performing fine-tuning of the Q-network by executing processes (I) through (III) using test data sets. Device. Claim 15 A computing device characterized in that, in claim 11, the processor, in the process (I), calculates the learning jet operation cost information including at least some of asset class information determined according to the degree of business impact when the operation of the security monitoring asset is interrupted, service dependency information determined according to the connection relationship of the security monitoring asset, active session information, and time-based weighting information by referring to the learning jet context information, calculates the learning jet security benefit information including at least some of risk information, attack phase information, and asset vulnerability information for the security monitoring asset, and calculates each of the plurality of learning jet reality reward values ​​for each of the plurality of learning jet response actions by referring to the difference between the value obtained by multiplying the learning jet security benefit information and the corresponding learning jet profit weight and the value obtained by multiplying the learning jet operation cost information and the corresponding learning jet cost weight. Claim 16 A computing device according to claim 15, wherein the processor, in the process (I), sets the initial value of the learning cost weight to be greater than the initial value of the learning profit weight, and calculates each of the plurality of learning reality reward values. Claim 17 A computing device according to claim 15, characterized in that the processor determines the asset class information by additionally referring to at least some of the number of other security control assets connected to the security control asset and the importance of the connection relationship of the security control asset in the process (I). Claim 18 A computing device characterized in that, in the above process (I), each of the plurality of learning response actions includes at least some of the following: a learning first response action for changing the IP address of the security monitoring asset, a learning second response action for changing the port of the security monitoring asset, a learning third response action for creating and distributing a decoy corresponding to the security monitoring asset, a learning fourth response action for controlling a network path using an SDN (Software Defined Networking) controller, and a learning fifth response action for continuing monitoring. Claim 19 A computing device according to claim 11, characterized in that the processor, in the process (II), performs normalization on the learning context information and converts it into a vector value, and then inputs it into the Q-network. Claim 20 A computing device according to claim 11, wherein the processor, in the process (I), obtains the learning jet operational state information from at least some of a service dependency information generating device, an asset management database, a switch, a load balancer, and a SIEM (Security Information and Event Management), and obtains the learning jet security state information from at least some of a SOAR (Security Orchestration, Automation, and Response), an IDS (Intrusion Detection System), an IPS (Intrusion Prevention System), a firewall, an EDR (Endpoint Detection and Response), a CTI (Cyber ​​Threat Intelligence), and the SIEM.

Images