A method and system for alarm research and judgment and empowerment optimization based on reinforcement adaptation

Abstract

The application provides a method and system for alarm research and judgment and empowerment optimization based on reinforcement adaptation, which comprises constructing an intelligent research and judgment model, modeling the environment of the intelligent research and judgment model, constructing a network attack path according to the intelligent research and judgment model, and identifying a network attack by using a known signature of the network attack or a rule-based detection technology; classifying and labeling data of a network security device by using an SVM classification algorithm and training the intelligent research and judgment model, establishing a threat identification strategy of the network attack, and identifying features of the network attack; performing context learning and state prediction by using the intelligent research and judgment model, and optimizing the threat identification strategy of the network attack; generating adversarial samples by using known threats of the network attack through slight perturbation of data fields of normal HTTP requests, and performing adversarial training in combination with fake HTTP requests to identify deceptive network attacks. The application realizes cross verification and correlation analysis of high-quality and multi-source threat intelligence, and improves attack defense capability.

Description

Technical Field

[0001] This invention relates to the fields of computer technology and network security technology, specifically to a method and system for alarm analysis and empowerment optimization based on reinforcement and adaptive methods. Background Technology

[0002] Traditional cyberattack assessment techniques primarily rely on anomaly detection models and predefined rule-based feature recognition to identify potential cybersecurity threats. This approach has limited detection capabilities for emerging or variant threats because it depends on timely updates to threat intelligence and attack event information to design appropriate assessment thresholds. This reliance prevents the system from being updated in real-time to detect new threats or predict attack variants, thus impacting the speed and accuracy of response to novel cyberattacks.

[0003] Furthermore, traditional assessment techniques typically have a high false positive rate, frequently misidentifying normal behavior as anomalous behavior or failing to accurately distinguish between real and non-threat activities. False positives not only consume valuable time and resources of security teams but can also lead to the overlooking of genuine threats. Regular management and analysis by professionals are necessary to reduce false positives and false negatives and improve the accuracy of assessments. However, such management and analysis are time-consuming and require a high level of expertise, significantly limiting the efficiency and scalability of the assessment process.

[0004] To address the aforementioned issues, this application proposes a method and system for alarm analysis and optimization based on enhanced adaptive features. Summary of the Invention

[0005] This application addresses one or more technical deficiencies in the prior art by proposing the following technical solution.

[0006] Based on the first aspect of this application, a method for alarm judgment and empowerment optimization based on reinforcement and adaptation is proposed, including:

[0007] S1: Establish the architecture of the intelligent judgment model based on the enterprise's internal IT infrastructure and application systems, obtain the data flow of the architecture components, the user's access path and the data of the network security devices, and perform environmental modeling on the intelligent judgment model based on the data flow, the access path and the data of the network security devices to obtain the intelligent judgment model;

[0008] S2: Construct network attack paths based on the intelligent analysis model, and match the data of the network security device using known signatures of network attacks or rule-based detection techniques;

[0009] S3: using the SVM classification algorithm to classify and label the data of the network security device, training the intelligent research and judgment model using the data of the network security device, establishing a threat identification strategy of the network attack in combination with the network attack path, and identifying the network attack;

[0010] S4: performing context learning and state prediction on the intelligent research and judgment model by using the Q-learning method, the SGD incremental learning method, and the Softmax method, analyzing the attack mode and attack event in the network historical data, performing parameter fine-tuning on the intelligent research and judgment model in combination with the analysis result and the characteristics of the network attack, optimizing the threat identification strategy of the network attack, and enhancing the generalization ability and autonomous correlation reasoning ability of the intelligent research and judgment model;

[0011] S5: generating an adversarial sample by slightly perturbing the data field of a normal HTTP request, obtaining a fake HTTP request using the known threat of the network attack, and performing adversarial training on the intelligent research and judgment model in combination with the normal HTTP request, the fake HTTP request, the characteristics of the network attack, and the adversarial sample, to obtain a final intelligent research and judgment model to identify fraudulent network attacks.

[0012] Further, the architecture of the intelligent research and judgment model includes a user terminal device, an internal network, a database server, an application server, a Web server, a firewall, an IDS / IPS, and a log server.

[0013] Further, the data of the network security device includes network traffic data, system logs, endpoint data, security event data, and user behavior data.

[0014] The network traffic data includes data in captured data packets.

[0015] The system logs include operating system logs, application program logs, Web server logs, firewall logs, IDS / IPS logs, and database logs.

[0016] The endpoint data includes behavior data of terminal devices, including process information, file operations, and registry changes.

[0017] The user behavior data includes login data, access path data, and operation behavior data of users.

[0018] The security event data includes IDS / IPS alert data and antivirus software scan result data.

[0019] Further, the network attack path includes:

[0020] An attacker invades a web server through a network, enters an internal application server through a vulnerability of the web server, and obtains access to an internal application database;

[0021] An attacker infects an internal device through malware, spreads the infection through an internal network, and attacks an internal database server;

[0022] An attacker obtains system management permissions by stealing an administrator account, modifies a database, and steals sensitive information.

[0023] According to the intelligent research and judgment model, the network attack path can be constructed to mine the network attack behavior that has occurred, improve the emergency response speed, reduce the continuous damage of the attacker to the system, shorten the network attack recovery time, and improve the overall security level.

[0024] Further, analyzing the attack patterns and attack events in the network history data specifically includes:

[0025] Extracting attack types, attack intensities, and attack durations from the network traffic data, and then using cosine similarity to calculate the similarity between different attack patterns to analyze the association between different attack patterns;

[0026] Using an ARIMA time series analysis algorithm to model the time series of attack events to predict future network attack trends;

[0027] Classifying attack sources, and then using IP address resolution methods and DNS query methods to determine the geographic location of the attack sources to analyze the relationship between attack events and attack sources.

[0028] Further, the fake HTTP request includes:

[0029] Faking the User-Agent header in the HTTP request to simulate normal browser identification and the legal user source of the HTTP request;

[0030] Faking the Rerferer header in the HTTP request to simulate the jump page of the HTTP request;

[0031] Faking the Authorization header in the HTTP request to simulate the legal identity of the user to bypass identity verification.

[0032] Further, the known signatures of the network attack include the hash value of the malware, the specific pattern of SQL injection, and the abnormal packet format of the specific protocol.

[0033] Conducting adversarial training on the intelligent research and judgment model in combination with normal HTTP requests and fake HTTP requests helps the intelligent research and judgment model learn how to distinguish between normal HTTP requests and fake HTTP requests, accurately identify deceptive network attacks, and thus effectively defend against attacks.

[0034] Based on the second aspect of the present application, a system for alarm research and judgment and empowerment optimization based on reinforcement adaptation is proposed, which comprises:

[0035] A model construction module: establishes the architecture of an intelligent research and judgment model according to the IT infrastructure and application systems within an enterprise, obtains data streams of architecture components, user access paths, and network security device data, and performs environment modeling on the intelligent research and judgment model according to the data streams, the access paths, and the network security device data to obtain the intelligent research and judgment model.

[0036] A network attack path module: constructs a network attack path according to the intelligent research and judgment model, and matches the network security device data using known signatures of network attacks or rule-based detection techniques.

[0037] An identification module: classifies and labels the network security device data using an SVM classification algorithm, trains the intelligent research and judgment model using the network security device data, establishes a threat identification strategy for the network attack in combination with the network attack path, and identifies network attacks.

[0038] An optimization module: performs context learning and state prediction on the intelligent research and judgment model using a Q-learning method, an SGD incremental learning method, and a Softmax method, analyzes attack patterns and attack events in network historical data, fine-tunes the parameters of the intelligent research and judgment model in combination with the analysis results and the features of the network attack, optimizes the threat identification strategy for the network attack, and enhances the generalization ability and autonomous correlation reasoning ability of the intelligent research and judgment model.

[0039] A re-optimization module: performs a small perturbation on the data fields of normal HTTP requests, generates adversarial samples using known threats of the network attack, obtains fake HTTP requests, conducts adversarial training on the intelligent research and judgment model in combination with normal HTTP requests, fake HTTP requests, features of the network attack, and the adversarial samples, and obtains a final intelligent research and judgment model to identify deceptive network attacks.

[0040] Based on the third aspect of the present application, a computer program product is proposed, which has one or more computer programs thereon, when the computer programs are executed by a computer processor, the method as described in any one of the above aspects is implemented.

[0041] The technical effect of the present application is that the present application introduces an alarm research and judgment based on a safe large model, intelligently analyzes the alarms reported by the security equipment, constructs an intelligent research and judgment model, trains and optimizes the model in combination with reinforcement learning and interaction with the environment, continuously improves the analysis capability of the research and judgment model, continuously enables and optimizes the intelligent research and judgment work when facing the changing environment or conditions, adapts to new attacks and attack strategy variants, improves the alarm analysis efficiency and research and judgment accuracy based on the cross-validation and correlation analysis of high-quality and multi-source threat intelligence, ensures that important security events are not overwhelmed by massive alarms, and improves the active defense capability of network attacks and threats. BRIEF DESCRIPTION OF DRAWINGS

[0042] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments made with reference to the accompanying drawings.

[0043] Figure 1 is a method flowchart of alarm research and judgment and empowerment optimization based on reinforcement adaptation according to an embodiment of the present application.

[0044] Figure 2 is a system framework diagram of alarm research and judgment and empowerment optimization based on reinforcement adaptation according to an embodiment of the present application.

[0045] Figure 3 is a structural diagram of an electronic device suitable for implementing a computer system according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related application, and not to limit the application. In addition, it should be noted that only parts related to the application are shown in the drawings for ease of description.

[0047] It should be noted that the embodiments and features in the embodiments in the present application can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and embodiments.

[0048] Figure 1 A method of alarm research and judgment and empowerment optimization based on reinforcement adaptation is shown, comprising:

[0049] S1: establishing an architecture of an intelligent research and judgment model according to the IT infrastructure and application system within an enterprise, obtaining data flow of architecture components, access path of users and data of network security equipment, and performing environment modeling on the intelligent research and judgment model according to the data flow, the access path and the data of the network security equipment, to obtain the intelligent research and judgment model;

[0050] S2: constructing a network attack path according to the intelligent research and judgment model, and matching data of the network security device using a known signature of the network attack or a rule-based detection technology;

[0051] S3: classifying and labeling data of the network security device using an SVM classification algorithm, training the intelligent research and judgment model using the data of the network security device, establishing a threat identification strategy of the network attack in combination with the network attack path, and identifying the network attack;

[0052] S4: performing context learning and state prediction on the intelligent research and judgment model through a Q-learning method, an SGD incremental learning method, and a Softmax method, analyzing attack patterns and attack events in network historical data, performing parameter fine-tuning on the intelligent research and judgment model in combination with an analysis result and features of the network attack, optimizing the threat identification strategy of the network attack, and enhancing generalization ability and autonomous correlation reasoning ability of the intelligent research and judgment model;

[0053] S5: generating an adversarial sample using known threats of the network attack by slightly perturbing data fields of a normal HTTP request, obtaining a fake HTTP request, performing adversarial training on the intelligent research and judgment model in combination with the normal HTTP request, the fake HTTP request, features of the network attack, and the adversarial sample, and obtaining a final intelligent research and judgment model to identify deceptive network attacks.

[0054] It should be noted that the architecture of the intelligent research and judgment model includes a user terminal device, an internal network, a database server, an application server, a Web server, a firewall, an IDS / IPS, and a log server.

[0055] It should be noted that the data of the network security device includes network traffic data, system logs, endpoint data, security event data, and user behavior data.

[0056] The network traffic data includes data in captured data packets.

[0057] The system logs include operating system logs, application program logs, Web server logs, firewall logs, IDS / IPS logs, and database logs.

[0058] The endpoint data includes behavior data of terminal devices, including process information, file operations, and registry changes.

[0059] The user behavior data includes login data, access path data, and operation behavior data of users.

[0060] The security event data includes alert data of IDS / IPS and scanning result data of antivirus software.

[0061] It should be noted that the network attack path includes:

[0062] The attacker intrudes the web server through the network, enters the internal application server through the vulnerability of the web server, and obtains access permission of the internal application database;

[0063] The attacker infects the internal device through malicious software, spreads the infection through the internal network, and attacks the internal database server;

[0064] The attacker obtains system management permission by stealing administrator accounts, modifies the database, and steals sensitive information.

[0065] It should be noted that analyzing the attack mode and attack event in the network history data specifically includes:

[0066] Extracting attack type, attack intensity, and attack duration from the network traffic data, and then using cosine similarity to calculate the similarity between different attack modes to analyze the association between different attack modes;

[0067] Using ARIMA time series analysis algorithm to model the time series of attack events to predict future network attack trends;

[0068] Classifying attack sources, and then using IP address resolution method and DNS query method to determine the geographic location of the attack source to analyze the relationship between attack events and attack sources.

[0069] It should be noted that the forged HTTP request includes:

[0070] Forging the User-Agent header in the HTTP request to simulate normal browser identification and legal user source of the HTTP request;

[0071] Forging the Rerferer header in the HTTP request to simulate the jump page of the HTTP request;

[0072] Forging the Authorization header in the HTTP request to simulate the user's legal identity to bypass identity verification.

[0073] It should be noted that the known signature of the network attack includes the hash value of the malicious software, the specific mode of SQL injection, and the abnormal packet format of the specific protocol.

[0074] It should be noted that the present application establishes an intelligent research and judgment model environment for the characteristics of the existing business system, identifies and extracts the characteristics of network attack threats, studies the identification strategy for known network attack threats, and continuously optimizes and corrects the identification strategy for network security threats through clustering prediction and analysis of known attack methods, thereby realizing intelligent analysis and research of network attack threats.

[0075] It should be noted that the known network attack threats are introduced in the training process to make the intelligent research and judgment model make correct judgments under abnormal disturbance, and a formal method is used to analyze the security vulnerabilities of the model, detect the adversarial samples and reinforce the model, so as to ensure that the verification and testing of the model can cover various security threats and edge cases, can comprehensively evaluate the performance of the model, and enhance the robustness and security of the intelligent research and judgment model in decision-making and judgment.

[0076] It should be noted that by fusing the reinforcement adaptive technology and the autonomous association reasoning technology, the generalization ability of the intelligent research and judgment model can be enhanced, the potential relationship between unknown threats and network attacks can be better understood, and the association relationship between attack threats and network environment can be better understood, the transparency of the learning decision-making process of the intelligent research and judgment model can be improved, the learning of the intelligent research and judgment model can be accelerated, and the associated knowledge in historical information can be identified and transferred, thereby better improving and strengthening the identification ability of unknown potential network attack threats.

[0077] It should be noted that by learning and predicting state changes through environment modeling, the environment state and the identification strategy of network attack threats can be learned from interaction, and the identification strategy of network attack threats can be adjusted to adapt to new model environments. In the uncertain network attack threat scenario, the research and judgment standard of the intelligent research and judgment model can be dynamically corrected, and the anti-interference ability of the intelligent research and judgment model can be improved.

[0078] It should be noted that the environment modeling is one of the core technologies of virtual reality technology, and a three-dimensional and interactive virtual world is generated using computer technology. The environment model used in the present application includes system architecture, network topology, key components, data flow, and user behavior model.

[0079] It should be noted that before identifying the characteristics of the network attack, the original data needs to be cleaned, formatted and standardized, irrelevant data and useless log entries are deleted, and data from different sources are converted into a unified standard format.

[0080] In specific embodiments, the specific code for classifying and labeling the data of the network security device using the SVM classification algorithm is as follows:

[0081] "from sk learn import svm

[0082] from sk learn.mode l_se lect ion import train_test_sp l it

[0083] from sk learn.preprocessing import StandardSca ler

[0084] from sk learn.metr ics import accuracy_score

[0085] #Assume X is the feature matrix and y is the label (0 represents normal, 1 represents attack).

[0086] X = #Feature Matrix

[0087] y = #tag

[0088] #Data is divided into training and test sets

[0089] X_train,

[0090] #Feature Standardization

[0091] sca ler = StandardSca ler()

[0092] X_train=scaler.fit_transform(X_train)

[0093] X_test=scaler.transform(X_test)

[0094] #SVM model using RBF kernel

[0095] clf=svm.SVC(kerne l='rbf', gamma='sca le', C=1)

[0096] #Training the SVM model

[0097] cl f.fit(X_train,y_train)

[0098] # Make predictions on the test set

[0099] y_pred=cl f.pred ict(X_test)

[0100] # Output classification accuracy

[0101] accuracy = accuracy_score(y_test, y_pred)

[0102] print(f'Accuracy: {accuracy * 100:.2f} %')”.

[0103] In a specific embodiment, the accuracy of classification is improved using cross-validation to adjust hyperparameters, and the code for optimizing the threat identification strategy of the network attack is as follows:

[0104] “from sk learn.mode l_se lect ion import Gr idSearchCV

[0105] # Define parameter range

[0106] param_grid = {'C': [0.1, 1, 10], 'gamma': ['scale', 'auto', 0.1, 1]}

[0107] # Use grid search and cross-validation to select the best parameters

[0108] grid_search = GridSearchCV(svm.SVC(kernel='rbf'), param_grid, cv=5)

[0109] grid_search.fit(X_train, y_train)

[0110] # Output best parameters and best score

[0111] print(f'Best Parameters: {grid_search.best_params_}')

[0112] print(f'Best Score: {grid_search.best_score_}')”.

[0113] It should be noted that by using the Q-learning method to continuously adjust the Q value to optimize the decision-making strategy of the intelligent research and judgment model, in the case of continuous arrival of new data, the intelligent research and judgment model can continuously update the parameters through incremental learning Stochastic Gradient Descent (SGD) and smooth exploration and utilization through the Softmax method, and make decisions based on the value of the current state.

[0114] It should be noted that the normal HTTP request is mixed with the anti-forgery HTTP request to train the intelligent judgment model, and the training process learns how to distinguish the normal HTTP request and the forged HTTP request, and the trained intelligent judgment model can detect the forged HTTP request with slight modification, accurately identify the potential fraudulent attack, thereby effectively preventing the attack and enhancing the resistance to network attack.

[0115] It should be noted that the architecture of the intelligent judgment model includes a user terminal device, an internal network, a database server, an application server, a Web server, a firewall, an IDS / IPS and a log server; wherein,

[0116] The user terminal device includes desktop computers, notebooks, mobile devices used by employees;

[0117] The internal network includes an enterprise local area network (LAN), and network devices such as switches and routers;

[0118] The database server is a database that stores enterprise data, and stores customer information, financial data and other data information;

[0119] The application server is a server that processes enterprise business logic, and is responsible for hosting business applications (such as ERP systems) of the enterprise;

[0120] The Web server provides enterprise websites and online services to the outside;

[0121] The firewall is used to monitor and control the traffic in and out of the network;

[0122] The IDS / IPS (intrusion detection / prevention system) is used to detect and prevent potential attack behavior in real time;

[0123] The log server is used to centrally record log data of all systems, networks and applications for analysis and auditing.

[0124] In specific embodiments, in environment modeling, after architecture modeling, data flow and user access path are further constructed, including how users access systems, applications and data, and how data flows between different system components, network security devices are modeled, which is beneficial to clarify the role and configuration of network security devices, and how network security devices interact with other parts of the system.

[0125] In specific embodiments, the user access path specifically includes:

[0126] User device to application server access path: the user accesses the internal business system through a browser or a client application, and sends a request to the application server.

[0127] Application server to database server access path: the application server communicates with the database server through the intranet to extract or update data from the database.

[0128] External access to the web server access path: external users access the company website through the Internet, and the web server provides public web services.

[0129] Database backup access path: regular database backups are transmitted to the backup server through the intranet.

[0130] In a specific embodiment, the process of generating a fake HTTP request sample simulates the process of an attacker forging a request, and the intelligent research and judgment model can identify and prevent fake HTTP requests;

[0131] The code of the User-Agent header in the fake HTTP request is:

[0132]

[0133] The code of the Rerferer header in the fake HTTP request is:

[0134]

[0135] The code of the Authorization header in the fake HTTP request is:

[0136]

[0137] In a specific embodiment, the code for training the intelligent research and judgment model is:

[0138]

[0139]

[0140] The trained intelligent research and judgment model can detect small changes in fake HTTP requests and accurately identify potential deceptive network attacks. The code for verifying the intelligent research and judgment model is:

[0141]

[0142] Figure 2 A system based on reinforcement adaptation for alarm research and judgment and empowerment optimization is shown, including a model construction module a, a network attack path module b, an identification module c, an optimization module d, and a re-optimization module e.

[0143] In specific embodiments, the model construction module a is configured to: establish an architecture of an intelligent research and judgment model according to an IT infrastructure and application systems within an enterprise, obtain data flow of an architecture component, an access path of a user, and data of a network security device, and perform environment modeling on the intelligent research and judgment model according to the data flow, the access path, and the data of the network security device to obtain the intelligent research and judgment model.

[0144] In specific embodiments, the network attack path module b is configured to: construct a network attack path according to the intelligent research and judgment model, and match the data of the network security device using a known signature of a network attack or a rule-based detection technology.

[0145] In specific embodiments, the identification module c is configured to: classify and label the data of the network security device using an SVM classification algorithm, train the intelligent research and judgment model using the data of the network security device, establish a threat identification strategy of the network attack in combination with the network attack path, and identify a network attack.

[0146] In specific embodiments, the optimization module d is configured to: perform context learning and state prediction on the intelligent research and judgment model through a Q-learning method, an SGD incremental learning method, and a Softmax method, analyze an attack mode and an attack event in network historical data, perform parameter fine-tuning on the intelligent research and judgment model in combination with an analysis result and a feature of the network attack, optimize the threat identification strategy of the network attack, and enhance a generalization ability and an autonomous correlation reasoning ability of the intelligent research and judgment model.

[0147] In specific embodiments, the re-optimization module e is configured to: generate an adversarial sample using a known threat of the network attack by performing a slight perturbation on a data field of a normal HTTP request to obtain a fake HTTP request, perform adversarial training on the intelligent research and judgment model in combination with the normal HTTP request, the fake HTTP request, a feature of the network attack, and the adversarial sample to obtain a final intelligent research and judgment model for identifying a deceptive network attack.

[0148] It should be noted that the present application performs intelligent analysis on alarms on a network security device by introducing security large model-based alarm research and judgment, collaborates with staff to complete processing of complex alarms in combination with big data analysis technology, ensures that network security information is not overwhelmed by massive alarms, further provides a fast and accurate security alarm and early warning function, establishes cross verification and correlation analysis of high-quality, multi-source threat intelligence, and improves defense capability of network attacks.

[0149] Reference will be made to the following Figure 3It shows a schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application. Figure 3 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0150] like Figure 3 As shown, the computer system includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 302 or programs loaded from storage section 308 into random access memory (RAM) 303. The RAM 303 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0151] The following components are connected to I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a liquid crystal display (LCD) and speakers, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN card and a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to I / O interface 305 as needed. A removable medium 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 310 as needed so that computer programs read from it can be installed into storage section 308 as needed.

[0152] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable storage medium, the computer program comprising program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication section 309, and / or installed from the removable medium 311. When the computer program is executed by the central processing unit (CPU) 301, the above-described functions defined in the methods of the present application are performed. It should be noted that the computer readable storage medium of the present application can be either a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, be - but is not limited to - an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the present application, the computer readable signal medium can include a data signal that is propagated in baseband or as part of a carrier wave, in which the computer readable program code is carried. Such propagated data signal can take a variety of forms, including but not limited to electro-magnetic, optical or any suitable combination of the above. The computer readable signal medium can also be any computer readable storage medium that can be used to carry or store program code in the form of instructions or data structures that can be accessed by a computer. The program code contained on the computer readable storage medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the above.

[0153] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0154] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0155] The modules involved in the embodiments of the present application can be implemented in the form of software or hardware.

[0156] As another aspect, the present application also provides a computer readable storage medium, which can be included in the electronic device described in the above embodiments, or can exist independently without being assembled into the electronic device. The computer readable storage medium carries one or more programs, when the one or more programs are executed by the electronic device, the electronic device is caused to: establish an intelligent research and judgment model architecture according to enterprise internal IT infrastructure and application systems, obtain data flow of architecture components, user access path and network security device data, and perform environment modeling on the intelligent research and judgment model according to the data flow, the access path and the network security device data, to obtain the intelligent research and judgment model; construct a network attack path according to the intelligent research and judgment model, match the network security device data using a known signature of network attack or a rule-based detection technology; classify and label the network security device data using an SVM classification algorithm, and train the intelligent research and judgment model using the network security device data, establish a threat identification strategy of the network attack in combination with the network attack path, and identify network attacks; perform context learning and state prediction on the intelligent research and judgment model through a Q-learning method, an SGD incremental learning method and a Softmax method, analyze attack patterns and attack events in network historical data, and perform parameter fine-tuning on the intelligent research and judgment model in combination with the analysis results and features of the network attack, optimize the threat identification strategy of the network attack, and enhance the generalization ability and autonomous correlation reasoning ability of the intelligent research and judgment model; generate an adversarial sample using a known threat of the network attack by performing a slight perturbation on data fields of a normal HTTP request, obtain a fake HTTP request, perform adversarial training on the intelligent research and judgment model in combination with the normal HTTP request, the fake HTTP request, the features of the network attack and the adversarial sample, and obtain a final intelligent research and judgment model to identify fraudulent network attacks.

[0157] Finally, it should be noted that the above description is only a preferred embodiment of the present application and a description of the applied technical principles. Those skilled in the art should understand that the scope of the application involved in the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and also covers other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with each other to form a technical solution with similar functions disclosed in the present application (but not limited to).

Claims

1. A method for alarm judgment and empowerment optimization based on reinforcement adaptation, characterized in that, The method comprises the following steps: S1: According to the IT infrastructure and application system in the enterprise, an intelligent research and judgment model is established, the data flow of the architecture component, the user access path and the network security device data are obtained, and the environment modeling of the intelligent research and judgment model is performed according to the data flow, the access path and the network security device data, so as to obtain the intelligent research and judgment model; S2: According to the intelligent research and judgment model, a network attack path is constructed, and the network security device data is matched using a known signature of network attack or a rule-based detection technology; S3: The network security device data is classified and labeled using an SVM classification algorithm, the intelligent research and judgment model is trained using the network security device data, the threat identification strategy of the network attack is established in combination with the network attack path, and the network attack is identified; S4: The intelligent research and judgment model is subjected to context learning and state prediction through a Q-learning method, an SGD incremental learning method and a Softmax method, the attack mode and attack event in the network historical data are analyzed, the parameters of the intelligent research and judgment model are fine-tuned in combination with the analysis result and the characteristics of the network attack, the threat identification strategy of the network attack is optimized, and the generalization ability and autonomous correlation reasoning ability of the intelligent research and judgment model are enhanced; S5: The network attack threat is generated by slightly disturbing the data field of the normal HTTP request, the forged HTTP request is obtained, the intelligent research and judgment model is subjected to adversarial training in combination with the normal HTTP request, the forged HTTP request, the characteristics of the network attack and the adversarial sample, and the final intelligent research and judgment model is obtained to identify the deceptive network attack.

2. The method of claim 1, wherein, The architecture of the intelligent research and judgment model comprises a user terminal device, an internal network, a database server, an application server, a Web server, a firewall, an IDS / IPS and a log server.

3. The method of claim 1, wherein, The network security device data comprises network traffic data, system logs, endpoint data, security event data and user behavior data; The network traffic data comprises data in the captured data packet; The system logs comprise operating system logs, application program logs, Web server logs, firewall logs, IDS / IPS logs and database logs; The endpoint data comprises the behavior data of the terminal device, including process information, file operation and registry change; The user behavior data comprises user login data, access path data and operation behavior data; The security event data comprises IDS / IPS alarm data and antivirus software scanning result data.

4. The method of claim 1, wherein, The network attack path comprises: The attacker intrudes the Web server through the network, enters the internal application server through the vulnerability of the Web server, and obtains the access permission of the internal application database; The attacker infects the internal device through malicious software, spreads the infection through the internal network and attacks the internal database server; The attacker obtains the system management permission by stealing the administrator account, modifies the database and steals the sensitive information.

5. The method of claim 3, wherein, The analysis of attack patterns and attack events in network history data specifically includes: Extracting attack types, attack intensities, and attack durations from the network traffic data, and using cosine similarity to calculate the similarity between different attack patterns and analyze the correlation between different attack patterns; Using ARIMA time series analysis algorithm to model the time series of attack events to predict future network attack trends; Classifying attack sources, and using IP address resolution methods and DNS query methods to determine the geographic location of attack sources to analyze the relationship between attack events and attack sources.

6. The method of claim 1, wherein, The forged HTTP request includes: Forging the User-Agent header in the HTTP request to simulate normal browser identification and legitimate user source of the HTTP request; Forging the Rerferer header in the HTTP request to simulate the jump page of the HTTP request; Forging the Authorization header in the HTTP request to simulate the user's legal identity to bypass identity verification.

7. The method of claim 1, wherein, The known signatures of network attacks include hash values of malicious software, specific patterns of SQL injection, and abnormal packet formats of specific protocols.

8. A system for alarm judgment and empowerment optimization based on reinforcement adaptation, characterized in that, It includes: A model construction module: according to the IT infrastructure and application systems within an enterprise, an intelligent research and judgment model architecture is established, data flows, user access paths, and network security device data are obtained, and environment modeling is performed on the intelligent research and judgment model according to the data flows, access paths, and network security device data to obtain the intelligent research and judgment model; A network attack path module: according to the intelligent research and judgment model, a network attack path is constructed, and the network security device data is matched using known signatures of network attacks or rule-based detection techniques; An identification module: using SVM classification algorithm to classify and label the network security device data, and using the network security device data to train the intelligent research and judgment model, combining the network attack path to establish the threat identification strategy of the network attack, and identifying network attacks; An optimization module: through Q-learning method, SGD incremental learning method and Softmax method, the intelligent research and judgment model is subjected to context learning and state prediction, attack patterns and attack events in network history data are analyzed, combining the analysis results and the characteristics of the network attack to fine-tune the parameters of the intelligent research and judgment model, optimize the threat identification strategy of the network attack, and enhance the generalization ability and autonomous correlation reasoning ability of the intelligent research and judgment model; A re-optimization module: by slightly perturbing the data fields of normal HTTP requests, using known threat of the network attack to generate adversarial samples, obtaining forged HTTP requests, combining normal HTTP requests, forged HTTP requests, characteristics of the network attack, and the adversarial samples to perform adversarial training on the intelligent research and judgment model, obtaining the final intelligent research and judgment model to identify fraudulent network attacks.

9. A computer program product having one or more computer programs embodied thereon, wherein the computer programs, when executed usefully employ at least one computer processor to cause the steps of claim 1 to be performed. When the computer program is executed by a computer processor, the method as claimed in any one of claims 1-7 is implemented.

Images