A network penetration behavior identification method, device, equipment and medium

Abstract

The present disclosure relates to the technical field of network security, and particularly relates to a network penetration behavior identification method and device, equipment and medium. The present disclosure is in line with the characteristics of network penetration attack behavior; the word segmentation method improves the algorithm parameter input, which can flexibly adapt to the change of attack behavior, so as to realize algorithm iteration; through the access of middleware logs, real-time security monitoring of B / S application system is realized, and the system security protection efficiency is improved. The present disclosure enhances the monitoring of web application system, fully utilizes the network layer and application layer protocol, collects system running data, automatically judges the system security state, saves the labor cost, improves the security monitoring effectiveness, guarantees the safe operation of information assets, avoids or reduces the economic and reputation loss caused by network security events.

Description

Technical Field

[0001] This disclosure relates to the field of network security technology, and in particular to a method, apparatus, device and medium for identifying network penetration behavior. Background Technology

[0002] With increasing business demands and the continuous upgrading of information technology, the number of self-built application systems within enterprises is increasing year by year. Artificial intelligence technology has seen significant development, but it mostly runs on the backend of application systems to complete tasks; the architecture of enterprise application systems remains primarily B / S (Browser / Server).

[0003] With increasing emphasis on cybersecurity, measures are being taken to strengthen the cybersecurity protection of application systems, including reducing network exposure, increasing cybersecurity equipment, and training cybersecurity teams.

[0004] Besides phishing emails and social engineering attacks, external network attack traffic mostly enters through open service ports of application systems, and penetration behaviors include SQL injection and sensitive information probing. Due to the inherent characteristics of the B / S architecture, access behavior is logged by middleware.

[0005] In existing technologies, the classification results of middleware log records after labeling, training, and classification are not ideal. Summary of the Invention

[0006] To address the aforementioned issues, this disclosure provides a method, apparatus, device, and medium for identifying network penetration behaviors.

[0007] Firstly, a method for identifying network penetration behavior, the method comprising:

[0008] Collect logs suspected of containing attack and penetration behaviors and extract them into unified log fields;

[0009] By examining the contents of the URI field in the log records, and using the URI to record access behavior, we can determine whether there is any attack or penetration behavior in the access process corresponding to each record. If attack or penetration behavior is found, it is marked as an abnormal tag; otherwise, it is marked as normal.

[0010] For the URI field in the log of the anomaly tag, special characters are used as delimiters to segment the numbers, letters, and special characters in the parameter value into independent words, and feature vectors are extracted from the independent words.

[0011] Based on feature vectors, a recognition model is obtained by training using the Naive Bayes algorithm.

[0012] The logs to be detected are identified and classified based on the recognition model.

[0013] Furthermore, logs suspected of containing attack or infiltration activities are collected, including:

[0014] For systems suspected of containing attack and penetration activities, use application-layer penetration tools to perform deep scans of the system and collect log texts recorded by IIS, Tomcat, and WebLogic middleware.

[0015] Furthermore, the data is extracted into unified log fields, including:

[0016] Logs suspected of containing attack or penetration activities are merged and stored separately by natural day.

[0017] Unified log fields include: date, time, client IP, server IP, request method, response status code, and request URL.

[0018] Furthermore, the extraction of unified log fields also includes:

[0019] After merging the logs, each log record is traversed one by one, the content of the URI field is decoded and processed and saved in plaintext, and blank field values, missing timestamps and unrecognizable characters in the log records are deleted.

[0020] Further, examine the URI field in the log records. By using the URI to record access behavior, determine if any attack or penetration activity exists in the access process corresponding to each record. If attack or penetration activity is found, mark it as abnormal; otherwise, mark it as normal. This also includes:

[0021] Use the Uri parameter value as an independent word segmentation, and extract feature vectors from the independent word segments;

[0022] Based on feature vectors, the Naive Bayes algorithm is used for identification and classification.

[0023] Furthermore, feature vectors are extracted from independent word segments, including:

[0024] The feature vector is extracted using a word set mode, where the occurrence of each word is taken as a feature, and all words are used as the feature vector.

[0025] Furthermore, feature vectors are extracted from independent word segments, including:

[0026] The bag-of-words approach is used to extract feature vectors. The Uri parameter value is used as the word segmentation, and the occurrence frequency of each word is used as a feature. All words are used as feature vectors.

[0027] In a second aspect, a network penetration behavior identification device includes: a collection unit, a labeling unit, a feature vector extraction unit, a training unit, and an identification unit;

[0028] The collection unit is used to collect logs suspected of containing attack and penetration behaviors and extract them into unified log fields;

[0029] The tagging unit is used to view the contents of the URI field in the log records. By using the URI to record the access behavior, it is determined whether there is any attack or penetration behavior in the access process corresponding to each record. If attack or penetration behavior is found, it is marked as an abnormal tag; otherwise, it is marked as normal.

[0030] The feature vector extraction unit is used to segment the URI field in the log of the anomaly label into independent words by using special characters as the delimiter, and extract feature vectors from the independent words.

[0031] The training unit is used to train the recognition model based on the feature vector using the Naive Bayes algorithm.

[0032] The identification unit is used to identify and classify the logs to be detected based on the identification model.

[0033] Thirdly, an electronic device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0034] Memory, which stores computer programs;

[0035] When a processor executes a computer program stored in memory, it implements the aforementioned method for identifying network penetration behaviors.

[0036] Fourthly, a computer-readable storage medium stores a computer program that, when executed by a processor, implements the aforementioned method for identifying network penetration behaviors.

[0037] This disclosure contains at least the following beneficial effects:

[0038] This disclosure enhances the monitoring of web application systems, makes full use of network layer and application layer protocols, collects system operation data, automatically determines the system security status, saves labor costs, improves the effectiveness of security monitoring, ensures the safe operation of information assets, and avoids or reduces economic and reputational losses caused by cybersecurity incidents.

[0039] This disclosure applies to a parametric word segmentation algorithm for classifying penetration behaviors. Practical application and comparison demonstrate that this algorithm can effectively extract behavioral features. By analyzing typical characteristics of penetration behaviors, the URI parametric word segmentation is improved, enhancing network attack behavior features and significantly increasing the algorithm's classification accuracy.

[0040] This disclosure closely aligns with the characteristics of network penetration attacks; the word segmentation method improves the algorithm parameter input, enabling flexible adaptation to changes in attack behavior, thereby achieving algorithm iteration; and through middleware log access, it achieves real-time security monitoring of B / S application systems, improving system security protection efficiency.

[0041] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objects and other advantages of this disclosure may be realized and obtained by means of the structures pointed out in the description and the accompanying drawings. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a schematic diagram of the identification method flow according to an embodiment of the present disclosure;

[0044] Figure 2 This is a schematic diagram of the identification device structure according to an embodiment of the present disclosure;

[0045] Figure 3 This is a schematic diagram of the electronic device structure according to an embodiment of the present disclosure;

[0046] Figure 4 This is a schematic diagram illustrating the process principle of an embodiment of this disclosure;

[0047] Figure 5 This is a schematic diagram illustrating the word segmentation principle of an embodiment of this disclosure;

[0048] Figure 6 This is a schematic diagram of the ROC execution result of an embodiment of this disclosure. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0050] like Figure 1 As shown, a method for identifying network penetration behavior includes:

[0051] S101, collect logs suspected of containing attack and penetration behaviors, and extract them into unified log fields;

[0052] S102, check the contents of the Uri field in the log records. By using the Uri to record the access behavior, determine whether there is any attack or penetration behavior in the access process corresponding to each record. If there is attack or penetration behavior, mark it as an abnormal tag; otherwise, mark it as normal.

[0053] S103, for the Uri field in the log of the abnormal label, use special characters as the delimiter to split the numbers, letters and special characters in the parameter value into independent words, and extract feature vectors from the independent words;

[0054] S104, based on feature vectors, uses the Naive Bayes algorithm to train and obtain the recognition model;

[0055] S105, based on the recognition model, identifies and classifies the logs to be detected.

[0056] The specific implementation details are as follows:

[0057] By collecting access logs from B / S application system middleware during security protection practice and security penetration testing (including automated and manual methods for network security software or devices), different log formats are standardized with fields merged, and unidentifiable noise entries are removed. Each log record is then tagged to indicate normal and abnormal behavior. By analyzing typical characteristics of penetration behavior, URI parameter segmentation is improved to enhance network attack behavior characteristics, significantly increasing the algorithm's classification accuracy.

[0058] Log collection,

[0059] Logs containing attack and penetration activity were comprehensively collected. The log sources primarily included two types of real-world penetration scenarios: those conducted manually and those using tools. These included routine cybersecurity operations organized by the Ministry of Public Security annually, important national conferences and events, and sensitive periods when sudden cyber threats caused frequent incidents. Deep system scanning was performed using application-layer penetration tools. Log texts recorded by middleware such as IIS, Tomcat, and WebLogic were collected, merged, and stored separately by natural day. The extracted and standardized log fields are shown in Table 1.

[0060] Table 1

[0061] Date Time C-ip S-ip Mothed Status Uri date time Client IP server IP Request method Response status code Request link

[0062] And save the logs of all different middlewares into a single file according to the above field format.

[0063] Noise removal

[0064] After the logs are merged, each log record is traversed one by one, and the contents of the URI fields are decoded (such as httpURI, base64, etc.) and saved in plaintext. If there are errors such as blank field values, missing timestamps, or unrecognizable characters in the log records, such invalid data is deleted from the logs.

[0065] Behavioral identifiers

[0066] Network security analysts are requested to review the URI field content of each log record, manually determine whether the access process corresponding to each record involves penetration or attack behavior based on the URI, and label the record as normal or abnormal; the tagged record is shown in Table 2:

[0067] Table 2

[0068]

[0069] Feature extraction,

[0070] Use the following two methods to perform log URI tokenization:

[0071] a: Parameter-based word segmentation: Uses the Uri standard format, and by default, the Uri parameter value is used as the word segmentation.

[0072] b: Improved word segmentation: The URI parameters in the log samples containing penetration attack actions are composed of various expressions. The special characters that connect these variable definitions, operators, reserved words, keywords, etc., are all special characters. These special characters are used as the splitting positions to segment the numbers, letters, and special characters in the parameter values ​​into independent words, and these words are used as feature vectors.

[0073] Based on word segmentation methods a and b, the following two feature extractions were performed respectively:

[0074] c: Word set mode, which uses the occurrence of each word as a feature and uses all words as a feature vector;

[0075] d: Bag-of-words mode, which uses the Uri parameter value as the word segmentation, takes the occurrence frequency of each word as a feature, and uses all words as the feature vector;

[0076] Data samples were generated according to the feature extraction methods of c and d, respectively.

[0077] Training and validation

[0078] After summarizing the log entries with behavioral labels, features are extracted, and four data samples are generated. 20%-30% of the data samples are randomly selected as the validation set, and the remaining data samples of the same number are selected as the training set to input into the algorithm, ensuring that there is no duplicate data in the validation set and the training set, and the retention cross-validation is completed respectively.

[0079] Iterate through the above process multiple times to calculate the average error rate for each method, namely: the error rate results for the ac, ad, bc, and bd methods.

[0080] like Figure 2 As shown, a network penetration behavior identification device includes: a collection unit 201, a labeling unit 202, a feature vector extraction unit 203, a training unit 204, and an identification unit 205;

[0081] Collection unit 201 is used to collect logs suspected of containing attack and penetration behaviors and extract them into unified log fields;

[0082] The tagging unit 202 is used to view the content of the Uri field in the log records. By recording the access behavior through the Uri, it is determined whether there is any attack or penetration behavior in the access process corresponding to each record. When attack or penetration behavior is found, it is marked as an abnormal tag; otherwise, it is marked as normal.

[0083] The feature vector extraction unit 203 is used to segment the URI field in the log of the anomaly label into independent words by using special characters as the splitting position, and extract feature vectors from the independent words.

[0084] Training unit 204 is used to train the recognition model based on the feature vector using the Naive Bayes algorithm.

[0085] The identification unit 205 is used to identify and classify the logs to be detected based on the identification model.

[0086] like Figure 3 As shown, this disclosure provides an electronic device, including a processor 301, a communication interface 302, a memory 303, and a communication bus 304, wherein the processor 301, the communication interface 302, and the memory 303 communicate with each other through the communication bus 304;

[0087] Memory 303 stores computer programs;

[0088] The processor 301 implements the above method when executing a computer program stored in the memory 303.

[0089] This disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0090] The computer-readable storage medium may be included in the device / apparatus described in the above embodiments; or it may exist independently and not assembled into the device / apparatus. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0091] According to embodiments of this disclosure, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0092] To enable those skilled in the art to better understand this disclosure, the principles of this disclosure are explained below in conjunction with the accompanying drawings:

[0093] like Figure 4 As shown, this disclosure includes: Data collection: Middleware logs are the main input data for the algorithm and are directly related to the accuracy of the algorithm's classification. Log samples generated during real-world network penetration attacks in the production environment of application systems are the most valuable. The middleware logs of this invention come from online production application systems during important network security protection phases such as the Ministry of Public Security's network protection operations, major national conferences and events, as well as penetration test execution results from various application-layer scanning devices.

[0094] Feature extraction: Extracting attack features from middleware logs requires familiarity with network scanning, penetration, and attack behaviors. In addition to correctly identifying the behavior category, the degree of familiarity with network security, access behavior, and even application development determines the accuracy of attack feature identification, which in turn affects the accuracy of algorithm classification.

[0095] Word segmentation: URL word segmentation can enhance the algorithm's ability to identify attack behaviors. Appropriate word segmentation may increase the overhead of training data for the algorithm, but it increases the accuracy of identifying attack behaviors.

[0096] Examples of parameter-based word segmentation and improved analysis are as follows: Figure 5 As shown;

[0097] The improved word segmentation pseudocode is as follows:

[0098]

[0099] Currently, various organizations monitor website operations primarily by monitoring network traffic. They utilize network security situational awareness systems, IDS / IPS, application-layer firewalls, and other network security devices, using the web system's IP address as the lookup criterion to determine the website's security status from alarms and logs across numerous devices. This method of side-channel protection is a passive, reactive approach.

[0100] If a potentially successful security incident is found in the alarms or logs of network security devices, especially high-risk incidents such as the leakage of sensitive information on a website, SQL and script injection, it is necessary to log in to the website server to further confirm the security status. This requires the administrator to have extensive network security experience and the ability to respond quickly.

[0101] Based on the Naive Bayes algorithm, parametric segmentation uses the overall parameter values ​​as training data for independent features of log entries. Even if a penetration test statement performs the same function, the features obtained by the algorithm will change if any assignment or expression in the statement is adjusted. This means that for the same type of access behavior with the same purpose or penetration action, existing in both the training and test sets, subtle differences in values ​​or expressions will cause the features learned by the algorithm in the training set to fail to identify the penetration behavior in the test set, resulting in low classification accuracy. The error rate for classifying access behavior of the application system is approximately 15%.

[0102] This invention utilizes machine learning technology to implement Naive Bayes classification for real-time monitoring of the network security status of application systems.

[0103] The sample data is extensive, authentic, and of high quality. It comprises middleware logs obtained from deep scans of application systems using three or more application-layer scanning devices, application system middleware logs collected during real-world cybersecurity trials, and host forensic logs gathered after actual cybersecurity incidents.

[0104] This invention filters invalid logs and performs HTML, URL, base64, and other decoding operations on the URL field parameters of the logs to obtain plaintext data;

[0105] Supervised learning is used, and manually labeled log samples are classified as training data for the algorithm to improve the probability and statistical accuracy of the algorithm's classification.

[0106] Starting from the essence of network attacks, features are extracted. By utilizing the syntactic characteristics of scripting languages, SQL statements, file paths, etc., URL parameters with attack characteristics are refined and segmented to form feature vectors.

[0107] It has room for continuous updates and improvements. During application, by collecting misclassified log entries and extracting features, the algorithm is iteratively retrained to enable it to synchronously identify changes in network attacks.

[0108] After collecting data from logs, removing noise, and identifying behaviors, 20% of the data samples were randomly selected as the training set, and the other 20% as the test set, ensuring that there was no duplicate data in the two sets. Using both word sets and bag-of-words methods, parametric segmentation and improved segmentation were performed respectively. After five iterations of the Naive Bayes classification algorithm, the test set was subjected to retention cross-validation. The cross-validation error rate results for parametric segmentation are shown in Table 3, and the cross-validation results for improved segmentation are shown in Table 4.

[0109] Table 3

[0110]

[0111] Table 4

[0112]

[0113] like Figure 6 As shown, the Receiver Operating Characteristic (ROC) curve plots the false positive rate on the x-axis and the true positive rate on the y-axis. In most cases, the ROC curve does not directly reflect the classifier's performance; instead, the Area Under the Curve (AUC) value is used as the evaluation metric. AUC is defined as the area under the ROC curve and the coordinate axes; a larger AUC value indicates a better classifier performance.

[0114] Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A method for identifying network penetration behavior, characterized in that, The method includes: Collect logs suspected of containing attack and penetration behaviors and extract them into unified log fields; By examining the contents of the URI field in the log records, and using the URI to record access behavior, we can determine whether there is any attack or penetration behavior in the access process corresponding to each record. If attack or penetration behavior is found, it is marked as an abnormal tag; otherwise, it is marked as normal. For the URI field in the log of the anomaly tag, special characters are used as delimiters to segment the numbers, letters, and special characters in the parameter value into independent words, and feature vectors are extracted from the independent words. Based on feature vectors, a recognition model is obtained by training using the Naive Bayes algorithm. The logs to be detected are identified and classified based on the recognition model.

2. The network penetration behavior identification method according to claim 1, characterized in that, Collect logs suspected of containing attack or infiltration activities, including: For systems suspected of containing attack and penetration activities, use application-layer penetration tools to perform deep scans of the system and collect log texts recorded by IIS, Tomcat, and WebLogic middleware.

3. The network penetration behavior identification method according to claim 1, characterized in that, Extracted as unified log fields, including: Logs suspected of containing attack or penetration activities are merged and stored separately by natural day. Unified log fields include: date, time, client IP, server IP, request method, response status code, and request URL.

4. The network penetration behavior identification method according to claim 1, characterized in that, Extracted as unified log fields, it also includes: After merging the logs, each log record is traversed one by one, the content of the URI field is decoded and processed and saved in plaintext, and blank field values, missing timestamps and unrecognizable characters in the log records are deleted.

5. The network penetration behavior identification method according to claim 1, characterized in that, By examining the URI field in the log records, and using the URI to record access behavior, we can determine whether each record corresponds to an attack or penetration attempt. If an attack or penetration attempt is found, it is marked as abnormal; otherwise, it is marked as normal. This also includes: Use the Uri parameter value as an independent word segmentation, and extract feature vectors from the independent word segments; Based on feature vectors, the Naive Bayes algorithm is used for identification and classification.

6. A method for identifying network penetration behavior according to claim 1 or 5, characterized in that, Extracting feature vectors from independent word segments includes: The feature vector is extracted using a word set mode, where the occurrence of each word is taken as a feature, and all words are used as the feature vector.

7. A method for identifying network penetration behavior according to claim 1 or 5, characterized in that, Extracting feature vectors from independent word segments includes: The bag-of-words approach is used to extract feature vectors. The Uri parameter value is used as the word segmentation, and the occurrence frequency of each word is used as a feature. All words are used as feature vectors.

8. A network penetration behavior identification device, characterized in that, include: The system comprises a collection unit, a labeling unit, a feature vector extraction unit, a training unit, and a recognition unit. The collection unit is used to collect logs suspected of containing attack and penetration behaviors and extract them into unified log fields; The tagging unit is used to view the contents of the URI field in the log records. By using the URI to record the access behavior, it is determined whether there is any attack or penetration behavior in the access process corresponding to each record. If attack or penetration behavior is found, it is marked as an abnormal tag; otherwise, it is marked as normal. The feature vector extraction unit is used to segment the URI field in the log of the anomaly label into independent words by using special characters as the delimiter, and extract feature vectors from the independent words. The training unit is used to train the recognition model based on the feature vector using the Naive Bayes algorithm. The identification unit is used to identify and classify the logs to be detected based on the identification model.

9. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, which stores computer programs; A processor, when executing a computer program stored in memory, implements a network penetration behavior identification method according to any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a network penetration behavior identification method according to any one of claims 1-7.

Images