Log security rule remodeling method and device based on security audit

Abstract

The invention provides a log security rule remodeling method and device based on security audit, and the method comprises the steps: obtaining historical operation logs and abnormal logs of an information system, and carrying out the sorting according to the generation time; the operation logs and the abnormal logs are combined and divided into a first log set, a second log set and a third log set according to different division modes; respectively taking the operation information and the abnormal information of the logs in the single log set as independent transaction items to form a single security transaction; forming three security transaction sets based on all the first log sets, all the second log sets and all the third log sets; and respectively carrying out association rule mining and screening on the three security transaction sets based on an association rule algorithm to obtain security rules. According to the method, the operation log and the abnormal log are divided in multiple corresponding modes based on the possibility of multiple different corresponding relations between the operation log and the abnormal log, and the accuracy of mining the association rule of the information system operation and the system abnormality is improved.

Description

Technical Field

[0001] The present invention relates to the field of log correlation analysis, and in particular to a log security rule reshaping method and device based on security auditing. Background Art

[0002] Network devices, databases, servers, application systems, hosts, service programs, etc. in information systems will generate logs during operation. Logs record the date, time, user, action and other related operation descriptions. Common logs include operation, alarm, operation, message, status and other logs. Log files contain a lot of effective information, which is very useful for diagnosing and solving information system problems, and can help determine the status, performance and stability of the system.

[0003] Log security audit refers to the real-time monitoring and analysis of log records of various events and behaviors to detect potential security threats, understand system performance and perform troubleshooting. Log security audit requires the collection, storage and analysis of log data from various sources, but how to efficiently implement log analysis has always been a huge challenge in the field of operation and maintenance. The main problems include: (1) The amount of log data is too large. (2) The problem of multiple log reports, that is, when a network problem occurs, different alarm logs may be generated at multiple levels of network devices. (3) Because there is a connection relationship between devices, a problem on one device may be transmitted to other devices and cause derivative problems. In this scenario, a large number of devices will generate different logs at the same time, and these logs are related. These problems make it difficult for manual analysis methods to quickly obtain effective information from massive log data.

[0004] In order to solve the above problems, traditional solutions generally use built-in log alarm association rules to implement log alarm cause analysis. However, static log alarm association rules need to be summarized based on a large amount of knowledge and experience. The complexity and coverage of the rules are limited by the experience level of the rule setter, and expert experience cannot cover the new types of problems that continue to arise with the rapid development of network technology. In order to avoid the risk of limitations of empirical rules, the existing technology uses association rule algorithms to mine the hidden association rules in historical log data, effectively improving the accuracy and efficiency of log data association analysis.

[0005] In the prior art, there is a technical solution for analyzing and mining security events and user behavior association rules of security logs using the FP-Growth algorithm. The main process is: segment the security log according to the set time segment length, then extract the user behavior and device status feature items for each segment log data to generate transaction data, and finally perform association rule mining on all transaction data based on the FP-Growth algorithm. In the prior art, the security log is segmented based on the set time period length. The time period length can be set according to the user operation delay. However, this design method cannot divide the user operation and its delayed security impact into the same section of log data. For user operations in the middle and later sections of the time period, the delayed security impact may be in the log data after the log data of the time period, while for the device status at the front end of the time period, the user operation that caused the device status may be in the log data before the log data of the time period. This log data segmentation method will affect the accuracy of subsequent association rule mining. Summary of the invention

[0006] The purpose of the present invention is to solve the above-mentioned problems in the prior art, and to provide a log security rule reshaping method and device based on security auditing, which divides operation logs and exception logs in multiple corresponding ways based on the possibility of multiple different corresponding relationships between operation logs and exception logs, thereby improving the accuracy of mining association rules between information system operations and system anomalies.

[0007] In a first aspect, the present invention provides a log security rule reshaping method based on security auditing, the method comprising: S1, obtain the historical operation logs and exception logs of the information system and sort them in chronological order; S2, taking a single exception log and all operation logs between it and the previous exception log as a single first log set; dividing all exception logs into a number of exception log groups based on the generation time interval, taking a single exception log group and all operation logs between it and the previous exception log group, and all operation logs between its previous exception log group as a single second log set; taking a single exception log and all operation logs that are arranged before it and whose generation time is within the set impact delay as a single third log set; S3, taking the operation information and exception information of the logs in a single log set as separate transaction items, extracting all transaction items corresponding to the single log set to form a single security transaction; forming a first security transaction set based on all first log sets, forming a second security transaction set based on all second log sets, and forming a third security transaction set based on all third log sets; S4, based on the association rule algorithm, the three security transaction sets are respectively mined and screened to obtain security rules.

[0008] Preferably, the performing association rule mining and screening on a single security transaction set based on an association rule algorithm in S4 includes: S41, calculating the support of each transaction item in each security transaction in a single security transaction set, filtering out transaction items whose support is less than a set minimum support, and sorting the remaining transaction items in order of support to form an item header table; S42, re-arranging the transaction items in each security transaction according to the order of the item header table and deleting the transaction items whose support is less than the set minimum support, to form transaction data; S43, establishing a transaction item tree based on all transaction data corresponding to a single security transaction set; S44, deleting all transaction items before the abnormal level transaction item that is ranked first in the item header table, and for a single transaction item in the item header table after the deletion operation, finding a transaction item subtree of which the transaction item is a descendant node in the transaction item tree, and mining association rules based on the transaction item subtree with a prefix of operation information and a suffix of abnormal level transaction item; S45, calculating the confidence and lift of the association rules, and selecting the association rules whose confidence and lift are higher than the set confidence and lift as the safety rules.

[0009] Preferably, the operation information includes a subject, an operation and an object, and the abnormal information includes a device identification and an abnormality level.

[0010] Preferably, the method also includes S5. After screening out the security rules, if the operations and subjects and / or objects of the operation information transaction items of two security rules are the same, then the operation information transaction items of the two security rules are considered to be substantially the same; if all the transaction items of the two security rules are substantially the same, then the two security rules are considered to be substantially the same, and the substantially the same security rules are merged into one security rule.

[0011] In a second aspect, the present invention provides an electronic device, including a memory, a computer program stored thereon, and a processor, wherein the processor implements the above-mentioned security audit-based log security rule reshaping method when executing the computer program.

[0012] In a third aspect, the present invention provides a storage medium storing computer executable instructions, which, when loaded and executed by a processor, implement the above-mentioned log security rule reshaping method based on security auditing.

[0013] In summary, the present invention has the following beneficial effects: first, based on the possibility that a single abnormal reminder in the information system is caused by the operation of the software and hardware devices between the time and the previous abnormal reminder, a single abnormal log and all operation logs between the time and the previous abnormal log are divided into a single first log set; second, based on the possibility that a single or multiple operations of the software and hardware devices in the information system may cause multiple consecutive related abnormal reminders, all abnormal logs are divided into a number of abnormal log groups based on the generation time interval, and a single abnormal log group and all operation logs between the time and the previous abnormal log group, and all operation logs between the previous abnormal log group are taken as a single second log set; third, based on the possibility that a single abnormal reminder in the information system is caused by the operation delay of the software and hardware devices before it, a single abnormal log and all operation logs whose generation time is within the set impact delay are taken as a single third log set; by dividing the operation logs and the abnormal logs in multiple corresponding ways based on the possibility of multiple different corresponding relationships between the operation logs and the abnormal logs, the accuracy of mining the association rules between the information system operation and the system abnormality is improved.

[0014] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 This is a schematic diagram of the process of an embodiment of the present application.

[0017] Figure 2 A schematic diagram of a process for performing association rule mining and screening on a single security transaction set according to an embodiment of the present application.

[0018] Figure 3 A schematic diagram of establishing a transaction item tree according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical scheme and advantages disclosed in the embodiments of the present invention clearer, the embodiments of the present invention are further described in detail in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present invention and are not intended to limit the embodiments of the present invention. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions.

[0020] It should be noted that the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products or devices.

[0021] In the prior art, there is a technical solution for analyzing and mining security events and user behavior association rules of security logs using the FP-Growth algorithm. The main process is: segment the security log according to the set time segment length, then extract the user behavior and device status feature items for each segment log data to generate transaction data, and finally perform association rule mining on all transaction data based on the FP-Growth algorithm. In the prior art, the security log is segmented based on the set time period length. The time period length can be set according to the user operation delay. However, this design method cannot divide the user operation and its delayed security impact into the same section of log data. For user operations in the middle and later sections of the time period, the delayed security impact may be in the log data after the log data of the time period, while for the device status at the front end of the time period, the user operation that caused the device status may be in the log data before the log data of the time period. This log data segmentation method will affect the accuracy of subsequent association rule mining.

[0022] To solve the above problems, an embodiment of the present application provides a log security rule reshaping method based on security auditing, which divides the operation log and the exception log into different corresponding ways based on the possibility of different corresponding relationships between the operation log and the exception log, thereby improving the accuracy of mining the association rules between information system operations and system anomalies.

[0023] Specifically, Figure 1As shown, the method includes S1, obtaining historical operation logs and exception logs of the information system and sorting them in chronological order. In this step, historical operation logs and exception logs of all software and hardware devices in the information system are obtained, including historical operation logs and exception logs of terminals, servers, databases, operating systems, applications, etc.

[0024] like Figure 1 As shown, the method of the present application also includes S2. On the first aspect, based on the possibility that a single abnormal alert in the information system is caused by the operation of software and hardware devices between it and the previous abnormal alert, a single abnormal log and all operation logs between it and the previous abnormal log are taken as a single first log set.

[0025] The second aspect is based on the possibility that single or multiple operations of the software and hardware devices in the information system will cause multiple continuous related exception alerts. All exception logs are divided into several exception log groups based on the generation time interval, and a single exception log group and all operation logs between it and the previous exception log group, and all operation logs between its previous exception log group are taken as a single second log set.

[0026] In some embodiments of the present application, the time interval between each two adjacent abnormal logs in a single abnormal log group does not exceed the set time interval, or the time interval between the first and last abnormal logs in a single abnormal log group does not exceed the set time interval, or the time interval between each two adjacent abnormal logs in a single abnormal log group does not exceed the set time interval and the time interval between the first and last abnormal logs does not exceed the set time interval. It is understandable that different time interval thresholds can be set for two adjacent abnormal logs and the first and last abnormal logs.

[0027] The third aspect is based on the possibility that a single abnormality alert in the information system is caused by an operational delay of the software and hardware devices that precede it, and the single abnormality log and all the operation logs that precede it and whose generation time is within the set impact delay are regarded as a single third log set.

[0028] like Figure 1 As shown, the method of the present application also includes S3, which takes the operation information and exception information of the logs in a single log set as separate transaction items, extracts all transaction items corresponding to the single log set to form a single security transaction; forms a first security transaction set based on all first log sets, forms a second security transaction set based on all second log sets, and forms a third security transaction set based on all third log sets.

[0029] Usually, the operation log may include multiple information such as date, time, subject, operation, object, etc., and the exception log may include multiple information such as date, time, device identification, exception level, etc. In some embodiments of the present application, the operation information includes subject, operation and object, and a single operation information transaction item can be represented as T1 xyz =A x O y B z , A x represents the subject identified as x, O y represents the yth operation, B z Indicates the object identified as z; the abnormal information includes the device identification and abnormal level. The abnormal levels range from low to high, including warning, error, critical, alarm, emergency, etc. A single abnormal information transaction item can be represented as T2 ij =D i E j , D i Indicates the device marked as i, E j Indicates the jth level of abnormality.

[0030] like Figure 1 As shown, the method of the present application also includes S4, which performs association rule mining and screening on the three security transaction sets based on the association rule algorithm to obtain security rules. It can be understood that S4 finally obtains security rules corresponding to the three security transaction sets, and there may be duplicate security rules. The security rules of the three security transaction sets can be merged and deduplicated to obtain the final security rules.

[0031] In some embodiments of the present application, the association rule algorithm used is the FP-Growth algorithm, such as Figure 2 As shown, the association rule mining and screening of a single security transaction set based on the association rule algorithm in S4 includes S41, calculating the support of each transaction item in each security transaction in the single security transaction set, filtering the transaction items whose support is less than the set minimum support, and sorting the remaining transaction items in order of support to form an item header table.

[0032] The transaction item support can be expressed as S(T)=m(T) / N, where N represents the number of security transactions in the security transaction set, and m(T) represents the number of security transactions containing transaction item T in the security transaction set.

[0033] like Figure 2 As shown, S4 also includes S42, reordering the transaction items in each security transaction according to the order of the item header table and deleting the transaction items whose support is less than the set minimum support, to form transaction data. Assume that a security transaction is {T1 111 ,T1 112 ,T1 213 ,T1 124 ,T2 12,T2 23}, where S(T1 111 ) support does not meet the requirements, S(T1 124 )>S(T2 12 )>S(T1 112 )>S(T2 23 )>S(T1 213 ), the corresponding transaction data is {T1 124 ,T2 12 ,T1 112 ,T2 23 ,T1 213}.

[0034] like Figure 2 As shown, S4 also includes S43, establishing a transaction item tree based on all transaction data corresponding to a single security transaction set. Figure 3 As shown, initially the transaction item tree has only one head node. When establishing the transaction item tree, the sorted transaction data is read one by one, and the transaction items in it are inserted into the head node of the transaction item tree in the sorted order. The transaction items with the highest order are used as ancestor nodes, and the ones with the lowest order are used as descendant nodes. The transaction item tree is established when all transaction data are inserted into the transaction item tree.

[0035] like Figure 2 As shown, S4 also includes S44, deleting all transaction items before the abnormal level transaction item that is ranked first in the item header table, and for a single transaction item in the item header table after the deletion operation, finding the transaction item subtree of which is a descendant node in the transaction item tree, and mining association rules with transaction items prefixed with operation information and transaction items suffixed with abnormal level based on the transaction item subtree.

[0036] Usually, the FP-Growth algorithm mines upward from the bottom item of the item header table. For each item in the item header table, its conditional pattern base is found corresponding to the transaction item tree. The conditional pattern base is the transaction item subtree corresponding to the transaction item to be mined as the descendant node; then, frequent item sets are recursively mined based on the conditional pattern base, and finally, association rules are mined based on the frequent item sets. This application needs to find the association rules between operations and exceptions in the information system, and the frequent item sets mined by the conventional FP-Growth algorithm include frequent item sets that do not contain operations and abnormal information transaction items at the same time. The association rules mined based on these frequent item sets are also association rules that do not contain operations and exceptions at the same time, which are not suitable for this application. In this application, since the transaction item subtree corresponding to the transaction item before the abnormal level transaction item ranked first in the item header table does not contain abnormal level transaction item nodes, deleting all transaction items before the abnormal level transaction item ranked first in the item header table can save time in finding its corresponding transaction item subtree. In addition, after finding the transaction item subtree, the association rules of transaction items with prefixes of operation information and transaction items with suffixes of exception levels are mined directly based on the transaction item subtree, eliminating the step of mining frequent item sets. At the same time, association rule mining with precise constraints is performed, saving the time of association rule mining and improving the efficiency of log security rule reshaping.

[0037] like Figure 2 As shown, S4 also includes S45, calculating the confidence and lift of the association rule, and selecting the association rule with a confidence and lift higher than the set confidence and lift as the security rule. The confidence is the ratio of the number of security transactions in the security transaction set that contain both the prefix and suffix transaction items of the association rule to the number of security transactions that contain the prefix transaction item of the association rule, and the lift is the ratio of the confidence of the association rule to the support of the suffix transaction item of the association rule.

[0038] In actual network security scenarios, the manifestation of the same threat operation may not be exactly the same. It may be disguised as a different subject to perform the same operation on the same object, causing the same exception; or the same operation may be performed on different objects, causing the same exception. In order to de-duplicate and merge these substantially identical operation-exception security rules, in some embodiments of the present application, the method also includes S5. After screening out the security rules, if the operations and subjects and / or objects of the operation information transaction items of the two security rules are the same, then the operation information transaction items of the two security rules are considered to be substantially the same; if all the transaction items of the two security rules are substantially the same, then the two security rules are considered to be substantially the same, and the substantially identical security rules are merged into one security rule.

[0039] For example, safety rule R1=(T1 111 ,T1 112 ,T1 124 ;T2 12), the rule prefix is ​​before the semicolon, and the rule suffix is ​​after the semicolon. Security rule R2=(T1 111 ,T1 212 ,T1 125 ;T2 12 ), then the two safety rules can be considered to be essentially the same and merged into one safety rule (T1 111 ,T1 a12 ,T1 12a ;T2 12 ), this safety rule indicates that T1 111 That is, the subject identified as 1 performs the first operation on the object identified as 1, T1 a12 That is, any subject performs the first operation, T1, on the object marked as 2. 12a That is, if the subject identified as 1 performs the second operation on any object, it is very likely to cause the device identified as 1 to produce a second-level anomaly. Based on the historical log data of the information system, the operation-anomaly security rules are reshaped. Later, threat detection and disposal strategies can be added to the information system based on the reshaped log security rules to discover and resolve potential threats to the system as soon as possible.

[0040] The embodiment of the present application also provides an electronic device, including a memory and a processor, and the memory and the processor can be connected via a bus or other means. The memory can be used to store software programs, computer programs and modules, such as the program / module corresponding to the above-mentioned flexible defense network architecture planning method; the processor implements the above-mentioned log security rule reshaping method based on security audit by executing the computer program and module in the memory.

[0041] The processor may be a central processing unit, a digital signal processor, an application-specific integrated circuit, a field programmable gate array, etc., and the memory may be a high-speed random access memory, a non-volatile memory, etc.

[0042] The embodiment of the present application also provides a storage medium, in which computer executable instructions are stored, and when the computer executable instructions are loaded and executed by the processor, the above-mentioned log security rule reshaping method based on security audit is implemented. Among them, the storage medium can be a combination of one or more of a magnetic disk, an optical disk, a read-only storage memory, a random access storage memory, a flash memory, a hard disk, etc.

[0043] It should be noted that the sequence of the above embodiments of the present invention is for description only and does not represent the advantages and disadvantages of the embodiments. The above describes specific embodiments of this specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0044] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0045] Those skilled in the art will appreciate that all or part of the steps of the above embodiments can be accomplished by hardware, or by programs instructing the relevant hardware to accomplish the steps. The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A log security rule reshaping method based on security audit, characterized in that: include: S1, obtain the historical operation logs and exception logs of the information system and sort them in chronological order; S2, taking a single exception log and all operation logs between it and the previous exception log as a single first log set; dividing all exception logs into a number of exception log groups based on the generation time interval, taking a single exception log group and all operation logs between it and the previous exception log group, and all operation logs between its previous exception log group as a single second log set; taking a single exception log and all operation logs that are arranged before it and whose generation time is within the set impact delay as a single third log set; S3, taking the operation information and exception information of the logs in a single log set as separate transaction items, extracting all transaction items corresponding to the single log set to form a single security transaction; forming a first security transaction set based on all first log sets, forming a second security transaction set based on all second log sets, and forming a third security transaction set based on all third log sets; S4, based on the association rule algorithm, the three security transaction sets are respectively mined and screened to obtain security rules.

2. According to a method for reshaping log security rules based on security auditing according to claim 1, it is characterized in that: The association rule mining and screening of a single security transaction set based on the association rule algorithm in S4 includes: S41, calculating the support of each transaction item in each security transaction in a single security transaction set, filtering out transaction items whose support is less than a set minimum support, and sorting the remaining transaction items in order of support to form an item header table; S42, re-arranging the transaction items in each security transaction according to the order of the item header table and deleting the transaction items whose support is less than the set minimum support, to form transaction data; S43, establishing a transaction item tree based on all transaction data corresponding to a single security transaction set; S44, deleting all transaction items before the abnormal level transaction item that is ranked first in the item header table, and for a single transaction item in the item header table after the deletion operation, finding a transaction item subtree of which the transaction item is a descendant node in the transaction item tree, and mining association rules based on the transaction item subtree with a prefix of operation information and a suffix of abnormal level transaction item; S45, calculating the confidence and lift of the association rules, and selecting the association rules whose confidence and lift are higher than the set confidence and lift as the safety rules.

3. According to a log security rule reshaping method based on security auditing according to claim 1, it is characterized in that: The operation information includes subject, operation and object, and the abnormal information includes device identification and abnormal level.

4. According to a method for reshaping log security rules based on security auditing according to claim 3, it is characterized in that: After the security rules are screened out in S45, if the operations and subjects and / or objects of the operation information transaction items of the two security rules are the same, the operation information transaction items of the two security rules are considered to be the same; if all the transaction items of the two security rules are the same, the two security rules are considered to be the same, and the same security rules are merged into one security rule.

5. An electronic device comprising a memory and a computer program and a processor stored thereon, characterized in that: When the processor executes the computer program, the method for reshaping log security rules based on security auditing as described in any one of claims 1 to 4 is implemented.

6. A storage medium, characterized in that The storage medium stores computer executable instructions, and when the computer executable instructions are loaded and executed by the processor, a log security rule reshaping method based on security auditing as described in any one of claims 1 to 4 is implemented.

Images