Audit log reduction method, system, electronic device, and storage medium

Abstract

The embodiment of the application provides a kind of audit log reduction method, system, electronic equipment and storage medium, belong to audit log technical field.The method comprises: obtaining original log data;Original log data is input to target cache for operation analysis, obtain the multiple triple data of target cache;Wherein, each triple data includes two node sub data and the node action information between node sub data;Based on node sub data and node action information, file reference reduction processing is carried out to triple data, to obtain file reference reduction data;Based on node sub data and node action information, time dependent relationship reduction processing is carried out to triple data, to obtain time dependent relationship reduction data;Based on file reference reduction data and time dependent relationship reduction data, association extraction is carried out, to obtain target reduction log.The embodiment of the application can reduce audit log.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of audit log, and particularly relates to an audit log reduction method and system, an electronic device and a storage medium. BACKGROUND

[0002] An audit log is a log file that records system activities. It is used to track and record user or system access and operation behavior on application programs, network devices, operating systems, etc. It is crucial for revealing and tracking complex network threats. Security analysts can convert audit logs into provenance graphs, which can more clearly illustrate the time sequence and causality of the execution history on the host. Once suspicious behavior is detected, forensic analysis will reconstruct the attack footprint to help security analysts investigate threats.

[0003] However, the massive and long-term storage requirements of audit logs bring excessive space and processing overhead, so that the life cycle of audit logs is limited to a few days, which is not enough to track multi-step attacks lasting for months or years. A large number of audit logs make it very difficult to detect and track potential malicious activities on the host. Therefore, how to reduce the audit log has become a technical problem to be solved. SUMMARY

[0004] The main purpose of the embodiments of the present application is to provide an audit log reduction method and system, an electronic device and a storage medium, which aims to reduce the audit log.

[0005] To achieve the above purpose, a first aspect of the embodiments of the present application provides an audit log reduction method, which comprises:

[0006] obtaining original log data;

[0007] inputting the original log data into a target cache for operation parsing to obtain a plurality of triple data of the target cache; wherein each triple data contains two node sub-data and node action information between the node sub-data;

[0008] performing file reference reduction processing on the triple data based on the node sub-data and the node action information to obtain file reference reduction data;

[0009] performing time-dependent relationship reduction processing on the triple data based on the node sub-data and the node action information to obtain time-dependent relationship reduction data;

[0010] Correlation extraction is performed based on the file reference reduction data and the time-dependent relationship reduction data, and target reduction logs are obtained.

[0011] In some embodiments, the node sub-data is process node sub-data or file node sub-data, each of the triple data includes a corresponding process node sub-data, a corresponding node action information and a corresponding file node sub-data, and file reference reduction processing is performed on the triple data based on the node sub-data and the node action information to obtain file reference reduction data, including:

[0012] The same process node sub-data possessed by each of the triple data is determined as composite node data.

[0013] For each of the triple data corresponding to the composite node data, a file dependency template matching the corresponding process node sub-data is constructed based on the corresponding node action information.

[0014] Based on the file dependency template, the node action information of the triple data is merged to obtain the file reference reduction data.

[0015] In some embodiments, the node action information includes action path sub-information and action type sub-information, and the file dependency template matching the corresponding process node sub-data is constructed based on the corresponding node action information for each of the triple data corresponding to the composite node data, including:

[0016] For each of the triple data corresponding to the composite node data, the file storage path corresponding to the file node sub-data is determined based on the action path sub-information, and the file action type corresponding to the file node sub-data is determined based on the action type sub-information.

[0017] The file storage path and the file action type of each of the triple data are integrated to obtain the file dependency template corresponding to the composite node data.

[0018] In some embodiments, the node action information includes action type sub-information and action time sub-information, and time-dependent relationship reduction processing is performed on the triple data based on the node sub-data and the node action information to obtain time-dependent relationship reduction data, including:

[0019] Based on the action time sub-information corresponding to each of the triple data, the corresponding triple data is sorted to obtain an action trigger sequence.

[0020] In response to the action type sub-information of two adjacent triple data in the action trigger sequence being consistent, the two adjacent triple data are merged to obtain the time-dependent relationship reduction data.

[0021] In some embodiments, the method further includes, after the time-dependent relationship reduction processing of the triple data based on the node sub-data and the node action information, generating a log analysis graph according to the time-dependent relationship reduction data, specifically including:

[0022] generating an analysis graph node based on the sequence of the triple data in the action trigger sequence;

[0023] generating a node pointing edge between each analysis graph node based on the action type sub-information;

[0024] generating a log analysis graph according to each analysis graph node and the node pointing edge between each analysis graph node.

[0025] In some embodiments, the analysis graph node includes an old class graph node and a new class graph node, and the generating of the analysis graph node based on the sequence of the triple data in the action trigger sequence includes:

[0026] generating the analysis graph node based on the sequence of the triple data in the action trigger sequence, and in response to the newly generated analysis graph node and the existing analysis graph node corresponding to the same node sub-data, determining the existing analysis graph node as the old class graph node and the newly generated analysis graph node as the new class graph node based on the action type sub-information related to the existing analysis graph node in the action trigger sequence.

[0027] In some embodiments, the node sub-data is a process node sub-data or a file node sub-data, and the method further includes, after the operation analysis of the original log data to obtain the target cache triple data:

[0028] based on the triple data, if the file node sub-data and at least two process node sub-data have corresponding node action information, determining the corresponding file node sub-data as a process interconnection sub-data;

[0029] performing time-dependent relationship reduction processing based on the process interconnection sub-data to obtain the target reduction log.

[0030] To achieve the above object, a second aspect of the embodiments of the present application proposes an audit log reduction system, the system comprising:

[0031] An original log acquisition module is configured to acquire original log data of a target cache;

[0032] An original log analysis module is configured to analyze the original log data to obtain a plurality of triple data of the target cache, wherein each triple data comprises two node sub-data and node action information between the node sub-data;

[0033] A file reference reduction module is configured to reduce the triple data based on the node sub-data and the node action information to obtain file reference reduction data;

[0034] A time dependency reduction module is configured to reduce the triple data based on the node sub-data and the node action information to obtain time dependency reduction data;

[0035] A reduced log acquisition module is configured to extract the file reference reduction data and the time dependency reduction data to obtain a target reduced log.

[0036] To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, which comprises a memory and a processor, the memory stores a computer program, and the processor implements the audit log reduction method of the first aspect when executing the computer program.

[0037] To achieve the above object, a fourth aspect of the embodiments of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the audit log reduction method of the first aspect.

[0038] The audit log reduction method, system, electronic device and storage medium provided by the present application acquire original log data, input the original log data into a target cache for operation analysis to obtain a plurality of triple data of the target cache, then reduce the triple data based on node sub-data and node action information to obtain file reference reduction data, and reduce the triple data based on time dependency to obtain time dependency reduction data. Then, the file reference reduction data and the time dependency reduction data are extracted to obtain a target reduced log. Therefore, the original log data is input into the target cache for real-time analysis, and the triple data obtained by real-time analysis is reduced based on file reference and time dependency, so that the original log data is reduced in file reference and time dependency, and the original log data is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 is a flowchart of the audit log reduction method provided by the embodiments of the present application;

[0040] Figure 2 is a flowchart of step S103 in Figure 1

[0041] Figure 3 is a flowchart of step S202 in Figure 2

[0042] Figure 4 is a flowchart of step S104 in Figure 1

[0043] Figure 5 is a flowchart after step S402 in Figure 4

[0044] Figure 6 is a flowchart after step S102 in Figure 1

[0045] Figure 7 is a specific example diagram of the file reference reduction processing and the time dependency reduction processing provided by the embodiments of the present application;

[0046] Figure 8 is a structural schematic diagram of the audit log reduction system provided by the embodiments of the present application;

[0047] Figure 9 is a hardware structural schematic diagram of the electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0048] In order to make the objects, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not intended to limit the present application.

[0049] It should be noted that although the functional modules are divided in the system schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the system or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0051] ​​​​​The embodiment of the present application provides an audit log reduction method, system, electronic device and storage medium, and aims to reduce the audit log.

[0052] The audit log reduction method, system, electronic device and storage medium provided by the embodiment of the present application are specifically described through the following embodiment. First, the audit log reduction method in the embodiment of the present application is described.

[0053] The audit log reduction method provided by the embodiment of the present application can be applied to a terminal, can be applied to a server end, and can also be software running in the terminal or the server end. In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer or the like; the server end can be configured as an independent physical server, can be configured as a server cluster or a distributed system composed of multiple physical servers, can also be configured as a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN and big data and artificial intelligence platform; and the software can be an application for implementing the audit log reduction method, but is not limited to the above forms.

[0054] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0055] Figure 1 The method in the optional flowchart of the audit log reduction method provided by the embodiment of the present application can include but is not limited to steps S101 to S105. Figure 1

[0056] Step S101, obtaining original log data;

[0057] ​In step S102, the original log data is input into the target cache for operation analysis to obtain a plurality of triple data of the target cache; each triple data includes two node sub-data and node action information between the node sub-data.

[0058] In step S103, file reference reduction processing is performed on the triple data based on the node sub-data and the node action information to obtain file reference reduction data.

[0059] In step S104, time-dependent relationship reduction processing is performed on the triple data based on the node sub-data and the node action information to obtain time-dependent relationship reduction data.

[0060] In step S105, association extraction is performed based on the file reference reduction data and the time-dependent relationship reduction data to obtain a target reduced log.

[0061] The steps S101 to S105 shown in the embodiments of the present application obtain original log data, input the original log data into the target cache for operation analysis to obtain a plurality of triple data of the target cache, then perform file reference reduction processing on the triple data based on the node sub-data and the node action information to obtain file reference reduction data, and perform time-dependent relationship reduction processing on the triple data to obtain time-dependent relationship reduction data. Then, association extraction is performed based on the file reference reduction data and the time-dependent relationship reduction data to obtain a target reduced log. Therefore, the original log data is input into the target cache for real-time analysis, and the triple data obtained by real-time analysis is subjected to file reference reduction processing and time-dependent relationship reduction processing, so that the original log data is reduced in file reference and time dimension, and the reduction of the original log data is realized.

[0062] In step S101 of some embodiments, the original log data is obtained, which is the data of the audit log without reduction processing or compression processing.

[0063] In step S102 of some embodiments, the original log data is input into the target cache for operation analysis to obtain a plurality of triple data, wherein the triple data refers to the triple form of data commonly used for conversion into graph data, the triple data includes two node sub-data and node action information between the node sub-data, and the node action information includes the relationship between the two node sub-data, the operation type and operation time between the nodes, and other attribute information.

[0064] It should be noted that the mass and long-term storage requirement of the audit log brings excessive space and processing overhead, so that the life cycle of the audit log is limited to a few days, which is not enough to track multi-step attacks lasting for months or years. A large number of audit logs make it very difficult to detect and track potential malicious activities on the host. The related technology can alleviate this problem through cold storage-oriented log reduction technology, but many of these technologies are limited to offline batch processing of data, which hinders timely response to attacks and generates huge storage costs and network transmission bandwidth costs at the terminal, and is not convenient for subsequent traceability investigation, reducing the timeliness of attack detection.

[0065] The present application can parse the original log data in real time by inputting the original log data into the target cache for operation analysis, and can reduce the processing in time, so as to facilitate the discovery of potential threats in time and increase the timeliness of attack detection.

[0066] In some embodiments, the target cache is a Least Recently Used (LRU) cache, and the target cache is connected to a log database for storing original log data. The operation of the LRU cache is based on the assumption that data that has not been used for a long time is also less likely to be used in the future. Therefore, when the amount of data in the LRU cache reaches a threshold, the least recently used data is removed, and the removed original log data is stored in the log database. In this way, the original log data can be parsed more efficiently, and the frequently used original log data can be efficiently retained in the target cache, and the infrequently used original log data can be saved in the log database. In the case where the original log data cannot be found in the target cache, the log database is queried. Thus, the costly database query operation is minimized.

[0067] It should be noted that the entities in the original log data and the relationships between the entities are stored separately, and the relationships between the entities generally do not contain specific attributes between two entities. In the related art, in order to generate triples containing attributes of entities, offline reduction technology uses a double offline traversal method to parse original log data to generate triple data, that is, all entities in the original log data are first offline traversed, and a mapping table is created to store all entities and a specific ID information, and then all relationships between entities are traversed again. The specific attribute information of the entity is obtained through the mapping table to generate the final triple data. This requires a costly database query operation.

[0068] The application can online index entities and corresponding relations by connecting the LRU cache and the log database, and online parsing the original log data without twice traversal query, so as to more efficiently parse the original log data and generate triple data.

[0069] In step S103 of some embodiments, based on the node sub-data and the node action information, the triple data is subjected to file reference reduction processing to obtain file reference reduction data, thereby helping to alleviate the dependency explosion problem.

[0070] Referring to Figure 2 In some embodiments, the node sub-data is process node sub-data or file node sub-data, each triple data includes a corresponding process node sub-data, a corresponding node action information and a corresponding file node sub-data, and step S103 can include but is not limited to steps S201 to S203:

[0071] In step S201, the same process node sub-data possessed by each triple data is determined as composite node data.

[0072] In step S202, for each triple data corresponding to the composite node data, a file dependency template matching the corresponding process node sub-data is constructed based on the corresponding node action information.

[0073] In step S203, based on the file dependency template, the triple data is subjected to node action information merging processing to obtain file reference reduction data.

[0074] In step S201 of some embodiments, the node sub-data usually includes three entity types, process type, file type and network port type, here only the process node sub-data corresponding to the process type entity and the file node sub-data corresponding to the file type entity are considered.

[0075] In some cases, there are some process node sub-data existing in multiple triple data, i.e. in multiple triple data, multiple process node sub-data are essentially the same, then this process node sub-data is determined as composite node data.

[0076] In step S202 of some embodiments, for each triple data corresponding to the composite node data, a file dependency template matching the corresponding process node sub-data is constructed according to the node action information of these triple data.

[0077] Referring to Figure 3 In some embodiments, the node action information includes action path sub-information and action type sub-information, and step S202 can include but is not limited to steps S301 to S302:

[0078] In step S301, for each triple data corresponding to the composite node data, a file storage path corresponding to the file node data is determined based on the action path sub-information, and a file action type corresponding to the file node data is determined based on the action type sub-information.

[0079] In step S302, the file storage path and the file action type of each triple data are integrated to obtain a file dependency template corresponding to the composite node data.

[0080] In step S301 of some embodiments, the node action information includes action path sub-information and action type sub-information, the action type sub-information represents operation information performed by an entity represented by node sub-data on another entity, which can be a read operation or a write operation, and the action path sub-information represents a file storage path accessed by the entity represented by the node sub-data when performing the operation.

[0081] For each triple data corresponding to the composite node data, the file storage path corresponding to the file node data is determined according to the action path sub-information, and the operation type performed by the composite node data on the file node data, that is, the file action type, is determined according to the action type sub-information.

[0082] In step S302 of some embodiments, the file storage path and the file action type of each triple data are integrated, and a file dependency template is constructed based on the same file action type and the similarity of the file storage path.

[0083] It should be noted that the entity represented by the composite node data, that is, a process, may interact with a large number of reference dependent files when performing a task, resulting in a dependency explosion problem, such as a large number of temporary files generated by process-related code, code libraries referenced by the process, and the like. The triple data generated from this information is highly related to a process node sub-data, resulting in a composite node data. The number of triple data generated from this information is extremely large, and the value of using this information to perform trace analysis to reconstruct an attack path or find potential threat behaviors is small for those skilled in the art. The goal of trace analysis is to identify traces of malicious activities by carefully reviewing audit logs, and the analysis process involves many trace graph walk queries. How to improve the performance of this type of query is a very important task. Therefore, it is necessary to reduce these referenced triple data to reasonably reduce the audit logs, which can not only reduce the storage cost of the audit logs, but also facilitate subsequent trace analysis.

[0084] The file dependency template is built by integrating the file storage path and the file action type, because the referenced files associated with the process represented by the composite node data, such as library files, are usually stored in a folder dedicated to storing library files, configuration files, script files, style sheets, etc. The file node sub-data represented by these files has highly similar action path sub-information in the triple data, and the operations recorded by the action type sub-information are all read operations. Therefore, a file dependency template with a read operation as the file action type and a referenced folder as the file storage path can be built, and the triple data generated due to the reference of files by the process node sub-data can be merged to reduce the amount of triple data.

[0085] Similarly, if the temporary files are generated by the process itself, they are usually stored in a specific temporary folder, and the operations recorded by the action type sub-information are mostly write operations. Therefore, a file dependency template with a write operation as the file action type and a temporary folder as the file storage path can be built, and the triple data generated due to the generation of temporary files can be merged.

[0086] In some exemplary embodiments, some file node sub-data associated with the composite node data is obtained from some triple data corresponding to the composite node data. These file node sub-data includes the following file storage paths: “ / project / mount / 8688.txt”, “ / project / mount / 1245.txt”, “ / project / mount / 8563.txt”, and the corresponding file action types are the same, all being write operations. Those skilled in the art can understand that these are temporary files generated by the entity represented by the process node sub-data during runtime. For this, the file dependency template can be generated by using steps S301 to S302: the file storage path is “ / project / mount / *”, and the file action type is a write operation. The “*” is a wildcard. Using this file dependency template, the three triple data corresponding to the three file node sub-data in the above example can be merged into triple data with the file dependency template as a representative entity, thereby achieving the reduction of the audit log. It should be noted that the above content is a demonstration example of the file dependency template proposed by the present application, and the purpose is to clarify rather than limit the specific implementation details of the technical solutions of the present application.

[0087] In step S203 of some embodiments, based on the file dependency template, the node action information of the triple data is merged to obtain file reference reduction data. In this process, if the same process node sub-data repeatedly operates on the same file node sub-data, the file dependency template can also be merged. It can be understood that the file storage path is the same and the file action type is also the same.

[0088] Through steps S201 to S203, the file storage path and the file action type of the file node sub-data in each triple data are obtained based on the action path sub-information and the action type sub-information contained in the node action information. Then, the file storage path and the file action type of each triple data are integrated. Based on the similarity of the same file action type and file storage path, the file dependency template is constructed. Thus, a large number of similar triple data are merged, the low-value information of the audit log is reduced, which helps to reduce the storage cost and is also beneficial to the analysis and management of the audit log by subsequent security analysts.

[0089] In step S104 of some embodiments, based on the node sub-data and the node action information, the triple data is processed to obtain time-dependent relationship reduction data.

[0090] Please refer to Figure 4 In some embodiments, the node action information includes action type sub-information and action time sub-information. Step S104 can include but is not limited to steps S401 to S402:

[0091] In step S401, based on the action time sub-information corresponding to each triple data, the corresponding triple data is sorted to obtain an action trigger sequence.

[0092] In step S402, in response to the action type sub-information of the adjacent two triple data in the action trigger sequence being consistent, the adjacent two triple data are merged to obtain time-dependent relationship reduction data.

[0093] In step S401 of some embodiments, the node action information includes action type sub-information and action time sub-information. The action type sub-information represents the operation information performed by the entity represented by the node sub-data on another entity. The operation can be a read operation or a write operation. The action time sub-information represents the time when the entity represented by the node sub-data performs the operation.

[0094] Based on the action time sub-information corresponding to each triple data, the corresponding triple data is sorted to obtain an action trigger sequence. The action trigger sequence is a plurality of triple data arranged in time sequence.

[0095] In step S402 of some embodiments, in the action trigger sequence, if the action type sub-information corresponding to the two adjacent triple data is the same, that is, the entity performing the operation, the entity being operated, and the operation performed are the same, then the two adjacent triple data are merged to obtain time-dependent relationship reduction data. It can be understood that this indicates that the two adjacent triple data represent the same operation performed by one entity on another entity at two different times. It should be noted that the merging operation here can be to eliminate the old triple data in the two triple data.

[0096] Through steps S401 to S402, the triple data is sorted in time sequence based on the action time sub-information to obtain the action trigger sequence, and the triple data with consistent action type sub-information is merged, which can clearly show how the operations between the entities represented by the node sub-data develop over time, and can simplify the audit log, reduce data redundancy, and improve the analysis efficiency of subsequent traceability analysis.

[0097] Referring to Figure 5 In some embodiments, after step S402, the method further includes generating a log analysis graph based on the time-dependent relationship reduction data, which can include but is not limited to steps S501 to S503:

[0098] In step S501, based on the order of the triple data in the action trigger sequence, an analysis graph node is generated;

[0099] In step S502, based on the action type sub-information, a node pointing edge between the analysis graph nodes is generated;

[0100] In step S503, a log analysis graph is generated according to each analysis graph node and the node pointing edge between the analysis graph nodes.

[0101] In step S501 of some embodiments, the log analysis graph is a traceability graph converted from the audit log, and based on the order of the triple data in the action trigger sequence, an analysis graph node corresponding to the node sub-data is generated;

[0102] In some embodiments, based on the analysis of the original log data by the target cache at different times, new triple data is generated, the new triple data is sorted according to the action time sub-information, and the action trigger sequence is updated, and then a new analysis graph node is generated;

[0103] In step S502 of some embodiments, in response to the newly generated parsing graph node and the existing parsing graph node corresponding to the same node sub-data, based on the action type sub-information related to the existing parsing graph node in the action trigger sequence, the existing parsing graph node is determined as an old class graph node, and the newly generated parsing graph node is determined as a new class graph node.

[0104] In the log parsing graph, in order to as clearly and simply as possible show the relationship between the entities represented by the node sub-data at different times, a new version of the parsing graph node is generated based on the existing parsing graph node as a new class graph node. If in the action trigger sequence, a node sub-data performs an operation at a first time, and after a number of triple data in the action trigger sequence, the same node sub-data performs an operation at a second time, in order to prevent the time sequence of the two operations from being confused in the generation of the log parsing graph, based on the action type sub-information related to the existing parsing graph node in the action trigger sequence, it is judged whether a new version of the parsing graph node is generated for the corresponding parsing graph node in the node sub-data. If the existing parsing graph node includes multiple versions, only the action type sub-information related to the latest old class graph node in the old version is used for judgment.

[0105] In response to the newly generated parsing graph node and the existing parsing graph node corresponding to the same node sub-data, based on the action type sub-information related to the existing parsing graph node before this moment in the action trigger sequence, if the action type sub-information indicates that for the existing parsing graph node, the operation is a write operation, the parsing graph node corresponding to the write operation is generated as a new class graph node. When the action type sub-information indicates that for the existing parsing graph node, the operation is a read operation, the parsing graph node corresponding to the read operation is generated as a new class graph node. After the old class graph node and the new class graph node are determined, the action type sub-information between the old class graph node and the new class graph node is determined as an update.

[0106] Based on the action type sub-information, node pointing edges between the parsing graph nodes are generated, based on the execution operation of the action type sub-information, node pointing edges between the parsing graph nodes are generated, the node pointing edges connect the parsing graph nodes and have direction information and action time sub-information. The direction is generally related to the flow direction of the data. For example, the action type sub-information between the parsing graph node A and the parsing graph node B is that the parsing graph node A performs a write operation on the parsing graph node B, then the node pointing edge should be from the parsing graph node A to the parsing graph node B. Similarly, if the parsing graph node A performs a read operation on the parsing graph node B, then the node pointing edge should be from the parsing graph node B to the parsing graph node A. For the old class graph node and the new class graph node, the node pointing edge should be from the old class graph node to the new class graph node.

[0107] In step S503 of some embodiments, a log analysis graph is generated according to each analysis graph node and the node pointing edges between the analysis graph nodes.

[0108] Through steps S501 to S503, the analysis graph nodes are generated based on the action trigger sequence after the time dependency reduction, and the analysis graph nodes are updated after new data is updated, and when the newly generated analysis graph nodes correspond to the same node sub-data as the existing analysis graph nodes, the node pointing edges between the analysis graph nodes are generated based on the action type sub-information, and then the node pointing edges between the analysis graph nodes are generated based on the action type sub-information, and the log analysis graph is generated according to each analysis graph node and the node pointing edges between the analysis graph nodes. Thus, the relationship between the entities represented by the analysis graph nodes can be clearly displayed, and by associating the newly generated analysis graph nodes with the existing analysis graph nodes and marking them as "new class graph nodes" or "old class graph nodes", confusion caused by repeated operations can be avoided. This enables users to clearly distinguish between what has recently changed and what has been in the past. And by distinguishing between read operations and write operations, the direction of the data flow can be more accurately understood. This refinement is particularly important for security audits, as it helps to identify potential data leakage risks or other unauthorized data access situations, facilitating analysts to quickly locate the problem.

[0109] And in the case where a risk has been found, through the log analysis graph, it is easier to trace the root cause of the problem, because all operations are recorded and arranged in chronological order, which helps to quickly locate the time point when the error occurred and its context.

[0110] It should be noted that the construction of the log analysis graph can not only be performed after the time dependency reduction processing, but also after the file reference reduction processing, and the log analysis graph can be constructed based on the processed triple data, because triple data is suitable for conversion into a provenance graph.

[0111] In step S105 of some embodiments, the target reduced log is obtained by association extraction based on the file reference reduced data and the time dependency reduced data. After the triple data is subjected to file reference reduction processing and time dependency reduction processing respectively, the data is extracted and associated based on the same node sub-data from the file reference reduced data and the time dependency reduced data, and the target reduced log is obtained. It should be noted that when the file reference reduced data and the time dependency reduced data are associated, the association is also based on the time sequence of the action time sub-information. The target reduced log can be extracted for local storage, or can be stored in a log database connected to the target cache.

[0112] In some embodiments, two areas are divided in the target cache, a file reference reduction area and a time dependency reduction area, the file reference reduction area performs file reference reduction processing on the triple data, and the time dependency reduction area performs time dependency reduction processing on the triple data.

[0113] When the file reference reduction area is insufficient in space, the file reference reduction data associated with the oldest process node child data in the file reference reduction area that has been subjected to file reference reduction processing is extracted to the time dependency reduction area and is associated with the time dependency reduction data to obtain the target reduction log.

[0114] When the time dependency reduction area is insufficient in space, the oldest node child data in the time dependency reduction area is popped out to generate the target reduction log. If the popped-out node child data is process node child data, the same process node child data is found in the file reference reduction area based on the process node child data, if no file reference reduction processing is performed, file reference reduction processing is immediately performed, then the file reference reduction result is extracted and associated with the time dependency reduction data associated with the popped-out process node child data to obtain the target reduction log.

[0115] The results of the experiment based on this embodiment show that when the target cache is used as an online log parser, the overhead and parsing efficiency remain almost constant, the plateau throughput for the log data set is 63,387 events / second, and the average memory overhead is 190.5MB. These values are completely acceptable for online real-time reduction. The overall average CPU utilization of the reducer is 1%, which increases only when the triple data is input to the time dependency reduction area to generate the log parsing graph. The experiment shows that the technical solution of the present application has strong real-time reduction capability, and the resource occupation is also acceptable for common terminal hosts.

[0116] In some embodiments, a log parsing graph is generated based on the time dependency reduction result, and when the target reduction log is obtained by associating and extracting based on the file reference reduction data and the time dependency reduction data, the log parsing graph is also extracted to be saved in the graph database, so as to perform subsequent trace analysis and other processing based on the log parsing graph. In some embodiments, since the time information in the file reference reduction data is different from the time information in the time dependency reduction data, a new parsing graph node can also be generated in steps S501 to S502 to update the log parsing graph.

[0117] In some embodiments, the file reference reduction processing and the time dependency reduction processing can be sequentially performed in order based on the triple data, and the same target log data can be obtained, but due to the different efficiencies of the triple data in the file reference reduction processing and the time dependency reduction processing, sequentially performing the two processes can make the time for obtaining the target log data longer.

[0118] Referring to Figure 6 In some embodiments, step S102 can include, but is not limited to, steps S601-S602:

[0119] In step S601, based on the plurality of triple data, if the file node sub-data and at least two process node sub-data have corresponding node action information, the corresponding file node sub-data is determined as process communication sub-data.

[0120] In step S602, the time dependency reduction processing is performed based on the process communication sub-data to obtain target reduction log.

[0121] In step S601 of some embodiments, based on the plurality of triple data, if the type of the node sub-data is file node sub-data, and the file node sub-data and at least two process node sub-data have corresponding node action information, it means that the entity represented by the file node sub-data has an association with two process type entities. In this case, it can be considered that the entity represented by the file node sub-data is very important, and the file node sub-data contains information flow records between two process type entities. Therefore, the reference information about the file node sub-data cannot be reduced. The file node sub-data meeting the above condition is determined as process communication sub-data.

[0122] In step S602 of some embodiments, the reference information of the process communication sub-data is not reduced, so the time dependency reduction processing can be performed to obtain the data flow information and the corresponding time sequence between the file type entity represented by the process communication sub-data and the two associated process type entities, to obtain time dependency reduction data, and to determine the target reduction log.

[0123] It should be noted that when performing the file reference reduction processing, the file dependency template is used to merge the triple data, and the file node sub-data that only has node action information with one process node sub-data is targeted. Thus, it is ensured that no key data flow information is lost during the file reference reduction processing.

[0124] The embodiment of the application obtains original log data; inputs the original log data into a target cache for operation analysis to obtain a plurality of triple data of the target cache; then respectively performs file reference reduction processing on the triple data based on node sub-data and node action information to obtain file reference reduction data, and performs time dependency relationship reduction processing on the triple data to obtain time dependency relationship reduction data. Then, based on the file reference reduction data and the time dependency relationship reduction data, association extraction is performed to obtain a target reduction log. As can be seen, the original log data is input into the target cache for real-time analysis, and the triple data obtained by real-time analysis is subjected to file reference reduction processing and time dependency relationship reduction processing, respectively, so that the original log data is reduced in file reference and time dimension dependency relationship, and the reduction of the original log data is realized.

[0125] Please refer to Figure 7In one exemplary instance, the file reference reduction zone is subjected to file reference reduction processing, and the time-dependent relationship reduction zone is subjected to time-dependent relationship reduction processing. In the file reference reduction processing, P1 is a process node child data. As can be seen from the file dependency template example section, F3, F4, and F5 associated with P1 are highly similar in file storage path and are both wr i te operations, that is, write operations. Therefore, a file dependency template is created, F3, F4, and F5 are merged into F5t, and the corresponding node action information is merged. As for F6 and P1, a file dependency template can also be constructed based on the same file storage path and read operations, that is, read operations. Therefore, a file dependency template is also created, and the node action information between F6 and P1 is merged. In the time-dependent relationship reduction zone, an action trigger sequence is constructed from the triad data at different times. In t0 and t1, the triad data "2: P1 wr i te F2", "3: P1 wr i te F2", "4: P1 wr i te F2" are the same in the entity performing the operation, the entity being operated on, and the operation performed. Therefore, these adjacent triad data are merged. The result can be seen from the t1 time updated log parsing graph that the action time sub-information attached to the node pointing edge between P1 and F2 is the latest action time sub-information after the merging processing, and there is no other node pointing edge between the new class diagram nodes between P1 and F2. As for "5: P1 read F1" in t1, in the action trigger sequence, there is "1: P1 read F1" in t0, which is actually the same node child data. Therefore, based on the action type sub-information related to the existing parsing graph node in the action trigger sequence, the existing parsing graph node is determined as an old class diagram node, and the newly generated parsing graph node is determined as a new class diagram node. In t0, the action type sub-information of P1 indicates that the operation of the existing parsing graph node is a read operation. Therefore, the parsing graph node corresponding to the read operation, that is, the P1 node, is generated as a new class diagram node. As can be seen from the t1 time updated log parsing graph, the P1 node has P1 1 old class diagram node and P1 2 new class diagram node. In t2, the file reference reduction data in the file reference reduction zone is merged based on the same associated time-dependent relationship reduction of the P1 node. It should be noted that the merging is performed based on the time sequence, and finally the t2 time updated log parsing graph is obtained. The target reduction log can be extracted from the time-dependent relationship reduction zone.

[0126] Please refer to Figure 8The embodiment of the application further provides an audit log reduction system, which can implement the audit log reduction method, and the system comprises:

[0127] An original log acquisition module is configured to acquire original log data;

[0128] An original log analysis module is configured to input the original log data into a target cache for operation analysis, so as to obtain a plurality of triadic data of the target cache; each triadic data comprises two node sub-data and node action information between the node sub-data;

[0129] A file reference reduction module is configured to perform file reference reduction processing on the triadic data based on the node sub-data and the node action information, so as to obtain file reference reduction data;

[0130] A time dependency reduction module is configured to perform time dependency reduction processing on the triadic data based on the node sub-data and the node action information, so as to obtain time dependency reduction data;

[0131] A reduced log acquisition module is configured to perform association extraction based on the file reference reduction data and the time dependency reduction data, so as to obtain a target reduced log.

[0132] The specific implementation of the audit log reduction system is basically the same as the specific embodiment of the audit log reduction method, and thus will not be described herein.

[0133] The embodiment of the application further provides an electronic device, which comprises a memory and a processor. The memory stores a computer program, and the processor implements the audit log reduction method when executing the computer program. The electronic device can be any intelligent terminal, such as a tablet computer or a vehicle-mounted computer.

[0134] Please refer to Figure 9 , Figure 9 The hardware structure of the electronic device of another embodiment is illustrated, and the electronic device comprises:

[0135] The processor 901 can be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute a related program to implement the technical solutions provided by the embodiments of the application.

[0136] The memory 902 can be implemented in the form of a read only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 902 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 902 and are called and executed by the processor 901 to perform the audit log reduction method of the embodiments of the present application;

[0137] The input / output interface 903 is configured to realize information input and output.

[0138] The communication interface 904 is configured to realize the communication interaction between the device and other devices, and the communication can be realized by a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, a WI-FI, a Bluetooth, etc.).

[0139] The bus 905 is configured to transmit information between various components (for example, the processor 901, the memory 902, the input / output interface 903, and the communication interface 904) of the device.

[0140] The processor 901, the memory 902, the input / output interface 903, and the communication interface 904 are connected to each other through the bus 905 to realize the communication connection between the device.

[0141] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the above-mentioned audit log reduction method.

[0142] The memory is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0143] The audit log reduction method, system, electronic device and storage medium provided by the embodiments of the present application obtain original log data; input the original log data into a target cache for operation analysis to obtain a plurality of triple data of the target cache; then, based on node sub-data and node action information, perform file reference reduction processing on the triple data to obtain file reference reduction data, and perform time dependency relationship reduction processing on the triple data to obtain time dependency relationship reduction data. Then, based on the file reference reduction data and the time dependency relationship reduction data, perform association extraction to obtain a target reduction log. Therefore, the original log data is input into the target cache for real-time analysis, and the triple data obtained by real-time analysis is subjected to file reference reduction processing and time dependency relationship reduction processing, so that the original log data is reduced in file reference and time dimension dependency, and the original log data is reduced.

[0144] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0145] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and can include more or fewer steps than the figures shown, or combine certain steps or different steps.

[0146] The system embodiments described above are only schematic, and the units illustrated as separate components can or can not be physically separated, that is, can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0147] Those skilled in the art can understand that all or some of the steps in the above disclosed method, the functions of the modules / units in the system and the device can be implemented as software, firmware, hardware and their appropriate combinations.

[0148] The terms "first", "second", "third", "fourth", and the like in the description of this application and in the claims hereof, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so termed herein is solely for the convenience of the reader and does not limit the scope of the application. It is also to be understood that the description and examples in this application are intended to cover all possible combinations where any of the several elements can represent one or more elements.

[0149] It should be understood that, in the application, "at least one" means one or more, "multiple" means two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0150] In several embodiments provided in the application, it should be understood that the disclosed system and method can be implemented in other ways. For example, the above-described system embodiments are only illustrative, for example, the division of the above-mentioned units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. The coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0151] The units described above as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment of the application.

[0152] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit.

[0153] When the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes multiple instructions used to cause a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various other media that can store programs.

[0154] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not intended to limit the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A method for reducing audit logs, characterized in that: The method comprises: Get raw log data; Inputting the original log data into a target cache for operation parsing to obtain a plurality of triplet data of the target cache; wherein each triplet data includes two node sub-data and node action information between the node sub-data; Based on the node sub-data and the node action information, performing file reference reduction processing on the triple data to obtain file reference reduction data; Based on the node sub-data and the node action information, performing time dependency reduction processing on the triple data to obtain time dependency reduced data; Correlation extraction is performed based on the file reference reduction data and the time dependency reduction data to obtain a target reduction log.

2. The method according to claim 1, characterized in that The node sub-data is process node sub-data or file node sub-data, and each triplet of data includes a corresponding process node sub-data, a corresponding node action information, and a corresponding file node sub-data. Based on the node sub-data and the node action information, the triplet of data is subjected to file reference reduction processing to obtain file reference reduction data, including: Determining the same process node sub-data possessed by each of the triple data as composite node data; For each triplet data corresponding to the composite node data, based on the corresponding node action information, constructing a file dependency template that matches the corresponding process node sub-data; Based on the file dependency template, the node action information is merged into the triple data to obtain the file reference reduction data.

3. The method according to claim 2, characterized in that The node action information includes action path sub-information and action type sub-information. For each triplet data corresponding to the composite node data, based on the corresponding node action information, a file dependency template matching the corresponding process node sub-data is constructed, including: For each triplet data corresponding to the composite node data, determining the file storage path corresponding to the file node sub-data based on the action path sub-information, and determining the file action type corresponding to the file node sub-data based on the action type sub-information; The file storage path and the file action type of each triple data are integrated to obtain a file dependency template corresponding to the composite node data.

4. The method according to claim 1, wherein The node action information includes action type sub-information and action time sub-information. The time dependency reduction processing is performed on the triple data based on the node sub-data and the node action information to obtain time dependency reduction data, including: Based on the action time sub-information corresponding to each triplet of data, sorting the corresponding triplet of data to obtain an action trigger sequence; In response to the consistency of the action type sub-information of two adjacent triplet data in the action trigger sequence, the two adjacent triplet data are merged to obtain the time dependency reduction data.

5. The method according to claim 4, characterized in that The method further includes, after performing time dependency reduction processing on the triple data based on the node sub-data and the node action information to obtain time dependency reduced data, generating a log parsing graph based on the time dependency reduced data, specifically including: Generate a parsing graph node based on the order of each triplet data in the action trigger sequence; Based on the action type sub-information, generating node-directed edges between the nodes of the parsing graph; A log parsing graph is generated according to each of the parsing graph nodes and the node-directed edges between the parsing graph nodes.

6. The method according to claim 5, characterized in that The parsing graph node includes an old class graph node and a new class graph node, and the generating of the parsing graph node based on the sequence of the triple data in the action trigger sequence includes: Based on the sequence of each triplet data in the action trigger sequence, the parsing graph node is generated. In response to the newly generated parsing graph node and the existing parsing graph node corresponding to the same node sub-data, based on the action type sub-information related to the existing parsing graph node in the action trigger sequence, the existing parsing graph node is determined as the old class graph node, and the newly generated parsing graph node is determined as the new class graph node.

7. The method according to claim 1, characterized in that The node sub-data is process node sub-data or file node sub-data. After inputting the original log data into the target cache for operation parsing and obtaining a plurality of triplet data of the target cache, the method further includes: Based on the plurality of triplet data, if the file node sub-data and at least two process node sub-data have corresponding node action information, determining the corresponding file node sub-data as process connection sub-data; A time dependency reduction process is performed based on the process connection sub-data to obtain the target reduction log.

8. An audit log reduction system, characterized in that: The system comprises: The original log acquisition module is used to obtain the original log data; The original log parsing module is used to input the original log data into the target cache for operation parsing to obtain a plurality of triplet data of the target cache; wherein each triplet data includes two node sub-data and node action information between the node sub-data; A file reference reduction module, configured to perform file reference reduction processing on the triple data based on the node sub-data and the node action information to obtain file reference reduction data; a time dependency reduction module, configured to perform time dependency reduction processing on the triple data based on the node sub-data and the node action information to obtain time dependency reduced data; The reduction log acquisition module is used to perform associated extraction based on the file reference reduction data and the time dependency reduction data to obtain a target reduction log.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the audit log reduction method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the audit log reduction method according to any one of claims 1 to 7 is implemented.

Images