Audit log reduction method and system, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an audit log reduction method and system, electronic equipment and a storage medium, and belongs to the technical field of audit logs. The method comprises the following steps: acquiring original log data; inputting the original log data into a target cache for operation analysis to obtain multiple pieces of triple data of the target cache; wherein each piece of triple data comprises two pieces of node sub-data and node action information between the node sub-data; based on the node subdata and the node action information, performing file reference reduction processing on the triple data to obtain file reference reduction data; on the basis of the node sub-data and the node action information, performing time dependency relationship reduction processing on the triple data to obtain time dependency relationship reduction data; and performing association extraction based on the file reference reduced data and the time dependency reduced data to obtain a target reduced log. According to the embodiment of the invention, the audit log can be reduced.

Description

Technical Field

[0001] The present application relates to the field of audit log technology, and in particular to an audit log reduction method, system, electronic device and storage medium. Background Art

[0002] Audit log is a log file that records system activities. It is used to track and record the user or system's access and operation behavior to applications, network devices, operating systems, etc. It is essential for revealing and tracking complex network threats. Security analysts can convert audit logs into provenance graphs to more clearly illustrate the chronological order and causal relationship of the execution history on the host. Once suspicious behavior is detected, forensic analysis will reconstruct the attack footprint to help security analysts investigate threats.

[0003] However, the massive and long-term storage requirements of audit logs bring too much space and processing overhead, which limits the life cycle of audit logs to a few days, which is not enough to track multi-step attacks that last for months or years. Large audit logs make it difficult to detect and track potential malicious activities on the host. Therefore, how to reduce audit logs has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The main purpose of the embodiments of the present application is to propose an audit log reduction method, system, electronic device and storage medium, aiming to reduce the audit log.

[0005] To achieve the above purpose, a first aspect of an embodiment of the present application proposes an audit log reduction method, the method comprising:

[0006] Get the original log data;

[0007] Input the original log data into the target cache for operation analysis to obtain a plurality of triplet data of the target cache; wherein each triplet data includes two node sub-data and node action information between the node sub-data;

[0008] Based on the node sub-data and the node action information, performing file reference reduction processing on the triple data to obtain file reference reduction data;

[0009] Based on the node sub-data and the node action information, performing time dependency reduction processing on the triple data to obtain time dependency reduced data;

[0010] Based on the file reference reduction data and the time dependency reduction data, associated extraction is performed to obtain a target reduction log.

[0011] In some embodiments, the node sub-data is process node sub-data or file node sub-data, each of the triple data includes a corresponding process node sub-data, a corresponding node action information and a corresponding file node sub-data, and the file reference reduction processing is performed on the triple data based on the node sub-data and the node action information to obtain the file reference reduction data, including:

[0012] Determine the same process node sub-data possessed by each of the triple data as composite node data;

[0013] For each of the triplet data corresponding to the composite node data, based on the corresponding node action information, construct a file dependency template that matches the corresponding process node sub-data;

[0014] Based on the file dependency template, the node action information is merged into the triple data to obtain the file reference reduction data.

[0015] In some embodiments, the node action information includes action path sub-information and action type sub-information, and for each triplet data corresponding to the composite node data, based on the corresponding node action information, constructing a file dependency template matching the corresponding process node sub-data includes:

[0016] For each of the triplet data corresponding to the composite node data, based on the action path sub-information, determine the file storage path corresponding to the file node sub-data, and based on the action type sub-information, determine the file action type corresponding to the file node sub-data;

[0017] The file storage path and the file action type of each triple data are integrated to obtain a file dependency template corresponding to the composite node data.

[0018] In some embodiments, the node action information includes action type sub-information and action time sub-information, and the time dependency reduction processing is performed on the triple data based on the node sub-data and the node action information to obtain the time dependency reduction data, including:

[0019] Based on the action time sub-information corresponding to each triplet of data, sorting the corresponding triplet of data to obtain an action trigger sequence;

[0020] In response to the consistency of the action type sub-information of two adjacent triplet data in the action trigger sequence, the two adjacent triplet data are merged to obtain the time dependency reduction data.

[0021] In some embodiments, after performing time dependency reduction processing on the triple data based on the node sub-data and the node action information to obtain time dependency reduction data, the method further includes generating a log parsing graph based on the time dependency reduction data, specifically including:

[0022] Generate a parsing graph node based on the sequence of each of the triplet data in the action trigger sequence;

[0023] Based on the action type sub-information, generating node-directed edges between the nodes of the parsing graph;

[0024] A log parsing graph is generated according to each of the parsing graph nodes and the node-directed edges between the parsing graph nodes.

[0025] In some embodiments, the parsing graph node includes an old class graph node and a new class graph node, and the generating of the parsing graph node based on the sequence of the triple data in the action trigger sequence includes:

[0026] Based on the sequence of each triplet data in the action trigger sequence, the parsing graph node is generated, and in response to the newly generated parsing graph node and the existing parsing graph node corresponding to the same node sub-data, based on the action type sub-information related to the existing parsing graph node in the action trigger sequence, the existing parsing graph node is determined as the old class graph node, and the newly generated parsing graph node is determined as the new class graph node.

[0027] In some embodiments, the node sub-data is process node sub-data or file node sub-data, and after performing operation analysis on the original log data to obtain a plurality of triplet data of the target cache, the method further includes:

[0028] Based on the plurality of triplet data, if the file node sub-data and at least two of the process node sub-data have corresponding node action information, determining the corresponding file node sub-data as process connection sub-data;

[0029] A time dependency reduction process is performed based on the process connection sub-data to obtain the target reduction log.

[0030] To achieve the above purpose, a second aspect of an embodiment of the present application proposes an audit log reduction system, the system comprising:

[0031] The original log acquisition module is used to obtain the original log data of the target cache;

[0032] The original log parsing module is used to perform operation parsing on the original log data to obtain a plurality of triplet data of the target cache; wherein each triplet data contains two node sub-data and node action information between the node sub-data;

[0033] A file reference reduction module, configured to perform file reference reduction processing on the triple data based on the node sub-data and the node action information to obtain file reference reduction data;

[0034] A time dependency reduction module, configured to perform time dependency reduction processing on the triple data based on the node sub-data and the node action information to obtain time dependency reduced data;

[0035] The reduction log acquisition module is used to perform associated extraction based on the file reference reduction data and the time dependency reduction data to obtain a target reduction log.

[0036] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and the processor implements the audit log reduction method described in the first aspect when executing the computer program.

[0037] To achieve the above objectives, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the audit log reduction method described in the first aspect above.

[0038] The audit log reduction method, system, electronic device and storage medium proposed in the present application obtain the original log data; input the original log data into the target cache for operation analysis to obtain multiple triple data of the target cache; then, based on the node child data and the node action information, perform file reference reduction processing on the triple data to obtain file reference reduction data, and perform time dependency reduction processing on the triple data to obtain time dependency reduction data. Then, association extraction is performed based on the file reference reduction data and the time dependency reduction data to obtain the target reduction log. It can be seen that the present application reduces the original log data in terms of file reference and dependency in the time dimension by inputting the original log data into the target cache for real-time analysis, and performs file reference reduction processing and time dependency reduction processing on the triple data obtained by the real-time analysis, thereby reducing the original log data. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flowchart of the audit log reduction method provided by an embodiment of the present application;

[0040] Figure 2 yes Figure 1 Flow chart of step S103 in FIG.

[0041] Figure 3 yes Figure 2 Flow chart of step S202 in FIG.

[0042] Figure 4 yes Figure 1 Flow chart of step S104 in FIG.

[0043] Figure 5 yes Figure 4 The flowchart after step S402 in FIG.

[0044] Figure 6 yes Figure 1 The flowchart after step S102 in FIG.

[0045] Figure 7 It is a specific example diagram of file reference reduction processing and time dependency reduction processing provided in the embodiment of the present application;

[0046] Figure 8 It is a structural diagram of the audit log reduction system provided in an embodiment of the present application;

[0047] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0049] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0051] Embodiments of the present application provide an audit log reduction method, system, electronic device, and storage medium, which are intended to reduce audit logs.

[0052] The audit log reduction method, system, electronic device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the audit log reduction method in the embodiments of the present application is described.

[0053] The audit log reduction method provided in the embodiments of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the audit log reduction method, etc., but is not limited to the above forms.

[0054] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0055] Figure 1 is an optional flowchart of the audit log reduction method provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S105.

[0056] Step S101, obtaining original log data;

[0057] Step S102, inputting the original log data into the target cache for operation analysis, and obtaining a plurality of triplet data of the target cache; wherein each triplet data contains two node sub-data and node action information between the node sub-data;

[0058] Step S103, performing file reference reduction processing on the triple data based on the node child data and the node action information to obtain file reference reduction data;

[0059] Step S104, performing time dependency reduction processing on the triple data based on the node child data and the node action information to obtain time dependency reduced data;

[0060] Step S105 , performing association extraction based on the file reference reduction data and the time dependency reduction data to obtain a target reduction log.

[0061] Steps S101 to S105 shown in the embodiment of the present application obtain the original log data; input the original log data into the target cache for operation analysis to obtain multiple triplet data of the target cache; then, based on the node child data and the node action information, perform file reference reduction processing on the triplet data to obtain file reference reduction data, and perform time dependency reduction processing on the triplet data to obtain time dependency reduction data. Then, association extraction is performed based on the file reference reduction data and the time dependency reduction data to obtain the target reduction log. It can be seen that the present application reduces the original log data in terms of file references and dependencies in the time dimension by inputting the original log data into the target cache for real-time analysis, and performs file reference reduction processing and time dependency reduction processing on the triplet data obtained by the real-time analysis, thereby reducing the original log data.

[0062] In step S101 of some embodiments, original log data is obtained, where the original log data is the audit log data that has not been reduced or compressed.

[0063] In step S102 of some embodiments, the original log data is input into the target cache for operation analysis to obtain multiple triple data, wherein the triple data refers to data in triple form commonly used to convert into graph data, and the triple data includes two node sub-data and node action information between the node sub-data. The node action information includes attribute information such as the relationship between the two node sub-data, the operation type and operation time between the nodes.

[0064] It should be noted that the massive and long-term storage requirements of audit logs bring too much space and processing overhead, which limits the life cycle of audit logs to a few days, which is not enough to track multi-step attacks that last for months or years. A large number of audit logs makes it very difficult to detect and track potential malicious activities on the host. In related technologies, log reduction technology for cold storage can alleviate this problem, but many of these technologies are limited to offline batch processing of data, which hinders timely response to attacks and generates huge storage costs and network transmission bandwidth costs at the terminal. It is also inconvenient to carry out subsequent tracing investigations, reducing the timeliness of attack detection.

[0065] By inputting the original log data into the target cache for operation analysis, the application can timely parse the original log data in real time and perform reduction processing in a timely manner, thereby facilitating the timely discovery of potential threats and increasing the timeliness of attack detection.

[0066] In some embodiments, the target cache is a least recently used (Least Recently Used, LRU) cache, and the target cache is connected to a log database for storing raw log data. The operation of the LRU cache is based on an assumption: the probability of data that has not been used for a long time being used in the future is not great. Therefore, when the amount of data in the LRU cache reaches a threshold, the least recently used data will be removed, and the removed raw log data will be stored in the log database. With such a design, the raw log data can be traversed and parsed more efficiently, and the commonly used raw log data can be efficiently retained in the target cache, and the infrequently used raw log data can be stored in the log database. When the raw log data cannot be found in the target cache, the log database is queried. Thereby minimizing the costly database query operation.

[0067] It should be noted that the entities and the relationships between entities in the original log data are stored separately, and the relationship between entities generally does not contain the specific attributes between the two entities. In the related art, in order to generate triples containing the attributes of the entities, the offline reduction technology uses a double offline traversal method to parse the original log data to generate triple data, that is, first traverse all entities in the original log data offline, and create a mapping table to store all entities and a specific ID information, and then traverse the relationship between all entities for the second time, and obtain the ID information corresponding to the entity to the specific attribute information of the entity through the mapping table to generate the final triple data. This requires costly database query operations.

[0068] The present application connects the LRU cache with the log database to perform online analysis of the original log data without the need for two traversal queries. It can index entities and corresponding relationships online, parse the original log data more efficiently, and generate triple data.

[0069] In step S103 of some embodiments, file reference reduction processing is performed on the triple data based on the node child data and the node action information to obtain file reference reduction data, thereby helping to alleviate the dependency explosion problem.

[0070] See also Figure 2 In some embodiments, the node sub-data is process node sub-data or file node sub-data, each triple data includes a corresponding process node sub-data, a corresponding node action information and a corresponding file node sub-data, and step S103 may include but is not limited to steps S201 to S203:

[0071] Step S201, determining the same process node sub-data possessed by each triple data as composite node data;

[0072] Step S202, for each triplet data corresponding to the composite node data, based on the corresponding node action information, construct a file dependency template that matches the corresponding process node sub-data;

[0073] Step S203: Based on the file dependency template, the node action information of the triple data is merged to obtain file reference reduction data.

[0074] In step S201 of some embodiments, the node sub-data usually includes three entity types, process type, file type and network port type. Here, only the process node sub-data of the entity corresponding to the process type and the file node sub-data of the entity corresponding to the file type are considered.

[0075] In some cases, some process node sub-data exist in multiple triple data at the same time, that is, in multiple triple data, multiple process node sub-data are essentially the same, then such process node sub-data is determined as composite node data;

[0076] In step S202 of some embodiments, for each triple data corresponding to the composite node data, a file dependency template matching the corresponding process node sub-data is constructed according to the node action information corresponding to these triple data.

[0077] See also Figure 3 In some embodiments, the node action information includes action path sub-information and action type sub-information. Step S202 may include but is not limited to steps S301 to S302:

[0078] Step S301, for each triplet data corresponding to the composite node data, based on the action path sub-information, determining the file storage path corresponding to the file node sub-data, and based on the action type sub-information, determining the file action type corresponding to the file node sub-data;

[0079] Step S302: Integrate the file storage path and file action type of each triple data to obtain a file dependency template corresponding to the composite node data.

[0080] In step S301 of some embodiments, the node action information includes action path sub-information and action type sub-information. The action type sub-information indicates the operation information performed by the entity represented by the node sub-data on another entity. The operation can be a read operation or a write operation. The action path sub-information indicates the file storage path accessed by the entity represented by the node sub-data when performing the operation.

[0081] For each triplet data corresponding to the composite node data, the file storage path corresponding to the file node sub-data is determined according to the action path sub-information, and the type of operation performed by the composite node data on the file node sub-data, that is, the file action type, is determined according to the action type sub-information.

[0082] In step S302 of some embodiments, the file storage paths and file action types of each triple data are integrated, and a file dependency template is constructed based on the same file action type and similarity of the file storage paths.

[0083] It should be noted that the entity represented by the composite node data, that is, a process, may interact with a large number of referenced dependent files when executing tasks, resulting in dependency explosion problems, such as a large number of temporary files generated by process-related codes, code libraries referenced by processes, etc. The triple data generated by this information will be highly related to a process node sub-data, resulting in composite node data. However, the triple data generated by this information itself is extremely large in number, and for technicians in this field, the value of using the triple data generated by this information for source tracing analysis to reconstruct attack paths or find potential threat behaviors is very small. The goal of source tracing analysis is to identify traces of malicious activities by carefully reviewing audit logs. The analysis process involves many source tracing graph walk queries. How to improve the query performance of this type is a very critical task. Therefore, it is necessary to reduce these referenced triple data to reasonably reduce the audit logs, which can not only reduce the storage cost of the audit logs, but also facilitate subsequent source tracing analysis.

[0084] The file dependency template is constructed by integrating the file storage path and the file action type because the reference files associated with the process represented by the composite node data, such as library files, are usually stored in a folder that is specifically used to store library files, configuration files, script files, style sheets, etc. The file node sub-data represented by these files generally have highly similar action path sub-information in the triple data, and the operations recorded in the action type sub-information are all read operations. Therefore, a file dependency template with a file action type of read operation and a file storage path of a reference folder can be constructed to merge these triple data, thereby reducing the large amount of triple data generated by the process node sub-data referencing files.

[0085] Similarly, if the temporary files are generated by the process itself, they are generally stored in a specific temporary folder, and most of the operations recorded by the action type sub-information are write operations. Therefore, a file dependency template with a file action type of write operation and a file storage path of a temporary folder can be constructed to merge the triple data generated by the generation of temporary files.

[0086] In some exemplary embodiments, some associated file node sub-data are obtained from some triple data corresponding to the composite node data, and these file node sub-data include the following file storage paths: " / project / mount / 8688.txt", " / project / mount / 1245.txt", " / project / mount / 8563.txt", and the corresponding file action types are the same, all of which are write operations. Those skilled in the art can understand that these may be temporary files generated by the entity represented by the process node sub-data during runtime. In this regard, a file dependency template can be generated using steps S301 to S302: the file storage path is " / project / mount / *", and the file action type is a write operation. Among them, "*" is used as a wildcard. Using this file dependency template, the three triple data corresponding to the three file node sub-data in the above example can be merged into triple data with the file dependency template as a representative entity, thereby achieving a reduction in the audit log. It should be noted that the above content is a demonstration example of the file dependency template proposed in the present invention, and its purpose is to illustrate but not limit the specific implementation details of the technical solution of the present invention.

[0087] In step S203 of some embodiments, the node action information of the triple data is merged based on the file dependency template to obtain file reference reduction data. In this process, if the same process node sub-data has repeated operations on the same file node sub-data, they will also be merged by the file dependency template. It can be understood that the file storage path is the same and the file action type is also the same.

[0088] Through steps S201 to S203, the file storage path and file action type of the file node sub-data in each triple data are obtained based on the action path sub-information and action type sub-information included in the node action information, and then the triple data are integrated according to the file storage path and file action type. Based on the similarity of the same file action type and file storage path, a file dependency template is constructed, so that a large number of similar triple data are merged, which reduces the low-value information of the audit log, helps to reduce storage costs, and is also beneficial to the subsequent analysis and management of the audit log by security analysts.

[0089] In step S104 of some embodiments, time dependency reduction processing is performed on the triple data based on the node child data and the node action information to obtain time dependency reduced data;

[0090] See also Figure 4 In some embodiments, the node action information includes action type sub-information and action time sub-information. Step S104 may include but is not limited to steps S401 to S402:

[0091] Step S401, based on the action time sub-information corresponding to each triple data, the corresponding triple data are sorted to obtain an action trigger sequence;

[0092] Step S402 , in response to the consistency of the action type sub-information of two adjacent triplet data in the action trigger sequence, the two adjacent triplet data are merged to obtain time dependency reduction data.

[0093] In step S401 of some embodiments, the node action information includes action type sub-information and action time sub-information. The action type sub-information indicates the operation information performed by the entity represented by the node sub-data on another entity. The operation can be a read operation or a write operation. The action time sub-information indicates the time when the entity represented by the node sub-data performs the operation.

[0094] Based on the action time sub-information corresponding to each triple data, the corresponding triple data are sorted to obtain an action trigger sequence, where the action trigger sequence is a plurality of triple data arranged in time order.

[0095] In step S402 of some embodiments, in the action trigger sequence, if the action type sub-information corresponding to two adjacent triple data is the same, that is, the entity performing the operation, the entity being operated on, and the operation performed are the same, then the two adjacent triple data are merged to obtain time dependency reduction data. It can be understood that this means that the two adjacent triple data represent the same operation performed by one entity on another entity at two different times. It should be noted that the merging operation here can be to eliminate the old triple data in the two triple data.

[0096] Through steps S401 to S402, the triple data are sorted in chronological order based on the action time sub-information to obtain an action trigger sequence, and the triple data with consistent action type sub-information are merged. This can clearly show how the operations between the entities represented by each node sub-data develop over time, streamline the audit log, reduce data redundancy, and improve the analysis efficiency of subsequent traceability analysis.

[0097] See also Figure 5 In some embodiments, after step S402, the process further includes reducing the data according to the time dependency relationship and generating a log parsing graph, which may include but is not limited to steps S501 to S503:

[0098] Step S501, generating a parsing graph node based on the sequence of each triplet data in the action trigger sequence;

[0099] Step S502, generating node-directed edges between nodes of each parsing graph based on the action type sub-information;

[0100] Step S503: Generate a log parsing graph according to each parsing graph node and the node-directed edges between the parsing graph nodes.

[0101] In step S501 of some embodiments, the log parsing graph is a traceability graph converted by using the audit log, and generates parsing graph nodes corresponding to the node sub-data based on the sequence data of each triple data in the action trigger sequence;

[0102] In some embodiments, based on the parsing of the original log data by the target cache at different times, new triple data is generated, the new triple data is sorted according to the action time sub-information, and the action trigger sequence is updated, thereby generating a new parsing graph node;

[0103] In step S502 of some embodiments, in response to the newly generated parsing graph node and the existing parsing graph node corresponding to the same node sub-data, based on the action type sub-information related to the existing parsing graph node in the action trigger sequence, the existing parsing graph node is determined to be an old class graph node, and the newly generated parsing graph node is determined to be a new class graph node.

[0104] In the log parsing graph, in order to express the relationship between entities represented by node sub-data at different times as clearly and simply as possible, a new version of the parsing graph node will be generated as a new class graph node based on the existing parsing graph node. If in the action trigger sequence, a certain node sub-data performs an operation at the first moment, and after a number of triple data in the action trigger sequence, the same node sub-data performs another operation at the second moment, in order to prevent the time sequence of these two operations from being confused when generating the log parsing graph, the action type sub-information related to the existing parsing graph node in the action trigger sequence will be used to determine whether the corresponding parsing graph node in the node sub-data generates a new version of the parsing graph node. If the existing parsing graph node includes multiple versions, the judgment is made only based on the action type sub-information related to the latest old class graph node in the old version.

[0105] In response to the newly generated parsing graph node and the existing parsing graph node corresponding to the same node sub-data, the action type sub-information related to the existing parsing graph node before this moment is queried based on the action trigger sequence. If the action type sub-information indicates that the operation is a write operation for the existing parsing graph node, the parsing graph node corresponding to the write operation will be executed to generate a new class graph node. When the action type sub-information indicates that the operation is a read operation for the existing parsing graph node, the parsing graph node corresponding to the read operation will be executed to generate a new class graph node. After determining the old class graph node and the new class graph node, determine that the action type sub-information between the old class graph node and the new class graph node is updated.

[0106] Based on the action type sub-information, a node pointing edge is generated between each parsing graph node, and based on the execution operation of the action type sub-information, a node pointing edge is generated between each parsing graph node, and the node pointing edge connects each parsing graph node and has direction information and action time sub-information. The direction is generally related to the flow direction of the data. Exemplarily, the action type sub-information between parsing graph node A and parsing graph node B is that parsing graph node A performs a write operation on parsing graph node B, then the node pointing edge should be from parsing graph node A to parsing graph node B. Similarly, if parsing graph node A performs a read operation on parsing graph node B, then the node pointing edge should be from parsing graph node B to parsing graph node A. For old class graph nodes and new class graph nodes, the node pointing edge should be from the old class graph node to the new class graph node.

[0107] In step S503 of some embodiments, a log parsing graph is generated according to each parsing graph node and the node-directed edges between the parsing graph nodes.

[0108] Through step S501 to step S503, by triggering the action sequence based on the time dependency reduction, generating a parsing graph node, and updating the parsing graph node after updating the new data, when the newly generated parsing graph node corresponds to the same node sub-data as the existing parsing graph node, generating the node pointing edge between each parsing graph node based on the action type sub-information, and then generating the node pointing edge between each parsing graph node based on the action type sub-information, and generating the log parsing graph according to each parsing graph node and the node pointing edge between each parsing graph node. Thus, the relationship between the entities represented by the parsing graph nodes can be clearly displayed, and by associating the newly generated parsing graph node with the existing parsing graph node and marking it as a "new class graph node" or "old class graph node", confusion caused by repeated operations can be avoided. This enables users to clearly distinguish which are the most recent changes and which are the past states. And by distinguishing between read operations and write operations, the direction of the data flow can be understood more accurately. This refinement is particularly important for security audits because it helps to identify potential data leakage risks or other unauthorized data access situations, making it easier for analysts to quickly locate the problem.

[0109] And when risks have been discovered, the log parsing graph makes it easier to track the root cause of the problem, because all operations are recorded and arranged in chronological order, which helps to quickly locate the time point when the error occurred and its context.

[0110] It should be noted that the log parsing graph can be constructed not only after the time dependency reduction processing and the file reference reduction processing, but also based on the processed triple data. After all, triple data is suitable for conversion into a traceability graph.

[0111] In step S105 of some embodiments, an associated extraction is performed based on the file reference reduction data and the time dependency reduction data to obtain a target reduction log. After the triple data has been processed for file reference reduction and time dependency reduction, data is extracted from the file reference reduction data and the time dependency reduction data based on the same node sub-data and associated and merged to obtain a target reduction log. It should be noted that when the file reference reduction data and the time dependency reduction data are associated and merged, the associated merging is also performed based on the time sequence of the action time sub-information. The target reduction log can be extracted for local storage, or it can be saved in a log database connected to the target cache.

[0112] In some embodiments, the target cache is divided into two areas, namely a file reference reduction area and a time dependency reduction area. The file reference reduction area performs file reference reduction processing on triple data, and the time dependency reduction area performs time dependency reduction processing on triple data.

[0113] When the space in the file reference reduction area is insufficient, the file reference reduction data related to the oldest process node sub-data that has undergone file reference reduction processing in the file reference reduction area is extracted to the time dependency reduction area and associated and merged with the time dependency reduction data to obtain the target reduction log.

[0114] When the space in the time dependency reduction area is insufficient, the oldest node sub-data in the time dependency reduction area is popped out to generate a target reduction log. If the popped-out node sub-data is a process node sub-data, the same process node sub-data is found in the file reference reduction area based on the process node sub-data. If the file reference reduction process has not been performed, the file reference reduction process is performed immediately, and then the file reference reduction result is extracted and associated and merged with the time dependency reduction data related to the popped-out process node sub-data to obtain the target reduction log.

[0115] The results of the experiments conducted based on this embodiment show that when the target cache is run as an online log parser, the overhead and parsing efficiency remain almost constant, with a mean throughput of 63,387 events / second for the log data set and an average memory overhead of 190.5MB. These values ​​are completely acceptable for online real-time reduction. The overall average CPU utilization of the reducer is 1%, which increases when the triple data is input into the time dependency reduction area for log parsing graph generation. The experiments show that the technical solution of the present application has a powerful real-time reduction capability, and the resource usage is also acceptable for common terminal hosts.

[0116] In some embodiments, a log parsing graph is generated based on the time dependency reduction result. When the target reduction log is obtained by performing association extraction based on the file reference reduction data and the time dependency reduction data, the log parsing graph is also extracted to be stored in the graph database for subsequent source tracing analysis based on the log parsing graph. In some embodiments, since the time information in the file reference reduction data is different from the time information in the time dependency reduction data, it may also trigger the generation of a new parsing graph node in steps S501 to S502 to update the log parsing graph.

[0117] In some embodiments, file reference reduction processing and time dependency reduction processing can be performed sequentially based on triple data. This method can obtain the same target log data, but since the efficiency of file reference reduction processing and time dependency reduction processing of triple data is different, executing them sequentially may increase the time used to obtain the target log data.

[0118] See also Figure 6 In some embodiments, step S102 may include but is not limited to step S601 to step S602:

[0119] Step S601: based on a plurality of triplet data, if there is corresponding node action information between the file node sub-data and at least two process node sub-data, the corresponding file node sub-data is determined as the process connection sub-data.

[0120] Step S602: Perform time dependency reduction processing based on the process connection sub-data to obtain a target reduction log.

[0121] In step S601 of some embodiments, based on multiple triple data, if the type of node sub-data is file node sub-data, and the file node sub-data has corresponding node action information with at least two process node sub-data, it means that the entity represented by the file node sub-data is associated with two process type entities. In this case, it can be considered that the entity represented by the file node sub-data is very important, and the corresponding information flow record between the two process type entities is included. Therefore, the reference information about the file node sub-data cannot be reduced. The file node sub-data that meets this situation is determined as process connection sub-data.

[0122] In step S602 of some embodiments, the reference information of the process connection sub-data is not reduced, so time dependency reduction processing can be performed to obtain the file type entity represented by the process connection sub-data, the data flow information and the corresponding time sequence between the two associated process type entities, and the time dependency reduction data, which can be determined as the target reduction log.

[0123] It should be noted that when performing file reference reduction processing, when merging triplet data based on the file dependency template, the target is the file node sub-data that has node action information with only one process node sub-data, so as to ensure that no key data flow information is lost during the file reference reduction processing.

[0124] The embodiment of the present application obtains original log data; inputs the original log data into the target cache for operation analysis to obtain multiple triplet data of the target cache; then, based on the node child data and the node action information, performs file reference reduction processing on the triplet data to obtain file reference reduction data, and performs time dependency reduction processing on the triplet data to obtain time dependency reduction data. Afterwards, association extraction is performed based on the file reference reduction data and the time dependency reduction data to obtain the target reduction log. It can be seen that the present application reduces the original log data in terms of file references and dependencies in the time dimension by inputting the original log data into the target cache for real-time analysis, and performs file reference reduction processing and time dependency reduction processing on the triplet data obtained by the real-time analysis, thereby reducing the original log data.

[0125] See also Figure 7In an exemplary example, the file reference reduction area performs file reference reduction processing, and the time dependency reduction area performs time dependency reduction processing. In the file reference reduction processing, P1 is the process node child data. Through the file dependency template example part, it can be seen that F3, F4 and F5 associated with P1 are highly similar in file storage path, and the same file action type is write operation, that is, write operation, so a file dependency template is created to merge F3, F4 and F5 into F5t, and the corresponding node action information is merged. For F6 and P1, a file dependency template can also be constructed. Based on the same file storage path and the file action type is read operation, that is, read operation, a file dependency template can also be created to merge the node action information between F6 and P1. In the time dependency reduction area, it can be seen that the triple data at different times are constructed and the action trigger sequence is constructed. At time t0 and time t1, in the triple data "2: P1 wr ite F2", "3: P1 wr ite F2", "4: P1 wr ite F2", the entity that performs the operation, the entity that is performed the operation, and the operation performed are the same, so these adjacent triple data will be merged. The result can be seen from the log parsing graph updated at time t1 that the action time sub-information attached to the node-pointing edge between P1 and F2 is the latest action time sub-information after the merge process, and there is no node-pointing edge between other new class graph nodes between P1 and F2. As for "5: P1 read F1" at time t1, in the action trigger sequence, there is "1: P1 read F1" at time t0, which is actually the same node sub-data. Therefore, it is necessary to determine the existing parsing graph node as the old class graph node and the newly generated parsing graph node as the new class graph node based on the action type sub-information related to the existing parsing graph node in the action trigger sequence. At time t0, for P1, the action type sub-information indicates that the operation of the existing parsing graph node is a read operation, so the parsing graph node corresponding to the read operation will be executed, that is, the P1 node will generate a new class graph node. From the log parsing graph updated at time t1, it can be seen that the P1 node has P1 1 Old class diagram nodes and P1 2 New class diagram node. At time t2, the file reference reduction data in the file reference reduction area is associated and merged based on the same associated time dependency reduction as the P1 node. It should be noted that when the association is merged, the merger is based on the time order, and finally the log parsing graph updated at time t2 is obtained. The log parsing graph can be extracted from the time dependency reduction area to obtain the target reduction log.

[0126] See also Figure 8The embodiment of the present application also provides an audit log reduction system, which can implement the above-mentioned audit log reduction method, and the system includes:

[0127] The original log acquisition module is used to obtain the original log data;

[0128] The original log parsing module is used to input the original log data into the target cache for operation parsing to obtain multiple triplet data of the target cache; wherein each triplet data contains two node sub-data and node action information between the node sub-data;

[0129] A file reference reduction module, used to perform file reference reduction processing on triple data based on node child data and node action information to obtain file reference reduction data;

[0130] A time dependency reduction module is used to perform time dependency reduction processing on triple data based on node sub-data and node action information to obtain time dependency reduction data;

[0131] The reduction log acquisition module is used to perform associated extraction based on file reference reduction data and time dependency reduction data to obtain a target reduction log.

[0132] The specific implementation of the audit log reduction system is basically the same as the specific implementation of the above-mentioned audit log reduction method, and will not be repeated here.

[0133] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the above audit log reduction method when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0134] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0135] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0136] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 902, and the processor 901 calls and executes the audit log reduction method of the embodiment of this application;

[0137] Input / output interface 903, used to implement information input and output;

[0138] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WI FI, Bluetooth, etc.);

[0139] A bus 905 that transmits information between the various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0140] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0141] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the above-mentioned audit log reduction method is implemented.

[0142] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0143] The audit log reduction method, system, electronic device and storage medium provided in the embodiments of the present application obtain original log data; input the original log data into the target cache for operation analysis to obtain multiple triple data of the target cache; then, based on the node child data and the node action information, perform file reference reduction processing on the triple data to obtain file reference reduction data, and perform time dependency reduction processing on the triple data to obtain time dependency reduction data. Then, association extraction is performed based on the file reference reduction data and the time dependency reduction data to obtain the target reduction log. It can be seen that the present application reduces the original log data in terms of file reference and dependency in the time dimension by inputting the original log data into the target cache for real-time analysis, and performs file reference reduction processing and time dependency reduction processing on the triple data obtained by the real-time analysis, thereby reducing the original log data.

[0144] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0145] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0146] The system embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0147] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0148] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0149] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0150] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0151] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0152] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0153] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0154] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A method for reducing audit logs, characterized in that: The method comprises: Get the original log data; Input the original log data into the target cache for operation analysis to obtain a plurality of triplet data of the target cache; wherein each triplet data includes two node sub-data and node action information between the node sub-data; Based on the node sub-data and the node action information, performing file reference reduction processing on the triple data to obtain file reference reduction data; Based on the node sub-data and the node action information, performing time dependency reduction processing on the triple data to obtain time dependency reduced data; Based on the file reference reduction data and the time dependency reduction data, associated extraction is performed to obtain a target reduction log.

2. The method according to claim 1, characterized in that , the node sub-data is process node sub-data or file node sub-data, each of the triple data includes a corresponding process node sub-data, a corresponding node action information and a corresponding file node sub-data, and based on the node sub-data and the node action information, the triple data is subjected to file reference reduction processing to obtain file reference reduction data, including: Determine the same process node sub-data possessed by each of the triple data as composite node data; For each of the triplet data corresponding to the composite node data, based on the corresponding node action information, construct a file dependency template that matches the corresponding process node sub-data; Based on the file dependency template, the node action information is merged into the triple data to obtain the file reference reduction data.

3. The method according to claim 2, characterized in that The node action information includes action path sub-information and action type sub-information. For each triplet data corresponding to the composite node data, based on the corresponding node action information, a file dependency template matching the corresponding process node sub-data is constructed, including: For each of the triplet data corresponding to the composite node data, based on the action path sub-information, determine the file storage path corresponding to the file node sub-data, and based on the action type sub-information, determine the file action type corresponding to the file node sub-data; The file storage path and the file action type of each triple data are integrated to obtain a file dependency template corresponding to the composite node data.

4. The method according to claim 1, characterized in that: The node action information includes action type sub-information and action time sub-information, and the time dependency reduction processing is performed on the triple data based on the node sub-data and the node action information to obtain time dependency reduction data, including: Based on the action time sub-information corresponding to each triplet of data, sorting the corresponding triplet of data to obtain an action trigger sequence; In response to the consistency of the action type sub-information of two adjacent triplet data in the action trigger sequence, the two adjacent triplet data are merged to obtain the time dependency reduction data.

5. The method according to claim 4, characterized in that The method further comprises, after performing time dependency reduction processing on the triple data based on the node sub-data and the node action information to obtain time dependency reduced data, generating a log parsing graph according to the time dependency reduced data, specifically comprising: Generate a parsing graph node based on the sequence of each of the triplet data in the action trigger sequence; Based on the action type sub-information, generating node-directed edges between the nodes of the parsing graph; A log parsing graph is generated according to each of the parsing graph nodes and the node-directed edges between the parsing graph nodes.

6. The method according to claim 5, characterized in that The parsing graph node includes an old class graph node and a new class graph node, and the generating of the parsing graph node based on the sequence of the triple data in the action trigger sequence includes: Based on the sequence of each triplet data in the action trigger sequence, the parsing graph node is generated, and in response to the newly generated parsing graph node and the existing parsing graph node corresponding to the same node sub-data, based on the action type sub-information related to the existing parsing graph node in the action trigger sequence, the existing parsing graph node is determined as the old class graph node, and the newly generated parsing graph node is determined as the new class graph node.

7. The method according to claim 1, characterized in that The node sub-data is process node sub-data or file node sub-data, and after inputting the original log data into the target cache for operation parsing and obtaining a plurality of triplet data of the target cache, the method further includes: Based on the plurality of triplet data, if the file node sub-data and at least two of the process node sub-data have corresponding node action information, determining the corresponding file node sub-data as process connection sub-data; A time dependency reduction process is performed based on the process connection sub-data to obtain the target reduction log.

8. An audit log reduction system, characterized in that: The system comprises: The original log acquisition module is used to obtain the original log data; The original log parsing module is used to input the original log data into the target cache for operation parsing to obtain a plurality of triplet data of the target cache; wherein each of the triplet data contains two node sub-data and node action information between the node sub-data; A file reference reduction module, configured to perform file reference reduction processing on the triple data based on the node sub-data and the node action information to obtain file reference reduction data; A time dependency reduction module, configured to perform time dependency reduction processing on the triple data based on the node sub-data and the node action information to obtain time dependency reduced data; The reduction log acquisition module is used to perform associated extraction based on the file reference reduction data and the time dependency reduction data to obtain a target reduction log.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the audit log reduction method described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the audit log reduction method according to any one of claims 1 to 7 is implemented.

Images