Log data processing method, device, equipment, storage medium and program product

Abstract

The present application discloses a method, device, equipment, storage medium and program product for processing log data, the method comprising: obtaining original log data, the original log data including system operation data of multiple log events; if there are multiple candidate log events whose system operation data meet the parameter requirements of a composite log event among the multiple log events, then generating a composite log event based on the multiple candidate log events; the parameter requirements are used to characterize the instructions and operands related to the composite log event; performing log audit processing on the composite log event to obtain the audit result of the composite log event. The method for processing log data provided by the present application can generate a composite log event based on multiple log events whose system operation data meet the parameter requirements of the composite log event, so as to fully reflect complex system behaviors and security events through the audit results of the composite log event, thereby improving the accuracy of detecting system security events.

Description

Technical Field

[0001] The present application relates to the fields of computer security and log analysis, and specifically to a method, apparatus, device, storage medium and program product for processing log data. Background Art

[0002] In modern computer systems, log data can usually be used as an important data source to record computer system activity data and user behavior data, and based on the analysis results of log data, the security risks of computer systems can be reduced and the management efficiency of computer systems can be improved.

[0003] However, the analysis methods for log data in the prior art generally process single events recorded in the log data, which results in the analysis results of the log data being unable to fully reflect complex system behaviors and security events, resulting in the existing log data analysis methods still having poor detection accuracy for security events. Summary of the invention

[0004] In view of the above-mentioned defects or deficiencies in the prior art, it is desired to provide a method, apparatus, device, storage medium and program product for processing log data. The method for processing log data can generate a composite log event based on multiple log events whose system operation data meet the parameter requirements of the composite log event, so as to comprehensively reflect complex system behaviors and security events through the audit results of the composite log event, thereby improving the accuracy of detecting system security events.

[0005] In a first aspect, the present invention provides a method for processing log data, the method comprising:

[0006] Obtaining original log data, where the original log data includes system operation data of multiple log events;

[0007] If there are multiple candidate log events whose system operation data meet the parameter requirements of the composite log event among the multiple log events, a composite log event is generated based on the multiple candidate log events; the parameter requirements are used to characterize the instructions and operands related to the composite log event;

[0008] Perform log audit processing on the compound log event to obtain the audit result of the compound log event.

[0009] In a second aspect, the present invention provides a log data processing device, the device comprising:

[0010] An acquisition module is used to acquire original log data, where the original log data includes system operation data of multiple log events;

[0011] A first processing module is used to generate a composite log event based on the multiple candidate log events if there are multiple candidate log events whose system operation data meets the parameter requirements of the composite log event among the multiple log events; the parameter requirements are used to characterize instructions and operands related to the composite log event;

[0012] The second processing module is used to perform log audit processing on the compound log event to obtain the audit result of the compound log event.

[0013] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the first aspect is implemented.

[0014] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the program is executed by a processor, the method described in the first aspect is implemented.

[0015] In a fifth aspect, a computer program product is provided. The computer program product includes instructions. When the instructions are executed, any method described in the first aspect is executed.

[0016] Compared with the log processing method of processing a single event in log data in the prior art, the log data processing method, apparatus, device, storage medium and program product provided in the embodiments of the present application can determine the log events whose system operation data meets the parameter requirements of the composite log event among the multiple events contained in the original log data as candidate log events, so as to generate a composite log event based on the candidate log events, so that there is a correlation between each log event in the composite log event corresponding to the parameter requirements of the composite log event, so that the audit results based on multiple log events with correlation can comprehensively reflect complex system behaviors and security events, thereby improving the accuracy of system security event detection and avoiding the problem of false positives or negative positives of security events. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0018] Figure 1 A schematic diagram of the implementation environment architecture provided for the embodiments of the present application;

[0019] Figure 2 A flowchart of a log data processing method provided in an embodiment of the present application;

[0020] Figure 3 A schematic diagram of the composition of an installation data package provided in an embodiment of the present application;

[0021] Figure 4 A schematic diagram of a process for generating a composite log event provided in an embodiment of the present application;

[0022] Figure 5 A block diagram of a log data processing device provided in an embodiment of the present application;

[0023] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of the present application;

[0024] In the above picture:

[0025] 301-log acquisition module; 302-policy configuration module; 303-log analysis module; 304-event generation module; 305-event extension module; 306-event sending module; 500-log data processing device; 501-acquisition module; 502-first processing module; 503-second processing module; 600-computer equipment; 601-central processing unit (CPU); 602-read-only memory (ROM); 603-random access memory (RAM); 604-bus; 605-input / output (I / O) interface; 606-input part; 607-output part; 608-storage part; 609-communication part; 610-drive; 611-removable medium. DETAILED DESCRIPTION

[0026] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0027] It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments. In addition, the term "and / or" herein is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order of objects.

[0028] First, the terms involved in this application are explained as follows:

[0029] (1) Log data: used to record process event data generated by IT systems such as computer systems, applications or network devices. It can be divided into text log data, system log data, simple network management protocol log data and database log data;

[0030] (2) Log pattern matching: It is a key step in analyzing and understanding log data. Specifically, by matching specific patterns, valuable information can be extracted from a large amount of log information, such as error information, exception information, warning information, performance indicators, etc.

[0031] Figure 1 Schematic diagram of the implementation environment architecture of a log data processing method provided in an embodiment of the present application. Figure 1 As shown, the implementation environment architecture includes: a computer device 10.

[0032] In a specific implementation, the processing flow of log data in the present application can be deployed in the computer device 10 in the format of a file, and the log data generated by the computer device 10 can be processed based on the processing flow, so as to detect security events of the computer device 10 in real time based on the processing results of the log data.

[0033] Exemplarily, the computer device 10 may be a terminal device in various application scenarios, such as a smart TV, a smart phone, a tablet computer, a TV, a laptop computer, a desktop computer, etc.

[0034] At present, in modern computer systems, log data can usually be used as an important data source to record computer system activity data and user behavior data, and based on the analysis results of log data, the security risks of computer systems can be reduced and the management efficiency of computer systems can be improved.

[0035] However, the analysis methods for log data in the prior art generally process a single event recorded in the log data, and the log data processing is generally a simple log pattern matching. This makes it difficult for the existing analysis methods to effectively identify and associate multiple related events in the log data, and it is also difficult to effectively identify the complex behavior patterns in the system composed of multiple related events.

[0036] For example, although the Audit log audit system that comes with the Linux system can report information about file modification, execution and other events, it cannot accurately determine a complex event from a single file modification, execution and other events.

[0037] Therefore, the analysis results of existing log data cannot fully reflect complex system behaviors and security events, resulting in poor detection accuracy of system security events.

[0038] Based on this, an embodiment of the present application provides a method for processing log data, the technical principle of which is: determining the log events whose system operation data satisfy the parameter requirements of the composite log event among the multiple events contained in the original log data as candidate log events, and generating a composite log event based on the candidate log events, so that there is a correlation between each log event in the composite log event corresponding to the parameter requirements of the composite log event.

[0039] It should be noted that although the operations of the method of the present invention are described in a particular order in the drawings, this does not require or imply that the operations must be performed in this particular order or that all illustrated operations must be performed to achieve desired results.

[0040] The present application embodiment provides a method for processing log data. Figure 2 is a flow chart of a method for processing log data provided by an embodiment of the present application, such as Figure 2 As shown, the method specifically comprises the following steps:

[0041] Step 201 : obtaining original log data, where the original log data includes system operation data of multiple log events.

[0042] In a possible implementation, a connection may be established with a log server in the computer system so as to utilize the log server to obtain system operation data of a plurality of log events generated in the computer system; wherein the log may also be referred to as an audit log.

[0043] Exemplarily, the system operation data of a log event may include the timestamp of the log event (i.e., the specific time when the event occurred), the execution object (i.e., the user or entity that performed the operation), the execution content (i.e., a description of the operation performed), the execution result (e.g., success or failure), the source address (the device or system that performed the operation), etc.

[0044] Exemplarily, a raw socket connection may be established with a log audit server to obtain raw log data by directly accessing network hardware.

[0045] It should be noted that the above-mentioned log audit server is not a server in the traditional sense, but a local service module in the computer system that can provide the function of sending log data, which can be specifically expressed in the form of a program.

[0046] Exemplarily, a filter condition for the log event may be preset to obtain the original log data based on the filter condition for the log event.

[0047] For example, a filter instruction of a log event may be preset to determine the system operation data of the log event corresponding to the filter instruction as the original log data.

[0048] It should be noted that the above-mentioned instruction of the log event can be used to characterize the event type of the log event; for example, the instruction of the log event can be a modification instruction, a deletion instruction, etc.

[0049] Exemplarily, the memory space for storing log data can be preset to ensure that log data with high network traffic can be processed; wherein the memory space for storing log data can be called the buffer size, and within a certain range, the log data processing capability is positively correlated with the buffer size.

[0050] Exemplarily, the log data sent by the log server may be received in real time, and the log data may be pre-processed to remove redundant data in the original log data.

[0051] In a possible implementation, the log data processing method provided in the embodiment of the present application can be deployed in the above-mentioned computer device 10 in the form of an installation data package; for example, the installation data package can be a configuration file composed of programming statements.

[0052] For example, Figure 3 is a schematic diagram of the composition of the installation data package provided in the embodiment of the present application, such as Figure 3 As shown, the installation data packet may include a log acquisition module 301 .

[0053] Specifically, based on the log acquisition module 301, the above-mentioned log audit server can be connected first, and then the original log data can be obtained based on the preset screening conditions of the log events and the memory space for storing the log data, and finally the obtained log data can be preprocessed.

[0054] Step 202: if there are multiple candidate log events whose system operation data meet the parameter requirements of the composite log event among the multiple log events, a composite log event is generated based on the multiple candidate log events; the parameter requirements are used to characterize instructions and operands related to the composite log event.

[0055] In a possible implementation, multiple candidate log events (ie, multiple sub-events) may be determined from the multiple log events obtained in step 201 based on parameter requirements of the composite log event.

[0056] Exemplarily, the parameter requirements of a compound log event may be preset definition rules for the compound log event, which may be used to characterize instructions and operands related to the compound log event, and filter log events that meet the triggering conditions of the compound log event based on the instructions and operands; wherein the parameter requirements of different compound log events may be represented by different policy configuration files.

[0057] Exemplarily, the instructions and operands of a compound log event can be represented by a key-value pair. For example, when a compound log event is defined as "modify user A's information", the instructions related to the compound log event can be the key value "modify", and the operands related to the compound log event can include the value value "user A" and the value value "user A's information".

[0058] Specifically, Figure 3 As shown, the installation data package may also include a policy configuration module 302. Based on the policy configuration module 302, a unique event identifier may be configured for each compound log event, and then different parameter requirements for different compound log events may be preset; wherein, the parameter requirements for the compound log event may be represented as "event_define" in the policy configuration file, specifically including the event name, event identifier, event correlation parameters, and event priority value of each compound log event.

[0059] For example, the event name of a compound log event can be represented as "event_name" in the policy configuration file; the event correlation parameter of a compound log event can be used to define the time interval between the individual log events that constitute the compound log event, which can be specifically represented as "event_interval" in the policy configuration file; the event priority value of a compound log event can be the weight value of the event, which can be specifically represented as "event_weights" in the policy configuration file; in addition, the parameter requirements of a compound log event can also include the triggering conditions of the compound log event, which can be specifically represented as "event_trigger" in the policy configuration file.

[0060] Exemplarily, the candidate log events may be determined based on the instructions corresponding to each log event in the original log data and the parameter requirements of the compound log event.

[0061] Specifically, for each log event in the original log data, the instructions of the log event can be determined based on the system operation data of the log event, so that when the instructions of the log event are the same as the relevant instructions represented by the parameter requirements of the compound log event, the log event is determined to be a candidate log event.

[0062] For example, when the event type of the log event (ie, the type of the log) is the same as the instruction represented by the compound log event (ie, the key value of the compound log event trigger condition), the log event can be determined to be a candidate log event.

[0063] Correspondingly, if Figure 3As shown, the installation data package may further include a log analysis module 303. Based on the log analysis module 303, a large amount of log data may be processed in real time while candidate log events that meet the requirements of the composite log event parameters may be quickly determined.

[0064] In addition, the log analysis module 303 can also set an expiration time for the log data of each log event and limit the maximum number of storage entries to prevent the log data from occupying too much memory, and store the data content of the log event in the form of key-value pairs.

[0065] Exemplarily, a compound log event may be generated based on the priority values ​​of the individual log events and the compound log event in the original log data.

[0066] Correspondingly, if Figure 3 As shown, the installation data package may further include an event generation module 304. Based on the event generation module 304, when multiple candidate log events satisfying the composite log event parameter requirements are determined, a corresponding composite log event may be automatically generated.

[0067] In an embodiment of the present application, a policy configuration file corresponding to a compound log event is generated by predefining a compound log event, so that multiple log events generated by a computer system are screened based on the policy configuration file and a compound log event is generated, thereby improving the flexibility and applicability of the log data analysis and processing method, and effectively improving the accuracy and real-time performance of system security event detection.

[0068] Step 203: perform log audit processing on the compound log event to obtain an audit result of the compound log event.

[0069] In a possible implementation, before performing log audit processing on the compound log event, matching processing may be performed on the actually generated compound log event based on parameter requirements of the compound log event.

[0070] Exemplarily, the parameter requirements of the compound log event may also include an extended instruction of the compound log event, which may be specifically represented as “event_extra” in the policy configuration file.

[0071] Specifically, when the operand of the extended instruction "event_extra" in the policy configuration file is "yes", the data of the composite log event can be modified accordingly based on the user's customized requirements.

[0072] For example, Figure 3 As shown, the installation data package may further include an event extension module 305, based on which a custom function for a compound log event may be implemented.

[0073] Specifically, a standard interface for implementing a composite log event customization function may be configured in the event extension module 305 so as to configure customized content through the standard interface.

[0074] In an embodiment of the present application, user-defined specific functions can be implemented by matching and processing composite log events, so that the analysis and processing requirements for log data can be flexibly expanded according to different environments and needs, thereby improving the flexibility of the log data processing method while enhancing the applicability of the method.

[0075] For example, Figure 3 As shown, the installation data packet may further include an event sending module 306 , based on which a compound log event may be sent to obtain an audit result of the compound log event.

[0076] Compared with the log processing method of processing a single event in log data in the prior art, the log data processing method provided in the embodiment of the present application can determine the log events whose system operation data meet the parameter requirements of the composite log event among the multiple events contained in the original log data as candidate log events, so as to generate a composite log event based on the candidate log events, so that there is a correlation between each log event in the composite log event corresponding to the parameter requirements of the composite log event, so that the audit results based on multiple log events with associated relationships can comprehensively reflect complex system behaviors and security events, improve the accuracy of system security event detection, and avoid the problem of false positives or negative positives of security events.

[0077] In another embodiment of the present application, a specific implementation method for generating a composite log event is also provided. Exemplarily, the aforementioned "generating a composite log event based on multiple candidate log events" specifically includes: determining the priority values ​​corresponding to multiple candidate log events based on the parameter requirements of the composite log event; weighting the priority values ​​of the candidate log events in sequence according to the chronological order of the generation time of the multiple candidate log events, and if the weighted result of the priority values ​​of the first N candidate log events among the multiple candidate log events exceeds the priority value of the composite log event, compounding the first N candidate log events to obtain the composite log event.

[0078] In a possible implementation, the priority value of each candidate log event may be determined based on the parameter requirements of the composite log event.

[0079] Exemplarily, for each candidate log event among multiple candidate log events, it is possible to first determine, based on the parameter requirements of the composite log event, whether the system operation data of the candidate log event satisfies the instructions represented by the parameter requirements of the composite log event, and then determine the priority value corresponding to the instruction as the priority value corresponding to the candidate log event.

[0080] For example, the parameter requirements of a compound log event can be expressed as follows:

[0081] {

[0082] "event_define": [

[0083] {

[0084] "event_name": "ssh_login",

[0085] "event_weights": "13",

[0086] "event_interval": "1",

[0087] "event_extra": "yes",

[0088] "event_trigger": {

[0089] "USER_LOGIN": {

[0090] "op": "login",

[0091] "exe": " / usr / sbin / sshd",

[0092] "res": "success",

[0093] "weight": "10",

[0094] "return": "uid, auid, hostname, addr"

[0095] }

[0096] It can be learned that the event name of the compound log event is specifically "ssh protocol login operation (ie, ssh_login)", the event priority value is specifically 13, the time interval between each sub-event is specifically 1, and the extended instruction of the event is specifically in the open state; secondly, the parameter requirement that one of the instructions used for characterization (ie, the trigger condition of the event) can include "USER_LOGIN".

[0097] Correspondingly, when the system operation data of the candidate log event satisfies the above instruction “USER_LOGIN” (ie, is the same as the above “USER_LOGIN” instruction), the priority value 10 corresponding to the instruction “USER_LOGIN” may be determined as the priority value of the candidate log event.

[0098] In a possible implementation, the priority values ​​of the candidate log events may be weighted in sequence according to the order of generation time of the multiple candidate log events.

[0099] Exemplarily, when the priority values ​​of the candidate log events are determined based on the parameter requirements of the composite log event, the priority values ​​of the candidate log events may be summed up in sequence.

[0100] Exemplarily, based on the sum of the priority values ​​of the candidate log events, when the sum of the priority values ​​of the first N candidate log events among multiple candidate log events is equal to or exceeds the priority value of the compound log event, the first N candidate log events are compounded to obtain the compound log event.

[0101] For example, when the priority value of the compound log event is 11 and the candidate log events include event 1 with a priority value of 10, event 2 with a priority value of 2, and event 3 with a priority value of 3, the sum of the priority values ​​of the first two candidate log events can be obtained as 12 and the sum of the priority values ​​of the first three candidate log events can be obtained as 15 in the order of generation time of candidate log events 1, 2, and 3. At this time, the first two candidate log events can be compounded, or the first three candidate log events can be compounded.

[0102] In a possible implementation, before compounding the first N candidate log events to obtain the compound log event, the correlation of adjacent events in the candidate log events may be determined to compound the corresponding candidate log events based on the correlation results of the adjacent events.

[0103] In the embodiment of the present application, the correlation of the candidate log events is judged so that each log event used to generate the composite log event has an associated relationship, thereby ensuring the accuracy of generating the composite log event.

[0104] Exemplarily, the correlation between adjacent log events in the candidate log events may be determined according to the interval between the generation times of adjacent log events in the candidate log events.

[0105] Specifically, when the interval between adjacent event generation times does not satisfy a preset time interval, it is determined that the correlation between adjacent events does not satisfy an association condition, otherwise it is determined that the correlation between adjacent events satisfies an association condition; wherein, the interval between adjacent event generation times may be less than or equal to a preset time interval.

[0106] It should be noted that the above-mentioned preset time interval can be defined as "event_interval" in the policy configuration file, so that the log analysis module 303 in the installation data package can determine the correlation between adjacent log events based on the preset time interval.

[0107] Exemplarily, when there is a correlation between adjacent events that does not satisfy the association condition, the previous candidate log event in the adjacent events is ignored.

[0108] For example, when the candidate log events include events 1, 2, and 3 and it can be determined based on a preset time interval that the correlation between events 1 and 2 does not satisfy the association condition but the correlation between events 2 and 3 satisfies the association condition, candidate log event 1 can be ignored.

[0109] Exemplarily, when the weighted result of the priority values ​​of other candidate log events except the ignored candidate log event among the current N candidate log events meets the priority value of the composite log event, a composite log event can be generated according to the other candidate log events.

[0110] For example, when the priority value of a compound log event is 11 and the candidate log events include event 1 with a priority value of 10, event 2 with a priority value of 2, and event 3 with a priority value of 9, based on the correlation between events 1 and 2 not satisfying the association condition, ignoring the sum of the priority values ​​of events 2 and 3 after event 1, the result is 11. At this time, candidate log events 2 and 3 can be compounded to generate a compound log event.

[0111] Optionally, when the weighted result of the priority values ​​of other candidate log events except the ignored candidate log event among the current N candidate log events does not meet the priority value of the compound log event, the weighted result of the priority values ​​of the first N candidate log events and the M candidate log events after the first N candidate log events can be determined until the weighted result meets the priority value of the compound log event.

[0112] For example, when the priority value of a compound log event is 11 and the candidate log events include event 1 with a priority value of 10, event 2 with a priority value of 2, and event 3 with a priority value of 3, based on the correlation between events 1 and 2 not satisfying the association condition, ignoring the sum of the priority values ​​of events 2 and 3 after event 1, the result is 5. At this time, it can be determined that the sum result does not satisfy the priority value of the compound log event.

[0113] Based on this, it can be determined that the priority value of event 4 after event 3 is 7, that is, the sum of the priority values ​​of events 2, 3, and 4 is 12. At this time, candidate log events 2, 3, and 4 can be combined to generate a composite log event.

[0114] It should be noted that, when the weighted result of the priority values ​​of the candidate log events is greater than or equal to the priority value of the compound log event, and the time interval between adjacent log events in the candidate log events is less than the preset time interval, the corresponding multiple candidate log events can be determined as a complete compound log event; secondly, when determining the weighted result of the priority values ​​of the M candidate log events after the first N candidate log events, it is also necessary to synchronously determine the correlation between adjacent log events to ensure the correlation between the candidate log events that constitute the compound log event.

[0115] In another embodiment of the present application, a specific method for generating a composite log event is also provided. Exemplarily, the aforementioned "generating a composite log event based on multiple candidate log events" specifically includes: obtaining metadata information of each candidate log event from system operation data of multiple candidate log events, sorting the metadata information based on a preset order, and generating a composite log event based on the sorted metadata information.

[0116] In a possible implementation, metadata information of each candidate log event may be sorted based on a preset order to generate a composite log event; wherein the metadata information may include basic information such as a timestamp, a user ID, and a resource identifier of the log event.

[0117] Exemplarily, in the policy configuration module 302 of the above-mentioned installation package, feedback data corresponding to each instruction (ie, trigger condition) may also be configured in the parameter requirements of the compound log event; wherein the feedback data corresponding to each instruction may be a fixed operand corresponding to each instruction.

[0118] Exemplarily, after the compound log event is generated, the event sending module 306 may be used to send the compound log event to a designated storage system of the computer device 10 .

[0119] In the embodiment of the present application, by sorting the metadata information of the candidate log data, the data content of the composite log event can clearly indicate the security event existing in the system.

[0120] In another embodiment of the present application, another method for generating a composite log event is provided. For example, Figure 4 is a flow chart of a method for generating a composite log event provided in an embodiment of the present application, such as Figure 4 As shown, the method specifically comprises the following steps:

[0121] Step 401, obtaining system operation data of multiple log events.

[0122] Exemplarily, the log acquisition module 301 may be used to acquire system operation data of multiple log events generated by the computer system.

[0123] Step 402 , for each log event, determine whether the system operation data of the log event meets the parameter requirements of the composite log event; if yes, execute step 403 , otherwise continue to execute step 401 .

[0124] Exemplarily, based on the specific configuration of the parameter requirements of the compound log event by the policy configuration module 302 , the system operation data of each log data can be judged by using the log analysis module 303 .

[0125] Step 403: determine the log events that meet the parameter requirements of the composite log event as candidate log events, and store the candidate log events in the cache space.

[0126] Step 404 , determining whether the correlation of adjacent events in the candidate log event satisfies the correlation condition; if so, executing step 405 , otherwise continuing to execute step 401 .

[0127] Exemplarily, the log analysis module 303 may be used to determine the interval between the generation times of adjacent log events in the candidate log events, so as to determine the correlation between adjacent log events in the candidate log events.

[0128] Step 405 , generating a composite log event based on the candidate log event, and determining whether the composite log event has an extended instruction based on the parameter requirements of the composite log event; if so, executing step 406 , otherwise continuing to execute step 407 .

[0129] Exemplarily, the event generation module 304 may be used to generate a composite log event from a candidate log event.

[0130] Step 406: Customize and extend the actually generated composite log event based on the extension instruction.

[0131] Exemplarily, the event extension module 305 may be used to perform custom extension on the actually generated composite log event based on the extension instruction.

[0132] Step 407: Send the compound log event to a designated storage system.

[0133] Exemplarily, the event sending module 306 may be used to send the compound log event to a designated storage system.

[0134] Figure 5 A block diagram of a log data processing device provided in an embodiment of the present application. Figure 5 As shown, the log data processing device 500 includes: an acquisition module 501, a first processing module 502 and a second processing module 503, wherein:

[0135] An acquisition module 501 is used to acquire original log data, where the original log data includes system operation data of multiple log events;

[0136] The first processing module 502 is used to generate a composite log event based on the multiple candidate log events if there are multiple candidate log events whose system operation data meets the parameter requirements of the composite log event among the multiple log events; the parameter requirements are used to characterize the instructions and operands related to the composite log event;

[0137] The second processing module 503 is used to perform log audit processing on the compound log event to obtain the audit result of the compound log event.

[0138] Reference below Figure 6 , Figure 6 A schematic diagram of the structure of a computer device suitable for implementing the embodiments of the present application is shown. Figure 6 As shown, the computer device 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage part 608 to the random access memory (RAM) 603. In the random access memory (RAM) 603, various programs and data required for the operation instructions of the system are also stored. The central processing unit (CPU) 601, the read-only memory (ROM) 602 and the random access memory (RAM) 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0139] The following components are connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, a modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed so that a computer program read therefrom is installed into the storage section 608 as needed.

[0140] In particular, according to an embodiment of the present application, the above reference flow chart Figure 2Any of the described processes may be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product including a computer program carried on a computer-readable medium, the computer program including a program code for executing the method shown in the flowchart. In such an embodiment, the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 609, and / or installed from a removable medium 611. When the computer program is executed by a central processing unit (CPU) 601, the above-mentioned functions defined in the system of the present application are executed.

[0141] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium that matches a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0142] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operating instructions of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the aforementioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operating instruction, or can be implemented with a combination of dedicated hardware and computer instructions.

[0143] The units or modules involved in the embodiments described in the present application may be implemented by software or by hardware. The described units or modules may also be provided in a processor, for example, may be described as: a processor including a semantic extraction unit, a weight allocation unit, and a determination unit. The names of these units or modules do not, in some cases, constitute limitations on the units or modules themselves.

[0144] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the computer device described in the above embodiment, or may exist independently without being installed in the computer device. The above computer-readable storage medium stores one or more programs, and when the above programs are used by one or more processors to execute the method described in the present application. For example, it can be executed Figure 2 The steps of any of the methods shown.

[0145] The present application embodiment provides a computer program product, which includes instructions. When the instructions are executed, the method described in the present application embodiment is executed. For example, Figure 2 The steps of any of the methods shown.

[0146] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the aforementioned disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present application (but not limited to) by each other to form a technical solution.

Claims

1. A method for processing log data, characterized in that: The method comprises: Based on a preset event type of a log event, determining system operation data of a log event corresponding to the preset event type as original log data, wherein the original log data includes system operation data of a plurality of log events; wherein the preset event type includes at least one of a modification event and a deletion event; If there are multiple candidate log events whose system operation data meet the parameter requirements of the composite log event among the multiple log events, the composite log event is generated based on the multiple candidate log events; the parameter requirements are used to characterize the instructions and operands related to the composite log event; wherein each of the composite log events is configured with a unique event identifier, and the parameter requirements of each of the composite log events include at least one of an event name, an event identifier, an event correlation parameter, and an event priority value; Performing log audit processing on the compound log event to obtain an audit result of the compound log event; The generating the composite log event based on the multiple candidate log events includes: Based on the parameter requirements of the composite log event, determining priority values ​​corresponding to the multiple candidate log events; According to the chronological order of generation time of the multiple candidate log events, the priority values ​​of the candidate log events are summed up in sequence. If the sum of the priority values ​​of the first N candidate log events among the multiple candidate log events exceeds the priority value of the compound log event, the first N candidate log events are compounded to obtain the compound log event.

2. The method according to claim 1, characterized in that The step of compounding the first N candidate log events to obtain the compound log event includes: Determine the correlation of adjacent events in the first N candidate log events, and if there is an adjacent event whose correlation does not satisfy the association condition, ignore the previous candidate log event in the adjacent event; If the sum of the priority values ​​of the other candidate log events except the ignored candidate log event among the first N candidate log events meets the priority value of the composite log event, then generate the composite log event according to the other candidate log events; Otherwise, determine the sum of the priority values ​​of the first N candidate log events and the M candidate log events after the first N candidate log events, until the sum meets the priority value of the composite log event.

3. The method according to claim 2, characterized in that Determining the correlation of adjacent events in the first N candidate log events includes: Determining the interval between the generation times of the adjacent events; If the interval between the generation times of the adjacent events does not satisfy the preset time interval, it is determined that the correlation of the adjacent events does not satisfy the association condition; otherwise, it is determined that the correlation of the adjacent events satisfies the association condition.

4. The method according to claim 1, characterized in that: Determining priority values ​​corresponding to the plurality of candidate log events based on the parameter requirements of the composite log event includes: For each candidate log event of the plurality of candidate log events, determining, based on the parameter requirement of the composite log event, that the system operation data of the candidate log event satisfies the instruction represented by the parameter requirement of the composite log event; The priority value corresponding to the instruction is determined as the priority value corresponding to the candidate log event.

5. The method according to claim 1, characterized in that The parameter requirements include the name, identifier, event correlation parameters, and priority value of the compound log event.

6. The method according to claim 1, characterized in that The method further comprises: For each of the log events, determining an instruction of the log event based on the system operation data of the log event; If the instruction of the log event is the same as the relevant instruction represented by the parameter requirement of the compound log event, if they are the same, the log event is determined as the candidate log event.

7. The method according to claim 1, characterized in that Generating the composite log event based on the multiple candidate log events specifically includes: The metadata information of each of the candidate log events is acquired from the system operation data of the multiple candidate log events, the metadata information is sorted based on a preset order, and the composite log event is generated based on the sorted metadata information.

8. The method according to claim 1, characterized in that: The method further comprises: Based on the parameter requirements of the compound log event, matching processing is performed on the actually generated compound log event.

9. A log data processing device, characterized in that: The device comprises: An acquisition module, configured to determine, based on a preset event type of a log event, system operation data of a log event corresponding to the preset event type as original log data, wherein the original log data includes system operation data of a plurality of log events; wherein the preset event type includes at least one of a modification event and a deletion event; A first processing module is used to generate the composite log event based on the multiple candidate log events if there are multiple candidate log events whose system operation data meets the parameter requirements of the composite log event among the multiple log events; the parameter requirements are used to characterize the instructions and operands related to the composite log event; wherein each of the composite log events is configured with a unique event identifier, and the parameter requirements of each of the composite log events include at least one of an event name, an event identifier, an event correlation parameter, and an event priority value; A second processing module is used to perform log audit processing on the compound log event to obtain an audit result of the compound log event; The first processing module is specifically configured to determine priority values ​​corresponding to the plurality of candidate log events based on parameter requirements of the composite log event; According to the chronological order of generation time of the multiple candidate log events, the priority values ​​of the candidate log events are summed up in sequence. If the sum of the priority values ​​of the first N candidate log events among the multiple candidate log events exceeds the priority value of the compound log event, the first N candidate log events are compounded to obtain the compound log event.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, characterized in that The computer program product comprises instructions, which, when executed, cause the method according to any one of claims 1 to 8 to be performed.

Images