Log event missing value filling method and device

Abstract

The invention discloses a missing value filling method and device for a log event, and the method comprises the steps: carrying out the event analysis of log data, obtaining a user name, a source IP and a preset related identifier of the event in the event analysis process, and enabling the preset related identifier to comprise a session identifier and / or a process identifier, and enabling a main value to comprise the user name and the source IP; and when it is determined that the attribute information of the first event lacks one or two of the subject values, the preset related identifier of the first event is not lacked, the preset related identifier of the second event is matched with the preset related identifier of the first event, and the attribute information of the second event does not lacks the subject value, determining that the first event is the first event. And filling the missing main body value of the first event based on the main body value of the second event. Thereby, based on context association, a basis fill value is provided for an event missing a user name or a source IP, subject information of the event is accurately restored instead of discarding a security event or using null fill, and relevance of the event and accuracy of analysis are improved.

Description

Technical Field

[0001] This application relates to the field of network security technology, and in particular to a method and apparatus for filling missing values ​​in log events. Background Technology

[0002] With the accelerated advancement of digital transformation, the Windows operating system has been widely deployed in enterprise environments, becoming a critical foundational platform supporting business operations. The Windows system uses an event log mechanism to record in detail the system's operational status, user actions, and various security-related activities.

[0003] In the field of Windows event log analysis, the preprocessing stage often encounters the problem of missing key attributes (such as username or source IP) in event records. Traditional processing methods usually choose to discard these events or fill them with null values, which leads to reduced correlation between events and insufficient analysis accuracy. For example, when an event lacks a username or IP address, it is impossible to effectively associate it with a specific user or source, making it difficult to construct a complete behavioral sequence or attack path.

[0004] Therefore, there is an urgent need in this field for a technical solution that can informatively fill the main values ​​of an event with the most probable values ​​in order to improve the relevance of events and the accuracy of analysis. Summary of the Invention

[0005] In a first aspect, this application provides a method for filling missing values ​​in log events. The method includes: parsing log data for events; during the event parsing process, obtaining the username, source IP, and preset related identifiers of the event; the preset related identifiers are preset identifiers used for context association to fill missing subject values; the preset related identifiers include session identifiers and / or process identifiers; the subject values ​​include username and source IP; and when it is determined that the attribute information of a first event is missing one or two of the subject values, the preset related identifiers of the first event are not missing, the preset related identifiers of a second event match the preset related identifiers of the first event, and the attribute information of the second event is not missing the subject values, the missing subject values ​​of the first event are filled based on the subject values ​​of the second event.

[0006] The missing value filling method for log events provided in this application establishes an intelligent filling mechanism based on context association. By using the two inherent association identifiers of the system, session identifier and / or process identifier, it provides a basis for filling values ​​for events with missing usernames or source IPs. Security events are no longer discarded or filled with null values ​​due to missing key attributes. This accurately restores the main information of the event, improves the relevance of the event and the accuracy of the analysis, and increases the utilization rate of the event log.

[0007] In some possible implementations, the method further includes: determining an attack information mapping table, which indicates the mapping relationship between the event ID, attack stage, and attack technique category of a preset attack event; during event parsing, obtaining the attack stage and attack technique category corresponding to the event ID based on the attack information mapping table, and writing the attack stage and attack technique category as new fields into the event object; after filling in the missing subject value of the first event, the method further includes: performing event aggregation and time interval aggregation on similar events with the same event ID, the same operation object, the same subject value, and the event time interval within a preset time interval, wherein the same user subject includes the same username or source IP; based on the aggregated events, constructing a cross-stage attack behavior sequence corresponding to the target subject, wherein nodes in the behavior sequence represent events corresponding to the attack stage, edges represent transitions between attack stages, and the attribute information of each node includes the attack stage name, attack technique category, time interval, and event list, the event list including all events corresponding to the attack stage represented by the node, and the target subject is a user subject at the username dimension or a user subject at the source IP dimension.

[0008] This approach assigns security semantics to discrete events by mapping event IDs to attack stages; similar event aggregation can significantly reduce data volume and improve analysis efficiency; and behavioral sequences can provide an intuitive view of attack progress for security analysis. Therefore, by adding attack chain modeling (determining the attack information mapping table), event aggregation, and behavioral sequence construction techniques to the data preprocessing mechanism for filling missing values, a complete technical chain from data preprocessing to cross-stage analysis of attack events is formed.

[0009] In some possible implementations, the attack phase includes at least two of the following: credential access and initial access, persistence and privilege escalation, discovery and lateral movement, and execution and persistence. Specifically, the attack techniques corresponding to the credential access and initial access phase include Kerberos authentication and login events; the attack techniques corresponding to the persistence and privilege escalation phase include user account operations and group management operations; the attack techniques corresponding to the discovery and lateral movement phase include file system access and shared resource access; and the attack techniques corresponding to the execution and persistence phase include process creation and service installation.

[0010] This approach maps each specific Windows event type to an attack phase, providing clear guidance for developing detection rules.

[0011] In some possible implementations, the preset related identifiers include a session identifier and a process identifier. The step of filling in the missing subject value of the first event based on the subject value of the second event, when it is determined that one or both of the subject values ​​of the first event are missing, the preset related identifiers of the first event are not missing, the preset related identifiers of the second event match the preset related identifiers of the first event, and the subject value of the second event is not missing, includes: during event parsing, storing the association between the username, source IP, and session identifier of the session corresponding to the login event in an active session list; and storing the process identifier of the reference process created corresponding to the process creation event, the username used to create the reference process, and the source IP. The relationships between the reference process and its parent process are stored in a process parent-child relationship tree. If it is determined that one or two of the main body values ​​are missing from the attribute information of the first event, and the session identifier of the first event is not missing, then if the active session list records a first session matching the session identifier of the first event, the missing main body value of the first event is filled in based on the username and source IP of the first session; or, if it is determined that one or two of the main body values ​​are missing from the attribute information of the first event, and the process identifier of the first event is not missing, then if the process parent-child relationship tree records a first process matching the process identifier of the first event, the missing main body value of the first event is filled in based on the username and source IP of the first process.

[0012] This approach offers several advantages. First, the dynamically maintained list of active sessions and the parent-child relationship tree of processes avoid repeated scanning of the original logs, significantly improving processing efficiency compared to traversing the large set of events one by one to find related events, thus meeting the requirements of real-time security detection. Second, by using both session and process approaches in parallel or complementary ways, the completeness and success rate of the data filling scheme are improved.

[0013] In some possible implementations, the step of filling in the missing subject value of the first event based on the username and source IP of the first process if a first process matching the process identifier of the first event is recorded in the process parent-child relationship tree includes: if a first process matching the process identifier of the first event is recorded in the process parent-child relationship tree, and the first process records a username and source IP, then the missing subject value of the first event is filled in based on the username and source IP of the first process; if a first process matching the process identifier of the first event is recorded in the process parent-child relationship tree, and the first process does not record a username and source IP, then the missing subject value of the first event is filled in based on the username and source IP corresponding to the session identifier of the parent process of the first process recorded in the active session list; or, the missing subject value of the first event is filled in based on the username and source IP corresponding to the process identifier of the parent process of the first process recorded in the process parent-child relationship tree, and if the process chain of the process parent-child relationship tree shows that the parent process of the first process records a username and source IP, then the missing subject value of the first event is filled in based on the username and source IP of the parent process of the first process.

[0014] This method, when direct process information is missing, traces upwards through process parent-child relationships, ensuring that filling can still be completed in complex process forking scenarios, thus improving the success rate. It is particularly suitable for detecting advanced threats that hide the real attacker through multi-level process forking.

[0015] In some possible implementations, the active session list is further used to record the session lifespan. The active session list includes a current active session list and a historical active session list. The current active session list records currently active, unregistered sessions, and the historical active session list records historically active, deregistered sessions. If the active session list records a first session matching the session identifier of the first event, then the missing subject value of the first event is filled based on the username and source IP of the first session. This includes: if the active session list records a first reference session matching the session identifier of the first event, and the event time of the first event matches the lifespan information of the first reference session, then the missing subject value of the first event is filled based on the username and source IP of the first reference session; if the active session list records a first reference session matching the session identifier of the first event, but the event time of the first event does not match the lifespan information of the first reference session, and the historical active session list records a second reference session corresponding to the session identifier of the first event, and the event time of the first event matches the lifespan information of the second reference session, then the missing subject value of the first event is filled based on the username and source IP of the second reference session.

[0016] This approach introduces a session lifespan verification mechanism to distinguish between currently active sessions and historical sessions, ensuring the rationality of the filling time and preventing the use of information from a new session to fill the events corresponding to an old session under the session identifier reuse mechanism, thus avoiding time logic errors.

[0017] In some possible implementations, the method further includes: in the event of a conflict between the first padding value and the second padding value, calculating a first confidence level and a second confidence level corresponding to the first padding value and the second padding value, respectively, wherein the first confidence level is related to a first baseline confidence level, time correlation, and event type reliability, and the second confidence level is related to the first baseline confidence level, time correlation, and process depth, wherein the first padding value is the username and source IP determined based on an association padding method with the same session identifier, and the second padding value is the username and source IP determined based on an association padding method with the same process identifier; and filling in the missing subject value of the first event based on the padding value with the higher confidence level between the first padding value and the second padding value.

[0018] This approach, based on a conflict resolution strategy using multi-dimensional confidence assessment, ensures the selection of the optimal solution when multiple filling methods yield inconsistent results.

[0019] Secondly, this application also provides a missing value filling device for log events, including a unit for performing any of the missing value filling methods for log events in the first aspect.

[0020] Thirdly, this application also provides a computer storage medium that can store multiple instructions adapted for loading and executing by a processor the missing value filling method for any of the log events in the first aspect.

[0021] Fourthly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a missing value filling method for any log event in the first aspect is executed.

[0022] Fifthly, embodiments of this application also provide a computer program product containing instructions that, when the computer program product is run on an electronic device, cause the electronic device to execute any of the missing value filling methods for log events in the first aspect.

[0023] In a sixth aspect, embodiments of this application also provide a chip module, including a transceiver component and a chip, wherein the chip is used to execute any of the missing value filling methods for log events in the first aspect.

[0024] It is understood that the missing value filling device for log events, computer storage medium, electronic device, computer program, computer program product, and chip system provided above are all used to execute the method shown in any implementation of the first aspect of the embodiments of this application. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods, and will not be repeated here. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating a method for filling missing values ​​in log events according to an embodiment of this application. Figure 2 This is a flowchart illustrating a method for filling missing values ​​by maintaining an active session list and a process parent-child relationship tree, as provided in an embodiment of this application. Figure 3 This is a flowchart illustrating another method for filling missing values ​​in log events provided in an embodiment of this application; Figure 4 This is a flowchart illustrating another method for filling missing values ​​in log events provided in an embodiment of this application; Figure 5 This is a schematic diagram of the architecture of a system for filling missing values ​​in log events provided in an embodiment of this application; Figure 6 This is a schematic diagram of a device for filling missing values ​​in log events provided in an embodiment of this application; Figure 7 This is a schematic diagram of another log event missing value filling device provided in the embodiments of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described below in conjunction with the accompanying drawings.

[0027] It should be noted that this application embodiment uses an electronic device as an example to illustrate the execution subject of the missing value filling method for log events provided in this application. The electronic device can also be understood as the missing value filling device for log events shown in this application embodiment. In this application embodiment, the electronic device can be a microprocessor or computer for executing program code, etc. Any electronic device that can be used to execute the method provided in this application embodiment falls within the protection scope of this application embodiment, and this application does not impose any limitations. For example, the electronic device can be a desktop computer, a laptop, a mobile terminal, a 32-bit microprocessor, or a 64-bit microprocessor, etc., and this application embodiment does not limit this.

[0028] Example 1: Please see Figure 1 , Figure 1This is a flowchart illustrating a method for filling missing values ​​in log events, provided in an embodiment of this application. Figure 1 As shown, the method for filling missing values ​​in this log event includes the following steps: S101, the electronic device performs event parsing on the log data to obtain the username, source IP, and preset related identifiers of the event.

[0029] In this embodiment of the application, the preset related identifier is a preset related identifier used for context association to fill in missing subject values. The preset related identifier includes session identifier (LogonID) and / or process identifier (ProcessID), and the subject value includes username and / or source IP.

[0030] S102, when the electronic device determines that one or two of the missing main values ​​of the attribute information of the first event, the preset related identifier of the first event is not missing, the preset related identifier of the second event matches the preset related identifier of the first event, and the attribute information of the second event is not missing a main value, the electronic device fills in the missing main value of the first event based on the main value of the second event.

[0031] In this embodiment of the application, the first event and the second event are any two events in the Windows event log data.

[0032] For example, the preset related identifiers include session identifiers. Based on session consistency rules, the electronic device assigns the username and source IP of the event with the complete subject value to the event without username and source IP values ​​for two events with the same session identifier.

[0033] For example, the preset related identifiers include process identifiers. Based on process consistency rules, the electronic device assigns the username and source IP of the event with complete subject values ​​directly to the event without username and source IP values ​​for two events with the same process identifier.

[0034] For example, the preset relevant identifiers include session identifiers and process identifiers. The electronic device first fills in the missing subject value of the first event based on session consistency rules. If the missing subject value of the first event cannot be filled in based on session consistency rules, then the missing subject value of the first event is filled in based on process consistency rules.

[0035] The missing value filling method for log events provided in this application establishes an intelligent filling mechanism based on context association. By using the two inherent association identifiers of the system, session identifier and / or process identifier, it provides a basis for filling values ​​for events with missing usernames or source IPs. Security events are no longer discarded or filled with null values ​​due to missing key attributes. This accurately restores the main information of the event, improves the relevance of the event and the accuracy of the analysis, and increases the utilization rate of the event log.

[0036] In some possible implementations, the electronic device can determine the second event that matches a preset related identifier of the first event by traversing and searching through events.

[0037] In other possible implementations, the electronic device can also maintain context association information (the association between preset related identifiers, usernames, and source IPs) and, when it is determined that the attribute information of the first event is missing a subject value, fill in the corresponding missing subject value in the first event based on the preset related identifier of the first event and the context association information, using the username and source IP recorded in the context association information that match the preset related identifier of the first event.

[0038] As an example, taking preset related identifiers including session identifier and process identifier as an example, combined with Figure 2 This explains how an electronic device fills in the missing subject value of the first event based on the subject value of the second event. For example... Figure 2 As shown, it includes the following steps: S1021, During event resolution, the electronic device maintains a list of active sessions and a parent-child relationship tree of processes.

[0039] Maintaining the active session list includes storing the association between the username, source IP, and session identifier of the corresponding session created by the login event in the active session list.

[0040] Generally, when the system detects a login event, it creates a unique session for the user based on the user principal values ​​(username and source IP). The Windows kernel and security subsystem update all active login sessions in real time. This information is distributed across various system components (LSA, SAM, Security Reference Monitor, etc.), but there is no unified, directly readable "list of active sessions." The same applies to process creation events. When a process is created, a user principal value is bound to it, and the detailed information about process creation is distributed across various system components; there is no unified, directly readable "process parent-child relationship tree."

[0041] This solution records session details (including session identifier, username, source IP, and lifespan) based on real login events. Upon detecting an actual login event, the session information is recorded in the active session list. Additionally, it records process details (including session identifier, username, source IP, and lifespan) based on real process creation events. Upon detecting an actual process creation event, the process information is recorded in the process parent-child relationship tree.

[0042] It should be noted that the session or process lifespan can include the creation time (start time) and the logout time (end event) (if logged out, the logout time is included; if not logged out, it is not included). Session logout means that the user logs out and the session ends (not that the account is deleted). Process logout means that the user exits the process and the process ends.

[0043] S1022, the electronic device fills in the missing subject value of the first event based on the association filling method with the same session identifier, or fills in the missing subject value of the first event based on the association filling method with the same process identifier.

[0044] In some possible implementations, the above-mentioned method of filling missing subject values ​​of the first event based on the same session identifier includes: if it is determined that one or two of the subject values ​​of the attribute information of the first event are missing, and the session identifier of the first event is not missing, if there is a first session in the active session list that matches the session identifier of the first event, then the missing subject values ​​of the first event are filled based on the username and source IP of the first session.

[0045] It should be noted that the missing subject value in the first event may include the username and / or source IP. Filling the missing subject value in the first event based on the username and source IP of the first session means filling the corresponding missing subject value in the first event using the username and source IP of the first session. For example, if the first event is missing only the username, then the username of the first session is used to fill the missing username in the first event; if the first event is missing only the source IP, then the source IP of the first session is used to fill the missing source IP in the first event; if the first event is missing both the username and the source IP, then the username of the first session is used to fill the missing username in the first event, and the source IP of the first session is used to fill the missing source IP in the first event. The corresponding meanings of other filling methods in this article are similar.

[0046] In some possible implementations, the active session list is also used to record the session lifespan. The active session list includes a currently active session list and a historical active session list. The currently active session list is used to record currently active, unregistered sessions, and the historical active session list is used to record historically active, unregistered sessions.

[0047] Accordingly, to avoid the incorrect use of new session information due to session identifier reuse, if the active session list records a first session that matches the session identifier of the first event, then the missing subject values ​​of the first event are filled in based on the username and source IP of the first session, including: 1) If the active session list contains a first reference session that matches the session identifier of the first event, and the event time of the first event matches the lifespan information of the first reference session, then fill in the missing subject value of the first event based on the username and source IP of the first reference session.

[0048] The matching of the event time of the first event with the lifespan information of the first reference session means that the event time of the first event is later than or equal to the creation time of the first reference session, and vice versa if they do not match.

[0049] 2) If the active session list contains a first reference session that matches the session identifier of the first event, the event time of the first event does not match the lifespan information of the first reference session, the historical active session list contains a second reference session that corresponds to the session identifier of the first event, and the event time of the first event matches the lifespan information of the second reference session, then fill in the missing subject value of the first event based on the username and source IP of the second reference session.

[0050] The matching of the event time of the first event with the lifespan information of the second reference session means that the event time of the first event is later than or equal to the creation time of the second reference session, and the event time of the first event is earlier than or equal to the deregistration time of the second reference session; otherwise, the opposite applies.

[0051] It should be noted that the aforementioned "first session" can refer to both the first reference session and the second reference session.

[0052] In some possible implementations, the above-mentioned method of filling missing subject values ​​of the first event based on the association of the same process identifier includes: if it is determined that one or two of the missing subject values ​​of the attribute information of the first event are missing, and the process identifier of the first event is not missing, if a first process matching the process identifier of the first event is recorded in the process parent-child relationship tree, then the missing subject values ​​of the first event are filled based on the username and source IP of the first process.

[0053] For example, if the process parent-child relationship tree records a first process that matches the process identifier of the first event, then the missing subject value of the first event is filled in based on the username and source IP of the first process, including: 1) If the parent-child relationship tree records a first process that matches the process identifier of the first event, and the first process records a username and source IP, then fill in the missing subject value of the first event based on the username and source IP of the first process.

[0054] 2) If the process parent-child relationship tree records a first process that matches the process identifier of the first event, and the first process does not record a username and source IP, then fill the missing subject value of the first event based on the username and source IP corresponding to the session identifier of the parent process of the first process recorded in the active session list; or, fill the missing subject value of the first event based on the username and source IP corresponding to the process identifier of the parent process of the first process recorded in the process parent-child relationship tree.

[0055] For example, if the process chain in the process parent-child relationship tree shows that the parent process of the first process has a recorded username and source IP, then the missing subject value of the first event is filled in based on the username and source IP corresponding to the process identifier of the parent process of the first process. Alternatively, if the process parent-child relationship tree records the session identifier of the parent process of the first process (but does not record the process identifier of the parent process), and the active session list records the username and source IP corresponding to the session identifier of the parent process of the first process, then the missing subject value of the first event is filled in based on the username and source IP corresponding to the session identifier of the parent process of the first process.

[0056] It should be noted that the parent process of the first process can be the direct parent process of the first process or the indirect parent process of the first process (such as the parent process of the parent process). That is, the process step size of the parent process of the first process and the first process can be greater than or equal to 1. This article does not limit this.

[0057] In some possible implementations, the process parent-child relationship tree is also used to record the lifespan of processes. The process parent-child relationship tree includes the current process parent-child relationship tree and the historical process parent-child relationship tree. The active process parent-child relationship tree is used to record currently active processes that have not been cancelled, and the historical process parent-child relationship tree is used to record historically active processes that have been cancelled.

[0058] Accordingly, to avoid the erroneous use of new process information due to process identifier reuse, the electronic device also performs a time range matching judgment when filling the missing main value of the first event based on the association filling method with the same process identifier. Specifically, it matches the event time of the first event with the lifespan of the target process (the target process is the process with the same process identifier as the first event or its parent process). This can also be understood as matching the lifespan of the process used to fill the missing main value of the first event with the event time of the first event.

[0059] For example, in the above 1) if the process parent-child relationship tree records a first process that matches the process identifier of the first event, and the first process records a username and source IP, then the missing subject value of the first event is filled based on the username and source IP of the first process, specifically including: 1.1) If the current process parent-child relationship tree records a first reference process that matches the process identifier of the first event, and the first reference process records a username and source IP, and the event time of the first event matches the lifespan information of the first reference process, then fill in the missing subject value of the first event based on the username and source IP of the first reference process.

[0060] The matching of the event time of the first event with the lifespan information of the first reference process means that the event time of the first event is later than or equal to the creation time of the first reference process, and vice versa if they do not match.

[0061] 1.2) If the current process parent-child relationship tree records a first reference process that matches the process identifier of the first event, the first reference process records a username and source IP, the event time of the first event does not match the lifespan information of the first reference process, the historical process parent-child relationship tree records a second reference process that matches the process identifier of the first event, the second reference process records a username and source IP, and the event time of the first event matches the lifespan information of the second reference process, then fill in the missing subject value of the first event based on the username and source IP of the second reference process.

[0062] The matching of the event time of the first event with the lifespan information of the second reference process means that the event time of the first event is later than or equal to the creation time of the second reference process, and the event time of the first event is earlier than or equal to the deregistration time of the second reference process; otherwise, the opposite applies.

[0063] In some possible implementations, the electronic device can first fill in the missing subject value of the first event based on the association padding method with the same session identifier; if it is determined that the missing subject value of the first event cannot be filled based on the association padding method with the same session identifier, then the missing subject value of the first event can be filled based on the association padding method with the same process identifier.

[0064] In some other possible implementations, the electronic device may simultaneously determine the first padding value and the second padding value based on the same association padding method for session identifiers and the same association padding method for process identifiers, respectively.

[0065] If the first padding value and the second padding value are the same, the missing subject value of the first event is filled based on either the first padding value or the second padding value. If the first padding value and the second padding value conflict (e.g., different usernames and / or different source IPs), the missing subject value of the first event is filled based on the first confidence level and the second confidence level corresponding to the first padding value and the second padding value, respectively, using the padding value with the higher confidence level between the first padding value and the second padding value. The first padding value can also be understood as the subject value of the first session, the subject value of the first reference session, or the subject value of the second reference session. The second padding value can also be understood as the subject value of the first process or the parent process of the first process, the subject value of the first reference process or the parent process of the first reference process, or the subject value of the second reference process or the parent process of the second reference process.

[0066] For example, the first confidence level corresponding to the first padding value is related to at least one of a first baseline confidence level, time correlation, and event type reliability. The second confidence level corresponding to the second padding value is related to at least one of a second baseline confidence level, time correlation, and process chain depth.

[0067] The first and second baseline confidence levels represent the baseline confidence levels for the association-based population methods based on identical session identifiers and identical process identifiers, respectively. In the default configuration of this scheme, the first baseline confidence level (e.g., 0.9) is greater than the second baseline confidence level (e.g., 0.7). The configuration is based on the following: session identifiers are directly associated with login events, which are the core of system security auditing, hence the higher confidence level. In contrast, process identifier populations rely on the process tree and process creation events, but processes can be deceived or injected, so the confidence level is relatively lower.

[0068] For example, the overall confidence level of the filled value is the product of the baseline confidence level, the time decay factor, and other adjustment factors.

[0069] Specifically, for the first confidence level based on the same session identifier association filling method: the baseline confidence level is the first baseline confidence level (e.g., 0.9), and the time decay factor is determined by calculating the absolute value of the time difference between the login event and the current first event (which can also be understood as the time difference between the initial time of session creation and the current first event). The larger the absolute value of the time difference, the smaller the time decay factor. For example, if the absolute value of the time difference is within 1 hour, the time decay factor is 1.0; 1 hour to 2 hours, the time decay factor is 0.9; 2 hours to 4 hours, the time decay factor is 0.8; and more than 4 hours, the time decay factor is 0.7. Other adjustment factors are the adjustment factors corresponding to the login type, for example, interactive login type is 1.0, network login type is 0.9, and others are 0.8.

[0070] For the second confidence level based on the association padding method with the same process identifier: the baseline confidence level is the second confidence level (e.g., 0.7). The time decay factor is calculated by the absolute value of the time difference between the process creation event and the current first event. The larger the absolute value of the time difference, the smaller the time decay factor. The mapping relationship between the time difference and the time decay factor is similar to the method for session identifiers, and will not be elaborated here. Other adjustment factors are the process depth, i.e., the number of hops from the root process. The fewer the hops, the higher the factor. For example, 1 hop is 1.0, 2 hops is 0.9, 3 hops is 0.8, and more than 3 hops is 0.7.

[0071] Example 2: In some possible implementations, during event parsing, the electronic device also acquires the attack stage and attack technique category of the event. After filling in the missing subject value of the first event, it aggregates similar events and constructs a cross-stage attack behavior sequence corresponding to the target subject based on the similar events. The following combines... Figure 3 This application describes one implementation of an embodiment, including: S301, Electronic device determines attack information mapping table.

[0072] In this embodiment of the application, the attack information mapping table is used to indicate the mapping relationship between the event ID, attack stage, and attack technique category of a preset attack event.

[0073] S302, the electronic device performs event parsing on the log data. During the event parsing process, it obtains the username, source IP, and preset related identifiers of the event, as well as the attack stage and attack technology category corresponding to the event ID based on the attack information mapping table, and writes the attack stage and attack technology category of the event as new fields into the event object.

[0074] S303, the electronic device fills in the missing subject value of the first event based on the preset related identifier association filling method.

[0075] For details on how to fill in the missing main value of the first event based on the same preset related identifier, please refer to the relevant descriptions in step S102 or steps S1021-S1022 of Embodiment 1 above, which will not be described in detail here.

[0076] S304, the electronic device performs event aggregation and time interval aggregation on similar events with the same event ID, the same operation object, the same subject value, and the event time interval within a preset time interval.

[0077] Among these, "same user entity" includes the same username or source IP. The preset time interval can be designed based on specific needs; this article does not impose any restrictions on it. For example, the preset time interval could be within 5 minutes.

[0078] S305, based on aggregated events, the electronic device constructs a cross-stage attack behavior sequence corresponding to the target subject, where nodes in the behavior sequence represent events corresponding to the attack stage, and edges represent transitions between attack stages.

[0079] In this embodiment, the attribute information of each node in the behavior sequence includes the attack phase name, attack technique category, time interval, and event list. The event list includes all events corresponding to the attack phase represented by the node, and the target subject is a user subject at the username level or a user subject at the source IP level. In some other possible implementations, after step S305, the electronic device also outputs a behavior path diagram in the form of an attack chain based on the behavior sequence.

[0080] Example 3: The following combination Figure 4 This paper introduces the specific implementation process of applying the missing value filling method based on log events to Windows log data analysis scenarios: S401, Determine the attack information mapping table: Establish the mapping relationship between event IDs and attack stages.

[0081] Specifically, the attack information mapping table can be entered based on human experience. It should be noted that the attack information mapping table is different for different application scenarios. Table 1 below is an example of a Windows event log analysis scenario, which is based on the analysis of attack chains from Windows events. The corresponding attack information mapping table is shown in Table 1 below.

[0082] Table 1 S402, Event Preprocessing: The original event is marked with the attack stage and attack technology category, and the username and source IP are extracted.

[0083] For example, step S402 specifically includes the following steps: 1) For each original Windows event, look up the corresponding attack stage and attack technique category in the mapping table based on its event ID, and write it as a new field into the event record.

[0084] 2) Extract key attributes from the event, such as username, source IP, and target object.

[0085] In this embodiment, the target object refers to the entity operated on by the Window event. For example, if the event type of 4720 (Event ID) is "Create User," then the target object is the username created; if the event type of 4663 (Event ID) is "File Object Access Attempt," then the target object is the path of the file or folder being accessed; and if the event type of 5140 (Event ID) is "Network Share Access," then the target object is the path of the shared resource being accessed.

[0086] 3) Determine if the event contains a username and source IP. If no value is found, fill in the missing value using the following sub-logic.

[0087] Taking the event ID in the attack information mapping table corresponding to Table 1 above as an example, an example table for verifying whether the original event ID contains the username and source IP is shown in Table 2 below.

[0088] Table 2 3.1) Collect all user-queried Windows event data and sort them in ascending order by timestamp. 3.2) Initialize the following three core data structures: active_sessions: Stores currently active sessions (key: LogonID, value: session details) process_tree: A parent-child relationship tree of processes (key: ProcessID, value: process information) user_timeline: Activity timeline organized by user (key: UserSID, value: list of events) 3.3) Association based on identical key field values Based on the current active sessions and the process parent-child relationship tree, for sessions with the same identifier or process ID, the username and source IP with values ​​are directly assigned to events without username and source IP values.

[0089] 3.4) Process chain association Record the ProcessId and ParentProcessId of the new process. If the filling task cannot be completed by using the same association method based on the session identifier or process ID value, use the ProcessID and ParentProcessId of the current event to enter the process_tree process parent-child relationship tree in the queried dataset, and perform association filling based on the session identifier and process identifier of the creator corresponding to the parent process event, in accordance with step 3.3.

[0090] S403, Duplicate Event Aggregation: Compresses and merges time intervals of similar events.

[0091] Specifically, events that meet the following conditions are merged: same event ID, same username or source IP, same target object, and within a preset time range. For example, if the event ID, username or source IP, and target object attribute values ​​are determined to be the same, then it is determined whether they occur consecutively (within a preset time range).

[0092] For example, only events that occur consecutively are merged. An illustration of consecutive occurrences is shown below: Before aggregation: 14:00:01 - 4624 User A logged in from 192.1xx.1.100; 14:00:03 - 4624 User A logged in from 192.1xx.1.100; 14:00:05 - 4624 User A logged in from 192.1xx.1.100; 14:15:00 - 4625 User B failed to log in from 192.1xx.1.101; 14:15:01 - 4624 User A logged in from 192.1xx.1.100.

[0093] After aggregation: 14:00:01-14:00:05 - 4624 User A logged in from 192.1xx.1.100 [3 times]; 14:15:00 - 4625 User B failed to log in from 192.1xx.1.101 [1 time]; 14:15:01 - 4624 User A logged in from 192.1xx.1.100 [1 time].

[0094] After merging, retain the time range (from minimum to maximum timestamp) and record the total number of events occurring within that range. Other attributes such as process name and login type can be merged into a list or count statistics.

[0095] S404, Behavior Sequence Construction: Construct a behavior sequence for cross-stage attacks by user or source IP dimension.

[0096] Using username or source IP as the dimension, aggregated events are sorted chronologically to construct a behavior sequence. Each node in the sequence represents a specific operation in an attack phase, and edges represent transitions between phases. The core objective of constructing the behavior sequence is to organize discrete security events into a coherent attack path according to logical relationships. This process, based on preprocessed and aggregated event data, reconstructs the attacker's complete activity trajectory from multiple dimensions.

[0097] 4.1) Dimensional Grouping Processing First, based on the analysis requirements, the main dimensions for event grouping are selected, supporting both username and source IP addresses. In user-based analysis, all events are categorized according to the associated user account; in IP-based analysis, they are organized according to the source network address. During the grouping process, a mapping relationship between dimension keys and event sets is established, laying the foundation for subsequent sequence construction.

[0098] 4.2) Time Series Sort For events within each dimension group, they are strictly sorted according to their occurrence time to ensure the temporal logical correctness of the event sequence. The sorting process considers the start timestamp of the events; for events that include time intervals after aggregation, the starting point of the time interval is used as the sorting basis. This step ensures that the attack path accurately reflects the temporal development sequence of the attack behavior.

[0099] S405, Visualization Output: Outputs a behavior path graph in the form of an attack chain based on the behavior sequence.

[0100] 5.1) Construction of Stage Nodes Iterate through the time-sorted list of events to identify points of change in the attack phase. When a new event is detected whose attack phase differs from the previous event, a new phase node is created. Each phase node records the following key information: attack phase name, type of attack technique used, phase start time, and the set of all events contained within that phase.

[0101] 5.2) Event aggregation within a phase For events occurring consecutively within the same attack phase, they are merged into the same phase node. During the merging process, the end time of the phase node is updated to the end time of the latest event, and this event is added to the event set of the phase node. This approach maintains the integrity of the attack phases while avoiding excessive subdivision within phases.

[0102] 5.3) Rich sequence metadata After constructing the phase sequence, the system generates a complete sequence description object for each dimension key. This object contains information such as the analysis dimension identifier, dimension key, attack phase sequence, total number of events, and the time span of the entire sequence. The time span is calculated based on the start time of the earliest event and the end time of the latest event in the sequence, accurately reflecting the duration of the attack activity.

[0103] This application also provides a device for filling missing values ​​in log events, including a unit for performing any of the missing value filling methods for log events in the above method embodiments.

[0104] Please refer to Figure 5 This is a schematic diagram of a device for filling missing values ​​in log events, provided in an embodiment of this application. Figure 5 As shown, the missing value filling device for log events in this application embodiment may include: The parsing unit 501 is used to parse log data for events. During the event parsing process, it obtains the username, source IP, and preset related identifiers of the event. The filling unit 502 is used to fill in the missing main value of the first event based on the main value of the second event when it is determined that one or two of the main values ​​of the attribute information of the first event are missing, the preset related identifier of the first event is not missing, the preset related identifier of the second event matches the preset related identifier of the first event, and the main value of the attribute information of the second event is not missing.

[0105] In some possible implementations, such as Figure 6 As shown, the above-mentioned device also includes: Determining unit 503 is used to determine the attack information mapping table; The parsing unit 501 is also used to obtain the attack stage and attack technology category corresponding to the event ID based on the attack information mapping table during the event parsing process, and write the attack stage and attack technology category of the event as new fields into the event object. Aggregation unit 504 is used to aggregate similar events with the same event ID, the same operation object, the same subject value, and the event time interval within a preset time interval, and to perform event aggregation and time interval aggregation. The behavior sequence construction unit 505 is used to construct a cross-stage attack behavior sequence corresponding to the target subject based on the aggregated events. In the behavior sequence, the nodes represent events corresponding to the attack stages, and the edges represent the transitions between attack stages.

[0106] In some possible implementations, the aforementioned filling unit 502 specifically includes: The association information maintenance subunit 5021 is used to store the association relationship between the username, source IP, and session identifier of the login event created in the login event into the active session list during the event parsing process; and to store the association relationship between the process identifier of the reference process created in the process creation event, the username and source IP used to create the reference process, and the parent process of the reference process into the process parent-child relationship tree. The session association filling subunit 5022 is configured to, when determining that one or two of the subject values ​​of the first event are missing from the attribute information of the first event, and the session identifier of the first event is not missing, if a first session matching the session identifier of the first event is recorded in the active session list, then fill in the missing subject values ​​of the first event based on the username and source IP of the first session; or, The process association filling subunit 5023 is used to fill in the missing main values ​​of the first event based on the username and source IP of the first process if a first process matching the process identifier of the first event is recorded in the process parent-child relationship tree when it is determined that one or two of the main values ​​of the first event are missing in the attribute information of the first event and the process identifier of the first event is not missing.

[0107] In some possible implementations, the aforementioned process association filling unit is specifically used to fill in the missing subject value of the first event based on the username and source IP of the first process if it is determined that a first process matching the process identifier of the first event is recorded in the process parent-child relationship tree, and the first process records a username and source IP; and if a first process matching the process identifier of the first event is recorded in the process parent-child relationship tree, and the first process does not record a username and source IP, then fill in the missing subject value of the first event based on the username and source IP corresponding to the session identifier of the parent process of the first process recorded in the active session list, or fill in the missing subject value of the first event based on the username and source IP corresponding to the process identifier of the parent process of the first process recorded in the process parent-child relationship tree, and if the process chain of the process parent-child relationship tree shows that the parent process of the first process records a username and source IP, then fill in the missing subject value of the first event based on the username and source IP of the parent process of the first process.

[0108] In some possible implementations, the session association fill sub-unit is specifically used to fill in the missing subject value of the first event based on the username and source IP of the first reference session if the active session list records a first reference session that matches the session identifier of the first event and the event time of the first event matches the lifespan information of the first reference session. If the active session list records a first reference session that matches the session identifier of the first event, the event time of the first event does not match the lifespan information of the first reference session, the historical active session list records a second reference session that corresponds to the session identifier of the first event, and the event time of the first event matches the lifespan information of the second reference session, then the missing subject value of the first event is filled in based on the username and source IP of the second reference session.

[0109] In some possible implementations, the above-mentioned apparatus further includes: The conflict handling unit 506 is configured to, in the event of a conflict between the first padding value and the second padding value, calculate a first confidence level and a second confidence level corresponding to the first padding value and the second padding value, respectively. The first confidence level is related to a first baseline confidence level, time correlation, and event type reliability. The second confidence level is related to the first baseline confidence level, time correlation, and process depth. The first padding value is the username and source IP determined based on an association padding method with the same session identifier. The second padding value is the username and source IP determined based on an association padding method with the same process identifier. The unit 506 is also configured to fill in the missing subject value of the first event based on the padding value with the higher confidence level between the first padding value and the second padding value.

[0110] In the embodiments of this application, any of the implementation methods mentioned in the method embodiments are also applicable to the missing value filling device for log events provided in this application. For specific execution steps, please refer to the description of the foregoing method embodiments, which will not be detailed here.

[0111] This application embodiment also provides a missing value filling device for log events, including a processor, the processor being used to execute any of the missing value filling methods for log events in the above method embodiments.

[0112] Please refer to Figure 7 This is a schematic diagram of another log event missing value filling device provided in an embodiment of this application, as shown below. Figure 7 As shown, the missing value filling device 700 for log events may include: at least one processor 701, such as a CPU, at least one communication interface 703, a memory 704, and at least one communication bus 702. The communication bus 702 is used to implement communication between these components. The communication interface 703 may optionally include a standard wired interface, a wireless interface (such as a Wi-Fi interface or a Bluetooth interface), etc. The memory 704 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 704 may also be at least one storage device located remotely from the aforementioned processor 701. Figure 7 As shown, the memory 704, which serves as a computer storage medium, may include an operating system, a network communication module, and program instructions.

[0113] exist Figure 7 In the missing value filling device 700 for log events shown, the processor 701 can be used to load program instructions stored in the memory 704 and specifically execute any of the missing value filling methods for log events in the above method embodiments.

[0114] It should be noted that the specific execution process can be found in the detailed description of the above method embodiments, and will not be elaborated here.

[0115] For specific execution steps, please refer to the description of the foregoing method embodiments, which will not be detailed here.

[0116] This application also provides a computer storage medium that can store multiple instructions. These instructions are adapted to be loaded and executed by a processor using the missing value filling method for log events provided in this application. For details of the execution process, please refer to the specific description of the method embodiments shown above, which will not be elaborated here.

[0117] This application also provides a computer program product containing instructions that, when run on an electronic device, cause the electronic device to execute the method steps of the method embodiments shown above.

[0118] This application also provides a chip module, including a transceiver component and a chip, wherein the chip is used to execute the method steps of the above-described method embodiments.

[0119] It is understood that the missing value filling system, missing value filling device, computer storage medium, computer program, computer program product, and chip provided above are all used to execute the method shown in any implementation of the corresponding aspect of the embodiments of this application. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding method, and will not be detailed here.

[0120] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes the processes of the embodiments of the above methods.

[0121] The term "at least one" in this application refers to one or more items. "More than one item" means two or more items. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, it should be understood that although the terms "first," "second," etc., may be used to describe objects in this application, these objects should not be limited to these terms. These terms are only used to distinguish the objects from each other.

[0122] The terms “including” and “having”, and any variations thereof, mentioned above are intended to cover non-exclusive inclusion.

[0123] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for missing value imputation of log events, characterized in that, The method comprises: event parsing on log data, in the event parsing process, obtaining a username, a source IP, and a preset related identifier of an event, the preset related identifier being a preset related identifier used for context association to fill in a missing subject value, the preset related identifier including a session identifier and / or a process identifier, and the subject value including the username and the source IP; in a case where it is determined that attribute information of a first event is missing one or both of the subject values, a preset related identifier of the first event is not missing, a preset related identifier of a second event matches the preset related identifier of the first event, and attribute information of the second event is not missing the subject values, filling in the missing subject values of the first event based on the subject values of the second event.

2. The method of claim 1, wherein, The method further comprises: determining an attack information mapping table, the attack information mapping table being used to indicate a mapping relationship among event IDs, attack stages, and attack technology categories of preset attack events; in the event parsing process, obtaining the attack stage and the attack technology category corresponding to the event ID of the event based on the attack information mapping table, and writing the attack stage and the attack technology category of the event into an event object as new fields; after the filling in of the missing subject values of the first event, the method further comprises: performing event aggregation and time interval aggregation on similar events with the same event ID, the same operation object, the same subject value, and the same event time interval within a preset time interval, the same user subject including the same username or the same source IP; based on the aggregated events, constructing a behavior sequence of a cross-stage attack corresponding to a target subject, wherein a node in the behavior sequence represents an event corresponding to an attack stage, an edge represents a transition between attack stages, and attribute information of each node includes an attack stage name, an attack technology category, a time interval, and an event list, the event list including all events corresponding to the attack stage represented by the node, and the target subject being a user subject in a username dimension or a user subject in a source IP dimension.

3. The method of claim 2, wherein the attack stages include at least two of credential access and initial access, persistence and privilege escalation, discovery and lateral movement, and execution and persistence; wherein the attack technology categories corresponding to the credential access and initial access stages include Kerberos authentication and login events, the attack technology categories corresponding to the persistence and privilege escalation stages include user account operations and group management operations, the attack technology categories corresponding to the discovery and lateral movement stages include file system access and shared resource access, and the attack technology categories corresponding to the execution and persistence stages include process creation and service installation.

4. The method according to any one of claims 1 to 3, characterized in that, The preset related identifier includes a session identifier and a process identifier, and the filling the subject value missing in the first event based on the subject value of the second event in the case that one or both of the subject values in the attribute information of the first event is missing, the preset related identifier of the first event is not missing, the preset related identifier of the second event matches the preset related identifier of the first event, and the attribute information of the second event is not missing the subject value includes: In the event analysis process, the association relationship among the username of the login event, the source IP, and the session identifier of the session corresponding to the login event is stored in the active session list; and the association relationship among the process identifier of the reference process corresponding to the process creation event, the username and the source IP used by the reference process, and the parent process of the reference process is stored in the process parent-child relationship tree; In the case that one or both of the subject values in the attribute information of the first event is missing, and the session identifier of the first event is not missing, if the first session matching the session identifier of the first event is recorded in the active session list, the missing subject value of the first event is filled based on the username and the source IP of the first session; or, In the case that one or both of the subject values in the attribute information of the first event is missing, and the process identifier of the first event is not missing, if the first process matching the process identifier of the first event is recorded in the process parent-child relationship tree, the missing subject value of the first event is filled based on the username and the source IP of the first process.

5. The method of claim 4, wherein, The filling the missing subject value of the first event based on the username and the source IP of the first process in the case that the first process matching the process identifier of the first event is recorded in the process parent-child relationship tree includes: If the first process matching the process identifier of the first event is recorded in the process parent-child relationship tree, and the username and the source IP of the first process are recorded, the missing subject value of the first event is filled based on the username and the source IP of the first process; If the first process matching the process identifier of the first event is recorded in the process parent-child relationship tree, and the username and the source IP of the first process are not recorded, the missing subject value of the first event is filled based on the username and the source IP corresponding to the session identifier of the parent process of the first process recorded in the active session list, or the missing subject value of the first event is filled based on the username and the source IP corresponding to the process identifier of the parent process of the first process recorded in the process parent-child relationship tree, and the process chain of the process parent-child relationship tree displays that the parent process of the first process records the username and the source IP, and the missing subject value of the first event is filled based on the username and the source IP of the parent process of the first process.

6. The method of claim 4 or 5, wherein, The active session list is also used to record the survival time of the session, and the active session list includes a current active session list and a historical active session list, the current active session list is used to record the currently active session which has not been logged out, and the historical active session list is used to record the historical active session which has been logged out, If the active session list records a first session matching the session identifier of the first event, the subject value missing in the first event is filled based on the username and source IP of the first session, including: If the active session list records a first reference session matching the session identifier of the first event, and the event time of the first event matches the survival time information of the first reference session, the subject value missing in the first event is filled based on the username and source IP of the first reference session. If the active session list records a first reference session matching the session identifier of the first event, the event time of the first event does not match the survival time information of the first reference session, the historical active session list records a second reference session corresponding to the session identifier of the first event, and the event time of the first event matches the survival time information of the second reference session, the subject value missing in the first event is filled based on the username and source IP of the second reference session.

7. The method according to any one of claims 4 to 6, characterized in that, The method further comprises: In the case of conflict between the first filling value and the second filling value, the first confidence degree and the second confidence degree corresponding to the first filling value and the second filling value are calculated, respectively, the first confidence degree is related to the first reference confidence degree, time correlation, and event type reliability, the second confidence degree is related to the first reference confidence degree, time correlation, and process depth, the first filling value is the username and source IP determined based on the same association filling mode of the session identifier, and the second filling value is the username and source IP determined based on the same association filling mode of the process identifier; The subject value missing in the first event is filled based on the filling value with higher confidence degree in the first filling value and the second filling value.

8. An apparatus for missing value imputation of log events, characterized in that, The unit for executing the method as claimed in any one of claims 1 to 7 is included.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium is used to store a computer program, when the computer program is executed, the method as claimed in any one of claims 1 to 7 is executed.

10. An electronic device, comprising: The memory, the processor, and the computer program stored on the memory and executable on the processor are included, when the processor executes the program, the method as claimed in any one of claims 1 to 7 is executed.

Images