Identifying siem event types

Abstract

A method is disclosed. The method includes determining that an event type of an event log of a security information and event management (SIEM) cannot be identified. The method also includes generating a vectorized log using a sanitized, tokenized, and populated version of the event log. Additionally, the method includes generating a classification of the vectorized log using a deep learning classification model trained to identify a potential event type of the event log based on deep learning training using a plurality of parsed logs. The method also includes determining that a confidence level of the classification satisfies a predetermined threshold. The method also includes parsing the event log based on the classification.

Description

Background Technology

[0001] This disclosure relates to Security Information and Event Management (SIEM), and more specifically, to identifying SIEM event types.

[0002] The term SIEM can refer to software tools and / or services that combine security information and the management of security incidents (i.e., security breaches). In this way, SIEM can analyze logs generated by computer systems and / or computer networks to identify potential security breaches in real time.

[0003] Typically, security analysts identify threats to networked computer systems by analyzing transactions (i.e., events) entering those systems. These events are logged. However, due to the sheer volume of logs, processing the raw log data in a timely manner to mitigate any potential damage is challenging for security analysts. Therefore, SIEMs process these logs in a process called event normalization and classification, from which the SIEM generates violations that security analysts can review. A violation is an event that the SIEM identifies as a potential security incident. In event normalization and classification, the SIEM identifies the type and source of the event and passes it to a parser based on the identified type. The parser can use regular expressions (regex) to apply rules to the logs, which can then determine whether a violation has been generated.

[0004] However, regular expressions seek exact matches against predefined patterns, meaning that new events or variations of old events may not be properly standardized and categorized. These unknown logs lack crucial information such as event names or categories. Because these logs cannot be found in regular searches and do not trigger relevant rules, they are essentially useless, potentially leading to missed destructive security incidents. In some scenarios, 5-20% of events are not properly standardized and categorized, meaning security analysts and other monitoring tools may overlook millions of events. Summary of the Invention

[0005] Embodiments of the method are disclosed. The method includes determining that event types in the Security Information and Event Management (SIEM) event logs cannot be identified. The method also includes generating vectorized logs using cleaned, tokenized, and populated versions of the event logs. Additionally, the method includes generating a classification of the vectorized logs using a deep learning classification model trained to identify potential event types in the event logs based on deep learning training using multiple parsed logs. The method further includes determining that the confidence level of the classification meets a predetermined threshold. The method also includes parsing the event logs based on the classification. Advantageously, this embodiment is useful for identifying event types in logs that SIEM cannot recognize.

[0006] Optionally, in some embodiments, the method further includes using a convolutional neural network to train a deep learning classification model. This embodiment is useful for identifying log parsers for which SIEM cannot recognize their parsers.

[0007] Another embodiment of the method is disclosed. The method includes generating vectorized logs using a cleaned, tokenized, and populated version of the event log. The method also includes generating a classification of the vectorized logs using a deep learning classification model trained to identify potential event types for the event logs based on deep learning training using multiple parsed logs. The method further includes determining that the confidence level of the classification meets a predetermined threshold. The method also includes parsing the event logs based on the classification. Advantageously, this embodiment is useful for identifying event types in logs that SIEM cannot recognize.

[0008] Another embodiment of the method is disclosed. The method includes determining that event types in the Security Information and Event Management (SIEM) event logs cannot be identified. The method also includes generating vectorized logs using cleaned, tokenized, and populated versions of the event logs. The method further includes generating a classification of the vectorized logs using a deep learning classification model trained to identify potential event types in the event logs based on the parsed logs. The method also includes determining that the confidence level of the classification meets a predetermined threshold. The method further includes parsing the event logs based on the classification. Advantageously, this embodiment is useful for identifying event types in logs that SIEM cannot recognize.

[0009] Further aspects of this disclosure relate to systems and computer program products having functionality similar to that discussed above regarding computer-implemented methods. This invention is not intended to illustrate every aspect and / or every embodiment of every implementation of this disclosure. Attached Figure Description

[0010] The accompanying drawings included in this application are incorporated in and form a part of the specification. The drawings illustrate embodiments of the present disclosure and, together with the specification, serve to explain the principles of the disclosure. The drawings are merely illustrative of certain embodiments and do not limit the scope of the disclosure.

[0011] Figure 1 This is a block diagram of an example system for identifying SIEM event types according to some embodiments of the present disclosure.

[0012] Figure 2 This is an example system for identifying SIEM event types according to some embodiments of the present disclosure.

[0013] Figure 3This is a flowchart illustrating a method for training a deep learning classification model according to some embodiments of the present disclosure.

[0014] Figure 4 This is a flowchart of a method for identifying SIEM event types according to some embodiments of the present disclosure.

[0015] Figure 5 This is a block diagram of an exemplary SIEM event type identification system according to some embodiments of the present disclosure.

[0016] Figure 6 This is a cloud computing environment based on some embodiments of the present disclosure.

[0017] Figure 7 It is a set of functional abstraction model layers provided by a cloud computing environment according to some embodiments of this disclosure.

[0018] This disclosure may have various modifications and alternatives, the details of which are illustrated by way of example in the accompanying drawings and will be described in detail. However, it should be understood that this disclosure is not intended to limit it to the specific embodiments described. Rather, the invention is intended to cover all modifications, equivalents, and alternatives falling within the scope of this disclosure. Detailed Implementation

[0019] As mentioned earlier, in event standardization and classification, SIEM identifies the source and type of events and passes them to the parser based on the identified type. The parser uses regular expressions (regex) to apply rules to the logs, which determine whether a violation has occurred. However, regular expressions look for exact matches to predefined patterns, meaning that new events or variations of old events may not be properly standardized and classified. These unknown logs lack important pieces of information, such as event names or categories. Because these logs cannot be found in a normal search and cannot trigger relevant rules, they are useless, potentially leading to missed destructive security incidents. In some scenarios, five out of 5% to 20% of events are not properly standardized and classified, meaning that security analysts and other monitoring tools may overlook millions of events.

[0020] Therefore, some embodiments of this disclosure can train machine learning models to identify event types in logs that do not match predefined patterns. Furthermore, when the SIEM encounters an unrecognized event type, it can pass the log to a trained machine learning model that can identify the event type. In this way, the SIEM can assign a corresponding parser, which can determine whether a violation has been generated for the log.

[0021] Figure 1This is a block diagram of an example system 100 for identifying SIEM event types according to some embodiments of the present disclosure. System 100 includes a network 102, a SIEM 104, and a machine learning system 106.

[0022] Network 102 may include one or more computer communication networks. Example network 102 may include the Internet, a local area network (LAN), a wide area network (WAN), a wireless network such as a wireless LAN (WLAN), etc. Network 102 may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. For example, a network adapter card or network interface in each computing / processing device implemented as part of SIEM 104 may receive messages and / or instructions from and / or through network 102, and forward messages and / or instructions for storage or execution (or similar) to the corresponding memory or processor of the respective computing / processing device. Although for illustrative purposes, Figure 1 In this paper, network 102 is depicted as a single entity, but in other examples, network 102 may include multiple private and / or public networks on which components of system 100 can communicate.

[0023] SIEM 104 can be a system of software tools and / or services that combine security information and security incidents (i.e., security incidents) for management. In this way, SIEM can analyze alerts (regarding potential security incidents) generated by computer systems and / or computer networks in real time. SIEM can perform historical analysis of security data using relevant indicators (e.g., alerts) of security incidents with machine-generated data. Using SIEM, security incidents can be investigated on an individual basis. However, using SIEM, it is possible to determine whether a relationship exists between two elements (e.g., computer applications) of past security incidents, based on SIEM's knowledge of these two elements.

[0024] According to some embodiments of the present invention, SIEM 104 may include a log 108, a Data Source Manager (DSM) 110, and an association engine 112. Log 108 may be a record of events occurring on a networked computer system (not shown). Log 108 may include multiple fields arranged in a specific order. These fields may include data used in computer transactions based on event types. Association engine 112 may include a parser 114 that can apply association rules to log 108, which can generate violations targeting potential security incidents.

[0025] Data source manager 110 can identify the event type of each log 108. More specifically, data source manager 110 can map each log 108 to a standard event name, category, and log source type based on a known log pattern defined by a regular expression. However, in some cases, log 108 may not match a known log pattern. For example, a malicious actor might attempt to spoof an event by inserting a poison pill into log 108 to prevent data source manager 110 from identifying the event type. In this case, data source manager 110 can assign parser 114 to log 108 based on the event name, category, and log source type identified by deep learning classification model 116.

[0026] Machine learning system 106 can use log 108 with identified events to train a deep learning classification model 116 to label log 108 with unidentified events. Therefore, deep learning classification model 116 can be a machine learning model that labels log 108 with event types. Machine learning is the process of training a computer algorithm to perform a specific classification. For example, a machine learning algorithm (learner) can be trained to identify categories of objects in a digital photograph, predict future shopping choices of potential customers in a market database, etc. According to some embodiments of this disclosure, deep learning classification model 116 is trained to label log 108 with event types based on fields included in log 108 and the order in which the fields occur. Deep learning is part of a broader family of machine learning methods based on neural networks—algorithms inspired by the human brain—that learn from relatively large amounts of data. Deep learning algorithms repeatedly perform tasks and gradually improve results by using deep layers capable of progressive learning. According to some embodiments of this disclosure, log 108 can be used to train deep learning classification model 116, and data storage manager 110 uses event normalization and classification to successfully identify log 108.

[0027] Figure 2 This is an example system 200 for identifying SIEM event types according to some embodiments of the present disclosure. System 200 includes a log 202, a SIEM 204, a parsed log store 206, a machine learning system 208, a deep learning classification model 210, and a classified log 212. The log 202, SIEM 204, machine learning system 208, and deep learning classification model 210 are respectively similar to reference... Figure 1 The description includes log 108, machine learning system 106, SIEM 104, and deep learning classification model 116. Furthermore, SIEM 204 includes data source manager 214 and association engine 216, which are similar to data source manager 110 and association engine 112.

[0028] The parsed log store 206 may include a set of logs 202 successfully labeled by SIEM 204 based on known log patterns. Therefore, the machine learning system 208 can continuously perform deep learning on the parsed log store 206 to generate a deep learning classification model 210. Lines 218-A and 218-B represent the input to the parsed logs in the parsed log store 206 and the output from the deep learning classification model 210 of the machine learning system 208.

[0029] According to some embodiments of this disclosure, when the data source manager 214 cannot determine the event type, log 202 can be input to the deep learning classification model 210. Lines 220-A, 220-B, and 220-C represent streams of logs that the data source manager 214 cannot recognize. Line 220-A represents input of unparsed logs to the deep learning classification model 210. Therefore, the deep learning classification model 210 can assign probabilities that the unparsed logs may correspond to one or more event types. Furthermore, the deep learning classification model 210 can output classified logs 212. Classified logs 212 may include unparsed logs of event types with the relatively highest probabilities. Additionally, classified logs 212 may include log source types and event names based on the probabilities assigned by the deep learning classification model 210.

[0030] In some embodiments of this disclosure, a predetermined threshold may indicate whether to use the classified logs 212 for parsing. For example, if the deep learning classification model 210 indicates a classification probability of less than a predetermined threshold of 50%, SIEM 204 may not use the classified logs for further parsing. Thus, line 220-B may represent a flow of classified logs 212 back to SIEM 204 for manual processing to determine the appropriate parser. However, if the classification probability meets or exceeds the predetermined threshold, line 220-C may represent a flow of classified logs 212 back to SIEM 204, which may allocate parsers based on classification. Furthermore, SIEM 204 may provide the classified logs 212 to the association engine 216, which may parse the classified logs 212 based on the identified event type.

[0031] Figure 3 This is a flowchart of a method 300 for training a deep learning classification model according to some embodiments of the present disclosure. In some embodiments of the present disclosure, reference is made to... Figure 1 The machine learning system 106 describes executable method 300.

[0032] In operation 302, machine learning system 106 may perform log cleanup on the parsed log. The parsed log may be a log that data source manager 110 can successfully identify based on a known log pattern, such as log 108. Log cleanup may involve filtering out irrelevant information from log 108, such as date, timestamp, filename, Uniform Resource Locator, punctuation, etc. After filtering out such irrelevant information, log 108 may be limited to field names and data within log 108.

[0033] In operation 304, machine learning system 106 can perform log tokenization on the cleaned log. Log tokenization may involve reducing the log to a list of field names and data within the cleaned log.

[0034] In operation 306, the machine learning system 106 may perform log padding on the tokenized logs. For ease of comparison between logs, it is useful to ensure that all fields on the logs have the same field length. Therefore, the machine learning system 106 can identify the field length of the longest token within the tokenized logs and pad each token shorter than the longest field with a character (e.g., a space or zero) so that all tokens across all tokenized logs are of the same length.

[0035] In operation 308, the machine learning system 106 can perform log vectorization on the populated log. Performing classification using text tokens can be challenging for a machine learning system. Therefore, the machine learning system 106 can assign a number to each token appearing in the populated log, such that the same token appearing in different logs has the same number. In this way, the machine learning system can learn to identify event types based on the tokens appearing in the log and the order in which they appear.

[0036] In operation 310, the machine learning system 106 can generate a deep learning classification model 116 by training it using vectorized logs. Because the source logs of the vectorized logs are successfully labeled, the deep learning classification model 116 can use the patterns of the vectorized logs to learn how to identify event types. According to some embodiments of this disclosure, the machine learning system 106 can use a convolutional neural network (CNN) to train the deep learning classification model 116.

[0037] A CNN is a neural network defined in layers, where the first layer recognizes relatively fewer complex features, while later layers can recognize more complex features based on features recognized in earlier layers. An example of such feature detection in a CNN involves detecting faces in an image. In this example, the first layer of the CNN can recognize vertical and horizontal lines within the image. Later layers in the CNN can recognize the nose and mouth based on the lines recognized earlier. According to some embodiments of this disclosure, earlier layers of the CNN can recognize specific tokens. Furthermore, later layers of the CNN can recognize specific token groups and sequences. In this way, a CNN can enable a deep learning classification model 116 to recognize event types in vectorized logs.

[0038] Figure 4 This is a process flowchart of a method 400 for identifying SIEM event types according to some embodiments of this disclosure. In some embodiments of this disclosure, reference is made to... Figure 1 The described data source manager 110 executable method 400.

[0039] In operation 402, data source manager 110 can determine that log 108 cannot be categorized. As previously stated, if log 108 does not match a predefined pattern of a regular expression, data source manager 110 can determine that log 108 cannot be categorized. For example, a malicious actor could insert poison data into log 108 to prevent data source manager 110 from identifying the event type of log 108, and thus prevent parsing that could generate violations for analysis.

[0040] In the event of an operation 404, Data Source Manager 110 can generate vectorized logs using cleaned, tokenized, and populated versions of uncategorized logs. Data Source Manager 110 can then refer to... Figure 3 Operations 302 to 308 described generate vectorized logs.

[0041] Return to reference Figure 4 In operation 406, the data source manager 110 can use a deep learning classification model 116 to classify vectorized logs. Classifying vectorized logs may involve generating one or more latent labels for log source type, event name, and parser, as well as the probability that a latent label is the correct label. Furthermore, the deep learning classification model 116 can therefore provide labels with the highest relative probability. This probability is also referred to herein as the confidence level of the classification.

[0042] In operation 408, the data source manager can determine whether the confidence level of the classification meets a predetermined threshold. The predetermined threshold may indicate that the data source manager 110 can determine that the classification is at the correct confidence level. In some embodiments of this disclosure, the predetermined threshold may be 50%. Therefore, if the confidence level of the classification is 50% or higher, method 400 may proceed to operation 410. If the confidence level of the classification is less than 50%, method 400 may proceed to operation 412.

[0043] In operation 410, the data source manager 110 can parse log 108 based on categories. Parsing log 108 may involve passing logs and categories to the association engine, such as information about... Figure 1 The associated engine described is 112.

[0044] In operation 412, the Data Source Manager can invoke manual processing for parsing logs. Because the confidence level of the classification is below a predetermined threshold, attempting to parse logs based on classification may lead to errors. Therefore, using manual processing to parse logs can be useful.

[0045] Figure 5 This is a block diagram of an example SIEM event type identification system 500 according to some embodiments of the present disclosure. In various embodiments, the SIEM event identification system 500 is similar to the one described above. Figure 1 The data source manager 110 and machine learning system 106 are described and can be executed. Figure 3-4 The methods and / or described in Figure 1 and 2 The functions discussed herein. In some embodiments, the SIEM event recognition system 500 provides instructions to a client machine for the methods and / or functionalities described above, causing the client machine to execute a method or a portion thereof based on the instructions provided by the SIEM event recognition system 500. In some embodiments, the SIEM event recognition system 500 includes software executed on hardware incorporated into multiple devices.

[0046] SIEM event recognition system 500 includes memory 525, storage 530, interconnect (e.g., BUS) 520, one or more CPUs 505 (also referred to herein as processors 505), I / O device interface 510, I / O device 512, and network interface 515.

[0047] Each CPU 505 retrieves and executes programming instructions stored in memory 525 or storage device 530. Interconnect 520 is used to move data, such as programming instructions, between CPU 505, I / O device interface 510, memory 530, network interface 515, and memory 525. Interconnect 520 may be implemented using one or more buses. In various embodiments, CPU 505 may be a single CPU, multiple CPUs, or a single CPU with multiple processing cores. In some embodiments, CPU 505 may be a digital signal processor (DSP). In some embodiments, CPU 505 includes one or more 3D integrated circuits (3DICs) (e.g., 3D wafer-level package (3DWLP), 3D intermediate-based integration, 3D stacked IC (3D-SIC), monolithic 3D IC, 3D heterogeneous integration, 3D system-in-package (3DSiP), and / or package-on-package (PoP) CPU configurations). Memory 525 is typically included to represent random access memory (e.g., static random access memory (SRAM), dynamic random access memory (DRAM), or flash memory). This typically includes memory 530 to represent non-volatile memory, such as hard disk drives, solid-state drives (SSDs), removable memory cards, optical storage, and / or flash memory devices. Additionally, storage device 530 may include storage area network (SAN) devices, cloud devices, or other devices connected to the SIEM event identification system 500 via I / O device interface 510 or to the network 550 via network interface 515.

[0048] In some embodiments, memory 525 stores instructions 560. However, in various embodiments, instructions 560 are partially stored in memory 525 and partially stored in storage device 530, or instructions 560 are entirely stored in memory 525 or entirely stored in storage device 530, or instructions 560 are accessed via network 550 through network interface 515.

[0049] Instruction 560 can be used to execute Figure 1 and 2 The functions and / or discussed in the text Figure 3-4 The processor-executable instructions of any part or all of the methods described herein.

[0050] In various embodiments, I / O device 512 includes an interface capable of presenting information and receiving input. For example, I / O device 512 may present information to a listener interacting with SIEM event recognition system 500 and receive input from the listener.

[0051] The SIEM event recognition system 500 is connected to a network 550 via a network interface 515. The network 550 may be a physical, wireless, cellular, or other network.

[0052] In some embodiments, the SIEM event recognition system 500 may be a multi-user mainframe computer system, a single-user system, or a server computer or similar device that has little or no direct user interface but receives requests from other computer systems (clients). Furthermore, in some embodiments, the SIEM event recognition system 500 may be implemented as a desktop computer, portable computer, laptop or notebook computer, tablet computer, pocket computer, telephone, smartphone, network switch or router, or any other suitable type of electronic device.

[0053] Note Figure 5 This description aims to depict the main representative components of an exemplary SIEM event recognition system 500. However, in some embodiments, the individual components may have more... Figure 5 The greater or lesser complexity represented in the text can exist with... Figure 5 The components shown are different components or as Figure 5 The components shown are alternative components, and the number, type, and configuration of these components can vary.

[0054] Although this disclosure includes a detailed description of cloud computing, the implementation of the teachings recorded herein is not limited to cloud computing environments. Rather, embodiments of this disclosure can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0055] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with service providers. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0056] The characteristics are as follows:

[0057] On-demand self-service: Cloud consumers can unilaterally and automatically provide computing power, such as server time and network storage, as needed, without requiring manual interaction with the service provider.

[0058] Wide area network access: Capabilities are available on the network and accessed through standard mechanisms that facilitate the use of heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0059] Resource sharing: The provider's computing resources are centralized to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. Location independence has significance because consumers typically do not control or know the exact location of the resources provided, but can specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0060] Rapid Flexibility: In some cases, the ability to scale outwards and inwards can be provided quickly and flexibly. The capabilities available to users often appear unlimited and can be purchased in any quantity at any time.

[0061] Measurement services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the service type (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both the providers and consumers of the services being utilized.

[0062] The service model is as follows:

[0063] Software as a Service (SaaS): The capability offered to consumers is the ability to use the provider's applications running on cloud infrastructure. Applications can be accessed from individual client devices via thin client interfaces such as web browsers (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating system, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0064] Platform as a Service (PaaS): This provides consumers with the ability to deploy consumer-created or acquired applications onto cloud infrastructure using programming languages ​​and tools supported by the provider. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they have control over the deployed applications and the configuration of any application hosting environments.

[0065] Infrastructure as a Service (IaaS): This provides consumers with the capability to offer processing, storage, networking, and other basic computing resources that enable consumers to deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but they have control over the operating system, storage, deployed applications, and possibly limited control over selected networking components (e.g., host firewalls).

[0066] The deployment model is as follows:

[0067] Private cloud: Cloud infrastructure operated solely by an organization. It can be managed by the organization or a third party and can exist on-site or off-site.

[0068] Community cloud: Cloud infrastructure shared by several organizations and supporting a specific community with shared concerns (e.g., tasks, security requirements, policies, and compliance considerations). It can be managed by an organization or a third party and can exist on-site or off-site.

[0069] Public cloud: Cloud infrastructure available to the general public or large industrial groups and owned by organizations that sell cloud services.

[0070] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain a single entity but are bound together by standardized or proprietary technologies that enable data and applications to be ported together (e.g., cloud bursting for load balancing between clouds).

[0071] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is the infrastructure of a network of interconnected nodes.

[0072] Figure 6 This is a cloud computing environment 610 according to some embodiments of the present disclosure. As shown, the cloud computing environment 610 includes one or more cloud computing nodes 600. The cloud computing node 600 can perform... Figure 3-4 The methods and / or described in Figure 1 and 2 The functionality discussed above. Additionally, cloud computing nodes 600 can communicate with local computing devices used by cloud consumers, such as personal digital assistants (PDAs) or cellular phones 600A, desktop computers 600B, laptop computers 600C, and / or automotive computer systems 600N. Furthermore, cloud computing nodes 600 can communicate with each other. Cloud computing nodes 600 can also be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds or combinations thereof described above. This allows cloud computing environment 610 to provide infrastructure, platforms, and / or software as services for which cloud consumers do not need to maintain resources on their local computing devices. It should be understood that... Figure 6 The types of computing devices 600A-N shown are for illustrative purposes only, and the computing node 600 and cloud computing environment 610 can communicate with any type of computing device over any type of network and / or network-addressable connection (e.g., using a web browser).

[0073] Figure 7 It is based on some embodiments of this disclosure and is provided by a cloud computing environment 610 ( Figure 6This provides a set of functional abstract model layers. It should be understood beforehand that... Figure 7 The components, layers, and functions shown are for illustrative purposes only, and embodiments of this disclosure are not limited thereto. The following layers and corresponding functions are provided.

[0074] The hardware and software layer 700 includes hardware and software components. Examples of hardware components include: a host 702; a server 704 based on a RISC (Reduced Instruction Set Computer) architecture; a server 706; a blade server 708; a storage device 710; and a network and networking component 712. In some embodiments, the software components include network application server software 714 and database software 716.

[0075] The virtualization layer 720 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 722; virtual storage 724; virtual network 726, including virtual private network; virtual application and operating system 728; and virtual client 730.

[0076] In one example, management layer 740 can provide the functionality described below. Resource provisioning 742 provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and pricing 744 provides cost tracking for resource utilization within the cloud computing environment, as well as billing or pricing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, and protection for data and other resources. User portal 746 provides access to the cloud computing environment for consumers and system administrators. Service level management 748 provides cloud resource allocation and management to meet the required service level. Service level management 748 can allocate appropriate processing power and memory to process static sensor data. Service level agreement (SLA) planning and fulfillment 750 provides pre-scheduling and procurement of cloud resources, where future needs are anticipated according to the SLA.

[0077] The workload layer 760 provides examples of functionalities that can be used in a cloud computing environment. Examples of workloads and functionalities that can be provided from this layer include: mapping and navigation 762; software development and lifecycle management 764; virtual classroom education delivery 766; data analysis and processing 768; transaction processing 770; and SIEM event type recognition system 772.

[0078] This disclosure can be a system, method, and / or computer program product at any possible level of technical detail integration. A computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to execute aspects of this disclosure.

[0079] Computer-readable storage media can be tangible devices capable of retaining and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. Specific examples of computer-readable storage media, in a non-exhaustive list, include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable optical disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or recessed structures with instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0080] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a suitable computing / processing device, or downloaded via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network) to an external computer or external storage device. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the suitable computing / processing device.

[0081] Computer-readable program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages ​​such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through an Internet service provider using an Internet service). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may execute computer-readable program instructions by utilizing the status information of the computer-readable program instructions to personalize the electronic circuitry in order to perform aspects of this disclosure.

[0082] This document describes aspects of the disclosure with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0083] These computer-readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, such that the computer-readable storage medium in which the instructions are stored includes an article of writing comprising instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0084] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus or other equipment to cause a series of operational steps to be performed on the computer, other programmable apparatus or other equipment to produce computer-implemented processing, such that the instructions, which execute on the computer, other programmable apparatus or other equipment, implement the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0085] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a sequence not shown in the figures. For example, two blocks shown consecutively may actually be performed as a single step, simultaneously, substantially simultaneously, or in a manner that partially or entirely overlaps in time within a single step, or sometimes these blocks may be executed in reverse order, depending on the functions involved. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0086] Example 1 is a computer-implemented method. The method includes: determining that event types in the Security Information and Event Management (SIEM) event logs cannot be identified; generating vectorized logs using cleaned, tokenized, and populated versions of the event logs; generating classifications for the vectorized logs using a deep learning classification model trained on multiple parsed logs to identify potential event types for the event logs; determining that the confidence level of the classification meets a predetermined threshold; and parsing the event logs based on the classification.

[0087] Example 2 includes the method of Example 1, with or without optional features. In this example, the deep learning classification model is trained by: cleaning the parsed logs; tokenizing the parsed logs; padding the parsed logs; and vectorizing the parsed logs. Optionally, tokenizing the parsed logs includes: identifying multiple tokens for each log in the parsed logs; and generating a log containing the tokens. Optionally, padding the parsed logs includes: identifying the length of the longest token among the tokens; and increasing the original length of each token to the length of the longest token using a predetermined character. Optionally, vectorizing the parsed logs includes: assigning a unique value to each token among the tokens; and replacing the token with the assigned unique value.

[0088] Example 3 includes the method of any of Examples 1 to 2, with or without optional features. In this example, a convolutional neural network is used to train a deep learning classification model.

[0089] Example 4 includes a method from any of Examples 1 through 3, with or without optional features. In this example, the parsed logs include multiple logs categorized by the SIEM's data source manager and the event type based on that category.

[0090] Example 5 is a computer program product including program instructions stored on a computer-readable storage medium. The computer-readable medium includes instructions that instruct a processor to determine that an event type in an event log of a Security Information and Event Management (SIEM) is unrecognizable; to generate a vectorized log using a cleaned, tokenized, and populated version of the event log; to generate a classification for the vectorized log using a deep learning classification model trained to identify potential event types in the event log using multiple parsed logs trained on deep learning; to determine that the confidence level of the classification meets a predetermined threshold; and to parse the event log based on the classification.

[0091] Example 6 includes the computer-readable medium of Example 5, which may or may not include optional features. In this example, the deep learning classification model is trained by: cleaning the parsed log; tokenizing the parsed log; padding the parsed log; and vectorizing the parsed log. Optionally, tokenizing the parsed log includes: identifying multiple tokens for each log in the parsed log; and generating a log including the tokens. Optionally, padding the parsed log includes: identifying the length of the longest token among the tokens; and increasing the original length of each token to the length of the longest token using a predetermined character. Optionally, vectorizing the parsed log includes: assigning a unique value to each token among the tokens; and replacing the token with the assigned unique value.

[0092] Example 7 includes a computer-readable medium of any of Examples 5 to 6, which may or may not include optional features. In this example, a convolutional neural network is used to train a deep learning classification model.

[0093] Example 8 includes a computer-readable medium of any of Examples 5 through 7, which may or may not include optional features. In this example, the parsed logs include multiple logs categorized by the SIEM's data source manager and event types based on that category.

[0094] Example 9 is a system. The system includes instructions that direct a processor to computer processing circuitry; and a computer-readable storage medium storing the instructions, which, when executed by the computer processing circuitry, are configured to cause the computer processing circuitry to perform a method comprising: determining that event types in an event log of a Security Information and Event Management (SIEM) are unrecognizable; generating vectorized logs using cleaned, tokenized, and populated versions of the event logs; generating a classification of the vectorized logs using a deep learning classification model trained to identify potential event types in the event logs using multiple parsed logs trained on a deep learning basis; determining that the confidence level of the classification meets a predetermined threshold; and parsing the event logs based on the classification.

[0095] Example 10 includes the system of Example 9, which may or may not include optional features. In this example, the deep learning classification model is trained by: cleaning the parsed logs; tokenizing the parsed logs; padding the parsed logs; and vectorizing the parsed logs. Optionally, tokenizing the parsed logs includes: identifying multiple tokens for each log in the parsed logs; and generating a log including the tokens. Optionally, padding the parsed logs includes: identifying the length of the longest token among the tokens; and increasing the original length of each token to the length of the longest token using a predetermined character. Optionally, vectorizing the parsed logs includes: assigning a unique value to each token among the tokens; and replacing the token with the assigned unique value.

[0096] Example 11 includes a system from any of Examples 9 to 10, with or without optional features. In this example, a convolutional neural network is used to train a deep learning classification model.

[0097] Example 12 is a computer-implemented method. The method includes instructions that guide a processor to generate a vectorized log using a cleaned, tokenized, and populated version of an event log; to generate a classification for the vectorized log using a deep learning classification model trained to identify potential event types in the event log using multiple parsed logs trained on deep learning; to determine if the confidence level of the classification meets a predetermined threshold; and to parse the event log based on the classification.

[0098] Example 13 includes the method of Example 12, with or without optional features. In this example, a convolutional neural network is used to train a deep learning classification model. Optionally, the method includes determining if the event type in the Security Information and Event Management (SIEM) event logs is unrecognizable.

[0099] Example 14 is a computer-implemented method. The method includes instructions that guide a processor to determine if an event type in a Security Information and Event Management (SIEM) event log is unrecognizable; generate a vectorized log using a cleaned, tokenized, and populated version of the event log; generate a classification for the vectorized log using a deep learning classification model trained to identify potential event types in the event log using multiple parsed logs trained on deep learning; determine if the confidence level of the classification meets a predetermined threshold; and parse the event log based on the classification.

[0100] Example 15 includes the method of Example 14, with or without optional features. In this example, the method includes identifying multiple event types corresponding to the parsed logs; and training a deep learning classification model.

Claims

1. A computer-implemented method, comprising: The event types in the event logs for security information and event management cannot be identified based on regular expressions; Use the cleaned, tokenized, and populated version of the event log to generate a vectorized log; A deep learning classification model is used to generate the vectorized log classification. The deep learning classification model is trained based on deep learning training using multiple parsed logs to identify the potential event types of the event logs. The classification includes event type and log source type. Determine that the confidence level of the classification meets a predetermined threshold; and Based on the classification, the data source manager for security information and event management assigns a corresponding parser to parse the event logs.

2. The method according to claim 1, wherein, The deep learning classification model is trained as follows: Clean up the parsed logs; Tokenize the parsed log; Populate the parsed log; and Vectorize the parsed log.

3. The method according to claim 2, wherein, The tokenized parsed logs include: Identify multiple tokens for each log entry in the parsed logs; and Generate a log that includes the token.

4. The method according to claim 2, wherein, The logs that are populated during the parsing include: Identify the length of the longest token among multiple tokens; and The original length of each of the tokens is increased to the length of the longest token using a predetermined character.

5. The method according to claim 2, wherein, The vectorized parsed log includes: Assign a unique value to each of the tokens; and Replace the token with the unique value assigned.

6. The method according to claim 1, further comprising: Identify multiple event types corresponding to the parsed logs; as well as A deep learning classification model is trained based on the identified event types.

7. The method according to claim 1, wherein, The deep learning classification model is trained using a convolutional neural network.

8. The method according to claim 1, wherein, The parsed logs include multiple logs categorized by SIEM's data source manager and by event types based on the classification.

9. A computer program product comprising program instructions executable by a processor to cause the processor to perform a method, the method comprising: The event types in the event logs for security information and event management cannot be identified based on regular expressions; Use the cleaned, tokenized, and populated version of the event log to generate a vectorized log; A deep learning classification model is used to generate the vectorized log classification. The deep learning classification model is trained based on deep learning training using multiple parsed logs to identify the potential event types of the event logs. The classification includes event type and log source type. Determine that the confidence level of the classification meets a predetermined threshold; and Based on the classification, the data source manager for security information and event management assigns a corresponding parser to parse the event logs.

10. The computer program product according to claim 9, wherein, The deep learning classification model is trained as follows: Clean up the parsed logs; Tokenize the parsed log; Populate the parsed log; and Vectorize the parsed log.

11. The computer program product according to claim 10, wherein, The tokenized parsed logs include: Identify multiple tokens for each log entry in the parsed logs; and Generate a log that includes the token.

12. The computer program product according to claim 10, wherein, The logs that are populated during the parsing include: Identify the length of the longest token among multiple tokens; and The original length of each of the tokens is increased to the length of the longest token using a predetermined character.

13. The computer program product according to claim 10, wherein, The vectorized parsed log includes: Assign a unique value to each of the tokens; and Replace the token with the unique value assigned.

14. The computer program product according to claim 9, wherein, The deep learning classification model is trained using a convolutional neural network.

15. The computer program product according to claim 9, wherein, The parsed logs include multiple logs categorized by SIEM's data source manager and by event types based on the classification.

16. A computer system, comprising: Computer processing circuitry; as well as A computer-readable storage medium storing instructions, wherein, when executed by said computer processing circuitry, said instructions are configured to cause said computer processing circuitry to perform a method comprising: The event types in the event logs for security information and event management cannot be identified based on regular expressions; Use the cleaned, tokenized, and populated versions of the event log to generate vectorized logs; A deep learning classification model is used to generate the vectorized log classification. The deep learning classification model is trained based on deep learning training using multiple parsed logs to identify the potential event types of the event logs. The classification includes event type and log source type. Determine that the confidence level of the classification meets a predetermined threshold; and Based on the classification, the data source manager for security information and event management assigns a corresponding parser to parse the event logs.

17. The computer system according to claim 16, wherein, The deep learning classification model is trained as follows: Clean up the parsed logs; Tokenize the parsed log; Populate the parsed log; and Vectorize the parsed log.

18. The computer system according to claim 17, wherein, The tokenized parsed logs include: Identify multiple tokens for each log entry in the parsed logs; and Generate a log that includes the token.

19. The computer system according to claim 17, wherein, The logs that are populated during the parsing include: Identify the length of the longest token among multiple tokens; and The original length of each of the tokens is increased to the length of the longest token using a predetermined character.

20. The computer system according to claim 17, wherein, The vectorized parsed log includes: Assign a unique value to each of the tokens; and Replace the token with the unique value assigned.

21. The computer system according to claim 16, wherein, The deep learning classification model is trained using a convolutional neural network.

Images