Log processing method and system and related product

Abstract

The embodiment of the invention discloses a log processing method and system and related products, and the method comprises the steps: receiving logs sent by all equipment ends, and classifying the logs to determine the log types of the logs; extracting a target keyword from the log according to the log type of the log; identifying a target attack type described by the log based on a target keyword in the log; and inputting the log and the log type of the log into a detection engine corresponding to the target attack type of the log. Through the target keyword of the log, the attack type of network attack described by the log can be deeply understood, the logs of the same attack type can be uniformly input into the same type of detection engine for further detection, and the workload of specially developing each detection code for each log is reduced. The detection engine can further detect whether the log truly describes the network attack, so that the detection result of the log is more refined and credible, the potential network attack can be accurately identified and prevented, and the difficulty of safe operation is reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of information security technology, and in particular to a log processing method, system, and related products. Background Art

[0002] Information security is an important issue in all walks of life. Logs, as important files that record the operation process and abnormal information of the system, are increasingly valued by relevant personnel and can provide detailed information for quickly locating threat problems that occur during system operation.

[0003] Nowadays, existing solutions often use Security Information and Event Management (SIEM) technology or Security Operations Center (SOC) technology to manage logs. However, after traditional SIEM and SOC access logs, they mainly perform statistical dimension correlation analysis on logs, such as parsing out the source IP, destination IP and other fields of the logs, and then converting these fields into preset data structures and storing them in the database.

[0004] It can be seen that traditional SIEM and SOC focus more on simple management such as log screening and rough analysis, and do not deeply analyze the accessed logs. The log descriptions of network attacks by security devices from different manufacturers are often different, which makes it easy for existing solutions to ignore and let go of potential network attacks, causing information leakage. Summary of the invention

[0005] The embodiments of the present application provide a log processing method, system and related products for accurately identifying potential network attacks in each log in a unified and indiscriminate manner for massive logs sent from different device ends, thereby ensuring information security at a low cost.

[0006] A first aspect of an embodiment of the present application provides a log processing method, including:

[0007] receiving logs sent by each device end, and classifying the logs to determine log types of the logs; at least some of the logs have different formats;

[0008] extracting target keywords from the log according to the log type of the log;

[0009] Based on the target keyword in the log, identifying the target attack type described in the log;

[0010] The log and the log type of the log are input into a detection engine corresponding to the target attack type of the log to detect whether the log describes a network attack.

[0011] Optionally, the identifying the target attack type described in the log based on the target keyword in the log includes:

[0012] Find the attack type corresponding to the target keyword from the pre-built mapping relationship between keywords and attack types;

[0013] The target attack type is determined based on the found attack type.

[0014] Optionally, determining the target attack type based on the found attack type includes:

[0015] Based on the keyword matching degree and the context similarity, the confidence of the found attack type is calculated as the target attack type; wherein the keyword matching degree includes the coverage rate between the target keyword and the known keyword library of the found attack type; the context similarity includes the similarity between the context information of the target keyword in the log and the context template of the found attack type;

[0016] If the confidence level is greater than or equal to the confidence level threshold, the found attack type is used as the target attack type.

[0017] Optionally, after calculating the confidence of the found attack type as the target attack type based on the keyword matching degree and the context similarity, the method further includes:

[0018] If the confidence is less than the confidence threshold, the log is input into a pre-trained recognition model, and the attack type output by the recognition model is used as the target attack type; the recognition model is obtained by training logs of known attack types.

[0019] Optionally, before inputting the log and the log type of the log into a detection engine corresponding to the target attack type of the log, the method further includes:

[0020] Adding a corresponding information field to the log to obtain an enriched new log; the information field includes at least one of the target keyword of the log, the target attack type of the log, the event asset attribute of the log, and the event intelligence attribute;

[0021] The step of inputting the log and the log type of the log into a detection engine corresponding to a target attack type of the log includes:

[0022] The new log and the log type of the new log are input into a detection engine corresponding to the target attack type of the new log.

[0023] Optionally, extracting target keywords from the log according to the log type of the log includes:

[0024] If the log type is a network security log recording network data packet information, according to the protocol rules adopted by the network data packet, the components of the data access request are extracted from the network data packet as the target keywords of the log.

[0025] Optionally, if it is detected that the log describes a network attack, the method further includes:

[0026] Perform security incident protection on the target of the network attack, and / or initiate an early warning prompt.

[0027] When the method described in the first aspect of the present application is implemented, the content described in the second aspect of the present application can be adopted to achieve it.

[0028] A second aspect of an embodiment of the present application provides a log processing system, including: a receiving unit and a processing unit;

[0029] The receiving unit is used to receive logs sent by each device end, and classify the logs to determine the log types of the logs; at least some of the logs have different formats;

[0030] The processing unit is used to extract target keywords from the log according to the log type of the log;

[0031] The processing unit is further configured to, based on the target keyword in the log, identify the target attack type described in the log;

[0032] The processing unit is further configured to input the log and the log type of the log into a detection engine corresponding to a target attack type of the log, so as to detect whether the log describes a network attack.

[0033] A third aspect of an embodiment of the present application provides an electronic device, including: a processor and a memory;

[0034] The processor is configured to communicate with the memory and execute instructions in the memory to implement the method described in the first aspect of the embodiment of the present application or any specific implementation of the first aspect.

[0035] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed by a processor, they implement the method described in the first aspect of the embodiment of the present application or any specific implementation of the first aspect.

[0036] A fifth aspect of an embodiment of the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed by a processor, they implement the method described in the first aspect of the embodiment of the present application or any specific implementation of the first aspect.

[0037] It can be seen from the above technical solutions that the embodiments of the present application have at least the following advantages:

[0038] The embodiment of the present application can deeply understand what type of network attack each log describes through the target keywords of each type of log, which also helps to uniformly input logs of the same attack type into the same type of detection engine for further detection, reducing the workload of developing each detection code specifically for each log. Among them, the use of the detection engine can further detect whether the log really describes a network attack, making the detection results of the log more refined and reliable, so as to accurately identify and prevent potential network attacks, ensure the security of information assets, and at the same time, reduce the possibility of false positive attacks and reduce the difficulty of security operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0040] It should be noted that, although the steps in the process diagrams (if any) involved in the embodiments are drawn in sequence as indicated by the arrows, unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

[0041] Figure 1 A schematic diagram of a system architecture of the log processing method according to an embodiment of the present application;

[0042] Figure 2 A flow chart of the log processing method according to an embodiment of the present application;

[0043] Figure 3 A structural diagram of a log processing method system according to an embodiment of the present application;

[0044] Figure 4A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0046] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims and drawings of the present application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0047] In the following description, similar expressions such as "a specific implementation" or "a specific example" are involved, which describe a subset of all possible embodiments, but it can be understood that "a specific implementation" or "a specific example" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict. In the following description, the term multiple refers to at least two. In some specific examples, the numerical value mentioned in this application reaches the threshold value (if any), which may include the case where the former is greater than the latter of the threshold value; if similar expressions such as "any" or "at least one" are mentioned, it can specifically refer to any one of the listed examples or any combination of these examples.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0049] See also Figure 1 , Figure 1 Schematic diagram of the system architecture provided by the embodiment of the present application. Figure 1As shown, the system architecture may include a business server 100 and a terminal cluster, and the terminal cluster (or device end) may include: terminal devices 200a, terminal device 200b, terminal device 200c, ..., terminal device 200n and other terminal devices. Among them, the above-mentioned business server 100 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud database, cloud service, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform. The terminal device (including terminal device 200a, terminal device 200b, terminal device 200c, ..., terminal device 200n) may be a smart phone, tablet computer, laptop computer, desktop computer, PDA, mobile internet device (mobile internet device, MID), wearable device (such as smart watch, smart bracelet, etc.), smart computer, smart car and other smart terminals. Among them, the business server 100 can establish a communication connection with each terminal device in the terminal cluster, and a communication connection can also be established between each terminal device in the terminal cluster. In other words, the business server 100 can establish a communication connection with each terminal device in the terminal device 200a, terminal device 200b, terminal device 200c, ..., terminal device 200n. For example, a communication connection can be established between the terminal device 200a and the business server 100. A communication connection can be established between the terminal device 200a and the terminal device 200b, and a communication connection can also be established between the terminal device 200a and the terminal device 200c. The above-mentioned communication connection does not limit the connection method, and can be directly or indirectly connected through a wired communication method, or directly or indirectly connected through a wireless communication method, etc., which can be determined according to the actual application scenario, and this application is not limited here.

[0050] In an embodiment of the present application, the business server 100 can receive logs sent by each device end, and classify the logs to determine the log type of the log; thereafter, the business server 100 can extract target keywords from the log according to the log type of the log, and identify the target attack type described by the log based on the target keywords in the log; the business server 100 can input the log and the log type of the log into a detection engine corresponding to the target attack type of the log, so that the detection engine can accurately detect whether the log describes a network attack in a targeted manner, thereby solving the problem of difficulty in uniformly analyzing logs from device ends of various manufacturers, reducing the difficulty of security operations, and ensuring information security.

[0051] The method of the present application will be described in further detail below.

[0052] See also Figure 2In a first aspect, the present application provides a specific embodiment of a log processing method, which includes the following steps:

[0053] Step S21, receiving logs sent by each device end, and classifying the logs to determine the log type of the log;

[0054] In actual situations, the log descriptions of the same network attack by the devices (security devices) of different manufacturers are often different, and these differences often include differences in format, which can be at least one of the following: timestamp format, log format, numerical unit, log threat level, source information, and encoding method. For example, the a, b, and c devices may use JSON, XML, and CSV formats to write logs, respectively, and their character encoding methods may be UTF-8, GBK, and ASCII encoding methods, respectively; of course, there may also be other inconsistencies, resulting in different log descriptions of the same network attack by each device. Therefore, in order to ensure that the accessed logs can be processed in a normalized manner, for example, to ensure that the logs describing the same attack type can be uniformly accessed to the same set of detection engines for in-depth detection, each log can be analyzed in a standardized manner based on the log type, so as to avoid writing a set of detection codes for each massive log, which increases the detection workload.

[0055] For example, a manufacturer's log may contain several types of logs, such as operation and maintenance logs, security logs, system logs, access logs, etc. Among them, the operation and maintenance logs can record the operating status and fault information of the system or network; the security logs can record security-related events, such as intrusion detection, malware detection, etc. By performing operations such as field learning and content learning, the logs can be classified. The fields learned here can include fields such as log format; the content learning operation here can include: text preprocessing of log messages, including removing irrelevant characters and stop words; extracting features from the preprocessed log messages; performing content analysis on the extracted features to understand the subject, intent, and other information about the log type of the log message. In some instances, after field learning and content learning, the trained classification model can be applied to each log to be classified, so that the classification model predicts the log type of each accessed log.

[0056] Step S22: extracting target keywords from the log according to the log type of the log;

[0057] Extracting target keywords from the log according to the log type of the log helps to better understand and analyze the log data and improve the efficiency and accuracy of log management. In specific operations, step S22 may include: part-of-speech (such as noun or verb) analysis, segmenting the log content according to the part-of-speech analysis results, extracting target keywords from each segmented word, etc. In actual situations, the target keyword can be called key evidentiary information because it can be used to determine whether the current log really describes a network attack.

[0058] Each type of log has its specific content and structure. For example, in an operation and maintenance log, the target keyword may include at least one of system crash, network failure, hardware failure, software error, service interruption, etc. If a line in an operation and maintenance log shows "system crash, error code: 001", then its target keywords may be "system crash" and "error code"; in a security log, the target keyword may include at least one of intrusion attempt, malware, security vulnerability, attack behavior, authentication failure, etc. If a line in a security log shows "intrusion attempt detected, source IP: 192.168.1.1", then its target keywords may be "intrusion attempt" and "source IP".

[0059] In actual operation, you can use a text editor or command line tool to extract the target keywords. For example, you can use the grep command line tool to search for lines containing specific keywords in the log file, and further analyze the target keywords through these lines.

[0060] Step S23: Based on the target keywords in the log, identify the target attack type described in the log;

[0061] In the process of understanding logs, the more detailed the mapping (or identification) of the log fields is, the deeper and more accurate the understanding of the logs will be. Therefore, the field of the attack type described in the log can be mapped.

[0062] For example, if the log description contains target keywords such as "SQL error" and "injection", then the target attack type described by the log is likely to be a SQL injection attack; if the log shows that login attempts from the same IP increase rapidly within a few minutes, and most attempts use the same username but different passwords, which is consistent with the frequency and pattern characteristics of a brute force attack, then the target attack type described by the log is likely to be a brute force attack.

[0063] For another example, if the log description generated by manufacturer A for the Ice Scorpion webshell detection is: Ice Scorpion webshell communication, and the log description generated by manufacturer B for the Ice Scorpion webshell detection is: Hacker tool Ice Scorpion web backdoor, then according to the operations of steps S21 to S23 above, it can be understood in the embodiment of the present application that the target keywords of these two logs contain "Ice Scorpion web", and it can be clearly seen that they are both detection logs about the attack type "Ice Scorpion webshell".

[0064] Step S24: input the log and the log type of the log into a detection engine corresponding to the target attack type of the log to detect whether the log describes a network attack.

[0065] For example, detection engines corresponding to various attack types can be built into the background of Extended Detection and Response (XDR), so that after the target attack type of the current log is clarified, the current log is specifically input into the detection engine corresponding to its attack type, and the detection engine further accurately identifies whether the current log really describes a network attack, so as to avoid misreporting as a network attack and wasting resources. For example, the detection log with the above-mentioned attack type of "Ice Scorpion webshell" can be input into the webshell detection engine for further detection; the log with the attack type of "Apache Log4j2 remote code execution vulnerability" can be input into the log4j vulnerability detection engine.

[0066] It should be noted that the above detection engine can also be used to identify whether the network attack described in the current log is successful. The above XDR is a system based on the Software as a Service (SaaS) model that aggregates multi-source security telemetry data and organically integrates the previously dispersed single-point security capabilities in a native way to enhance threat detection, investigation, response and hunting capabilities; this XDR system can be used to detect and deal with network security risks.

[0067] In summary, the embodiments of the present application can deeply understand what type of network attack each log describes through the target keywords of various logs, which also helps to uniformly input logs of the same attack type into the same type of detection engine for further detection, reducing the workload of developing each detection code specifically for each log. Among them, the use of the detection engine can further detect whether the log really describes a network attack, making the detection results of the log more refined and reliable, so as to accurately identify and prevent potential network attacks, ensure the security of information assets, and at the same time, reduce the possibility of false positive attacks and reduce the difficulty of security operations.

[0068] Based on the above example descriptions, some specific possible implementation examples will be provided below. In actual applications, the implementation contents between these examples can be combined or implemented separately as needed according to the corresponding functional principles and application logic. If implemented in combination, the execution order between the combined examples can be determined according to their respective processing logics, which can be determined by the actual scenario.

[0069] based on Figure 2 In some specific examples, the specific implementation operation of step S23 includes: searching for the attack type corresponding to the target keyword from the pre-constructed mapping relationship between the keyword and the attack type; and determining the target attack type based on the found attack type.

[0070] Exemplarily, a mapping relationship table may be pre-constructed, and the mapping relationship table records the mapping relationship between keywords and attack types. The mapping relationship may be statistically generated based on historical data, expert knowledge, or machine learning algorithms. Specifically, after the target keyword of the current log is identified, the attack type associated with the target keyword may be found based on the mapping relationship table. As a possible implementation method, at least one of the found attack types may be directly determined as the target attack type of the current log.

[0071] As another possible implementation, the specific implementation operation of the above-mentioned "determining the target attack type based on the found attack type" may include: calculating the confidence of the found attack type as the target attack type based on keyword matching and context similarity; wherein the keyword matching includes the coverage between the target keyword and the known keyword library of the found attack type; the context similarity includes the similarity between the context information of the target keyword in the log and the context template of the found attack type; if the confidence is greater than or equal to the confidence threshold, the found attack type is used as the target attack type.

[0072] For example, the target keyword that can be extracted from the current log is "SQL injection". If the known keyword library for SQL injection attacks (or the keywords in the mapping relationship) includes "SQL", "injection", "XSS", etc., the coverage rate calculated at this time can be calculated as: Coverage rate = number of overlapping keywords ÷ total number of keywords in the corresponding known keyword library = 2 / 3. This coverage rate is generally less than 1. Of course, the total number of keywords in the known keyword library can set a maximum threshold.

[0073] The above-mentioned "XSS" is the abbreviation of Cross-Site Scripting, which is a code injection attack. By injecting XSS malicious scripts into the website, the attacker can make it run on the user's browser, thereby stealing the user's information such as cookies, etc. Of course, in some examples, in addition to the coverage between the target keyword and the known keywords, the above-mentioned keyword matching degree may also include the similarity between the target keyword and the known keywords (such as cosine similarity). In short, the above-mentioned keyword matching degree can be the weighted summation result between this coverage and this similarity. The weight value (greater than or equal to 0) used for weighting can be determined according to the situation and is not limited here.

[0074] On the other hand, context templates related to the found attack type (such as SQL injection attack) can be obtained. For example, the predefined context template for "SQL injection attack" may contain typical scenarios or behavior descriptions of the attack. Furthermore, a text similarity algorithm can be used to calculate the similarity between the context information of the target keyword in the log and the context template. This similarity can be cosine similarity and / or Jaccard similarity, etc.

[0075] Furthermore, the keyword matching degree and context similarity can be combined (such as weighted summation) to calculate a comprehensive confidence. The calculated confidence is compared with a preset confidence threshold. If the confidence is greater than or equal to the confidence threshold, the corresponding attack type found can be considered reliable enough and can be used as the target attack type of the current log.

[0076] based on Figure 2 In some specific examples, after calculating the confidence of the found attack type as the target attack type based on keyword matching and context similarity, the method of the embodiment of the present application also includes: if the confidence is less than the confidence threshold, inputting the log into a pre-trained recognition model, and using the attack type output by the recognition model as the target attack type; the recognition model is obtained by training logs of known attack types.

[0077] In other words, in addition to the above-mentioned method of using mapping relationships to determine the target attack type, a recognition model (or natural language processing model) can also be used to predict the target attack type of the log. This is equivalent to giving logs with confidence levels lower than the confidence threshold another chance to detect, so as to avoid detection errors and security incidents.

[0078] In order to understand the description of the log more deeply and accurately, the log can be enriched to enhance the ability to understand the content of the log. Figure 2In some specific examples, before step S24, the method of the embodiment of the present application may further include: adding corresponding information fields to the log to obtain an enriched new log; the information field includes at least one of the target keyword of the log, the target attack type of the log, the event asset attribute of the log, and the event intelligence attribute.

[0079] In log management, the above-mentioned event asset attributes usually refer to: information about the system or resources related to the log event, such as server IP, host name, application name, service port, etc. This information is crucial for locating problems and tracking the source of events. They provide the physical or logical location where the event occurred. The above-mentioned intelligence attributes: focus more on the field of security analysis, and refer to security context information related to log events, such as user ID, user role, access rights, operation results (success or failure), timestamp, etc. Correspondingly, the specific implementation operation of step S24 may include: inputting the enriched new log and the log type of the new log into the detection engine corresponding to the target attack type of the new log. Generally, the log type before and after enrichment remains unchanged.

[0080] based on Figure 2 In some specific examples, the specific implementation operations of step S22 may include: if the log type is a network security log that records network data packet information, according to the protocol rules adopted by the network data packet, extracting the component content of the data access request from the network data packet as the target keyword of the log.

[0081] For example, if the current log is a network security log that records HTTP protocol network data packet information, at least one of the request method (such as GET or POST), request URI, HTTP protocol version (such as HTTP / 1.1), request header, and request body (including actual transmitted data) can be found in this network data packet as the target keywords of the log. These target keywords have important reference value for subsequent security analysis and incident response.

[0082] based on Figure 2 In some specific examples, after step S24, the method of the embodiment of the present application may further include: performing security incident protection on the attack target of the network attack, and / or initiating an early warning prompt.

[0083] The above security events may include at least one of: opening the firewall function of the security protection software, limiting the access speed, performing correlation analysis and monitoring on the connection end of the attack target, blocking the attack source, etc. Generally, if the current network attack has or will affect the customer's business, a security event can be generated; if there is no harm to the customer, for example, the attack is not successful, or it is just a scan and there is no harm, the security event can be generated by default. The specific situation can be customized and is not limited here. On the other hand, the above warning prompts may include at least one of: prompting to change the password, shutting down unnecessary services, etc.

[0084] Of course, in some specific examples, for logs that are falsely reported as attacks or non-attacks, the method of the embodiment of the present application can also choose to eliminate these logs to avoid occupying resources, etc.

[0085] In summary, the method of the embodiment of the present application provides a unified security detection solution for the differential logs sent by each device end, such as the operation solution of steps S21 to S24, so that the security logs generated by various security devices can be automatically and uniformly connected to a corresponding set of detection engines for in-depth detection, thereby specifically ensuring the reliability of the detection results. At the same time, it helps to reduce the workload of writing detection codes for massive logs separately and improve detection efficiency.

[0086] Compared to Figure 2 The examples shown illustrate that the above-mentioned additional or detailed example contents or possible implementation methods do not necessarily have to be executed during the specific implementation. If more than two examples or possible implementation methods are added, these examples or possible implementation methods can be implemented in combination or separately. If implemented in combination, the execution order between the combined examples can be determined according to their respective processing logic, which can be determined according to the actual scenario.

[0087] See also Figure 3 , the second aspect of the present application provides a specific embodiment of a log processing system, the system comprising: a receiving unit 301 and a processing unit 302;

[0088] The receiving unit 301 is used to receive logs sent by each device end, and classify the logs to determine the log types of the logs; at least some of the logs have different formats;

[0089] The processing unit 302 is used to extract target keywords from the log according to the log type of the log;

[0090] The processing unit 302 is further configured to identify the target attack type described in the log based on the target keyword in the log;

[0091] The processing unit 302 is further configured to input the log and the log type of the log into a detection engine corresponding to the target attack type of the log, so as to detect whether the log describes a network attack.

[0092] In some specific examples, the processing unit 302 is specifically configured to:

[0093] Find the attack type corresponding to the target keyword from the pre-built mapping relationship between keywords and attack types;

[0094] The target attack type is determined based on the found attack types.

[0095] In some specific examples, the processing unit 302 is specifically configured to:

[0096] Based on the keyword matching and context similarity, the confidence of the found attack type as the target attack type is calculated; wherein the keyword matching includes the coverage between the target keyword and the known keyword library of the found attack type; the context similarity includes the similarity between the context information of the target keyword in the log and the context template of the found attack type;

[0097] If the confidence is greater than or equal to the confidence threshold, the found attack type is used as the target attack type.

[0098] In some specific examples, the processing unit 302 is further configured to:

[0099] If the confidence is less than the confidence threshold, the log is input into a pre-trained recognition model, and the attack type output by the recognition model is used as the target attack type; the recognition model is trained by logs of known attack types.

[0100] In some specific examples, the processing unit 302 is further configured to:

[0101] Adding corresponding information fields to the log to obtain an enriched new log; the information field includes at least one of a target keyword of the log, a target attack type of the log, an event asset attribute of the log, and an event intelligence attribute;

[0102] The new log and the log type of the new log are input into the detection engine corresponding to the target attack type of the new log.

[0103] In some specific examples, the processing unit 302 is specifically configured to:

[0104] If the log type is a network security log that records network data packet information, according to the protocol rules adopted by the network data packet, the components of the data access request are extracted from the network data packet as the target keyword of the log.

[0105] In some specific examples, the processing unit 302 is further configured to:

[0106] If a network attack is detected in the log description, security incident protection is performed on the target of the network attack, and / or an early warning is issued.

[0107] In the embodiment of the present application, the operations performed by each unit of the log processing system are similar to the operations described in the first aspect or any specific method embodiment of the first aspect, and will not be described in detail here. Of course, the specific implementation process of each operation in the first aspect of the present application can also refer to the relevant description of the second aspect.

[0108] See also Figure 4 The electronic device 400 of the embodiment of the present application may include one or more processors (such as central processing units (CPU)) 401 and a memory 405, in which one or more applications or data are stored.

[0109] The memory 405 may be a volatile storage or a persistent storage. The program stored in the memory 405 may include one or more modules, each of which may include a series of instruction operations in the electronic device. Furthermore, the processor 401 may be configured to communicate with the memory 405 and execute a series of instruction operations in the memory 405 on the electronic device 400.

[0110] The electronic device 400 may also include one or more power supplies 402, one or more wired or wireless network interfaces 403, one or more input and output interfaces 404, and / or one or more operating systems, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0111] The processor 401 can execute the operations performed by the aforementioned first aspect or any specific method embodiment of the first aspect, and the details will not be repeated here.

[0112] The present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enable the computer to execute the method described in the first aspect or any specific implementation of the first aspect.

[0113] The present application provides a computer program product comprising instructions or a computer program. When the computer program product is run on a computer, the computer is enabled to execute the method described in the first aspect or any specific implementation of the first aspect.

[0114] It is understood that in various embodiments of the present application, the sequence number of each step does not mean the order of execution, and the execution order of each step should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The operation content added or refined in each example scheme of the above method, system or device (if any) does not necessarily have to be executed in the specific implementation. If more than two operations are added, these operations can be implemented in combination or separately, depending on the actual scenario.

[0115] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system (if any) and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0116] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system or device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0117] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0118] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0119] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product (or computer program product) is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a business server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A log processing method, characterized in that: include: Receiving logs sent by each device end, and classifying the logs to determine the log types of the logs; The formats of at least some of said logs differ; extracting target keywords from the log according to the log type of the log; Based on the target keyword in the log, identifying the target attack type described in the log; The log and the log type of the log are input into a detection engine corresponding to the target attack type of the log to detect whether the log describes a network attack.

2. The log processing method according to claim 1, characterized in that: The identifying the target attack type described in the log based on the target keyword in the log includes: Find the attack type corresponding to the target keyword from the pre-built mapping relationship between keywords and attack types; The target attack type is determined based on the found attack type.

3. The log processing method according to claim 2, characterized in that: The determining the target attack type based on the found attack type includes: Based on the keyword matching degree and the context similarity, the confidence of the found attack type is calculated as the target attack type; wherein the keyword matching degree includes the coverage rate between the target keyword and the known keyword library of the found attack type; the context similarity includes the similarity between the context information of the target keyword in the log and the context template of the found attack type; If the confidence level is greater than or equal to the confidence level threshold, the found attack type is used as the target attack type.

4. The log processing method according to claim 3, characterized in that: After calculating the confidence of the found attack type as the target attack type based on the keyword matching degree and the context similarity, the method further includes: If the confidence is less than the confidence threshold, the log is input into a pre-trained recognition model, and the attack type output by the recognition model is used as the target attack type; the recognition model is obtained by training logs of known attack types.

5. The log processing method according to claim 1, characterized in that: Before inputting the log and the log type of the log into a detection engine corresponding to the target attack type of the log, the method further includes: Adding a corresponding information field to the log to obtain an enriched new log; the information field includes at least one of the target keyword of the log, the target attack type of the log, the event asset attribute of the log, and the event intelligence attribute; The step of inputting the log and the log type of the log into a detection engine corresponding to a target attack type of the log includes: The new log and the log type of the new log are input into a detection engine corresponding to the target attack type of the new log.

6. The log processing method according to claim 1, characterized in that: The extracting target keywords from the log according to the log type of the log includes: If the log type is a network security log recording network data packet information, according to the protocol rules adopted by the network data packet, the components of the data access request are extracted from the network data packet as the target keywords of the log.

7. The log processing method according to claim 1, characterized in that: If it is detected that the log describes a network attack, the method further includes: Perform security incident protection on the target of the network attack, and / or initiate an early warning prompt.

8. A log processing system, characterized in that: include: a receiving unit and a processing unit; The receiving unit is used to receive logs sent by each device end, and classify the logs to determine the log types of the logs; at least some of the logs have different formats; The processing unit is used to extract target keywords from the log according to the log type of the log; The processing unit is further configured to, based on the target keyword in the log, identify the target attack type described in the log; The processing unit is further configured to input the log and the log type of the log into a detection engine corresponding to a target attack type of the log, so as to detect whether the log describes a network attack.

9. An electronic device, characterized in that: include: Processor and memory; The processor is configured to communicate with the memory and execute instructions in the memory to implement the method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product, characterized in that The computer program product comprises computer instructions, which implement the method according to any one of claims 1 to 7 when executed by a processor.

Images