Log data monitoring method and device, electronic equipment and storage medium

Abstract

The invention discloses a log data monitoring method and device, electronic equipment and a storage medium. The log data monitoring method comprises the steps of obtaining log data collected by a log server; extracting the log data to obtain attack data; fusing each attack field in the attack data to obtain network security log big data; inputting the network security log big data into a pre-trained network attack model to obtain network attack information corresponding to the log data; summarizing the network attack information through the monitoring analysis platform, and converting non-structural text data in the summarized network attack information into a list form to obtain target network attack information; and displaying and monitoring the target network attack information. Therefore, timely monitoring and alarm feedback are carried out through the network attack information identified from the log data, the network attack information can be effectively processed, and the security of the system is ensured.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to a method and device for monitoring log data, an electronic device, and a storage medium. Background Art

[0002] In recent years, with the in-depth development of digitalization and networking, the integration of various industries has continued to increase, and network technology has continued to advance, making network security issues more complicated. In existing technologies, network security equipment will be used to identify, detect, and alert different network attacks, thereby forming different network security logs, and then extract truly effective network attack information from the network security log big data to solve network security issues.

[0003] However, the existing technology subsequently failed to accurately monitor and issue real-time alerts for effective network attack information, resulting in the inability to timely identify and locate attacks, which in turn affected the security and reliability of the system. Summary of the invention

[0004] Based on the above-mentioned deficiencies of the prior art, the present application provides a log data monitoring method and device, electronic device, and storage medium to solve the problems of the prior art that affect system security.

[0005] In order to achieve the above objectives, this application provides the following technical solutions:

[0006] The first aspect of the present application provides a method for monitoring log data, comprising:

[0007] Get the log data collected by the log server;

[0008] Extracting the log data to obtain attack data;

[0009] Merging each attack field in the attack data to obtain network security log big data;

[0010] Inputting the network security log big data into a pre-trained network attack model to obtain network attack information corresponding to the log data; wherein the network attack model is pre-trained using a plurality of historical network security log big data;

[0011] Summarize the network attack information through the monitoring and analysis platform, and convert the unstructured text data in the summarized network attack information into a list form to obtain the target network attack information;

[0012] The target network attack information is displayed and monitored.

[0013] Optionally, in the above-mentioned log data monitoring method, the fusing of each attack field in the attack data to obtain network security log big data includes:

[0014] extracting all attack fields from the attack data;

[0015] Detecting whether there are multiple fields with similar semantics in all the attack fields;

[0016] If there are multiple fields with similar semantics in all the attack fields, each of the fields with similar semantics is merged to obtain multiple semantic fields;

[0017] For each of the semantic fields, a plurality of numerical values ​​corresponding to the semantic field are merged to obtain a target attack field corresponding to the semantic field;

[0018] The target attack fields corresponding to all the semantic fields are used as network security log big data.

[0019] Optionally, in the above-mentioned log data monitoring method, for each of the semantic fields, fusing multiple values ​​corresponding to the semantic field to obtain a target attack field corresponding to the semantic field includes:

[0020] For each of the semantic fields, detecting whether there are multiple structured values ​​among the multiple values ​​corresponding to the semantic field;

[0021] If there are multiple structured values ​​among the multiple values ​​corresponding to the semantic field, then select each of the structured values ​​from the multiple values ​​corresponding to the semantic field;

[0022] Converting the expression form of each structured numerical value into a preset form to obtain each target structured numerical value;

[0023] Performing fusion determination on the remaining values ​​of the semantic field to obtain fused values ​​corresponding to the semantic field;

[0024] Each of the target structured values ​​and each of the fused values ​​is used as a target attack field corresponding to the semantic field.

[0025] Optionally, in the above-mentioned log data monitoring method, the network attack model training method includes:

[0026] Obtain multiple historical network security log big data;

[0027] Input each of the historical network security log big data into the network attack model to obtain the network attack information corresponding to each of the historical network security log big data;

[0028] For each piece of the historical network security log big data, determine whether the network attack information corresponding to the historical network security log big data is actual network attack information;

[0029] If the network attack information corresponding to the historical network security log big data is actual network attack information, determining the network attack model as a trained network attack model;

[0030] If the network attack information corresponding to the historical network security log big data is not actual network attack information, the parameters of the network attack model are adjusted, and the step of inputting each of the historical network security log big data into the network attack model is returned to execute to obtain the network attack information corresponding to each of the historical network security log big data.

[0031] Optionally, in the above-mentioned log data monitoring method, it also includes:

[0032] Convert the network attack information into an alarm signal through a monitoring and analysis platform;

[0033] The alarm signal is sent to the front end through the monitoring and analysis platform.

[0034] A second aspect of the present application provides a log data monitoring device, comprising:

[0035] A data acquisition unit, used to acquire log data collected by the log server;

[0036] An extraction unit, used to extract the log data to obtain attack data;

[0037] A fusion unit, used for fusing each attack field in the attack data to obtain network security log big data;

[0038] A data input unit, used to input the network security log big data into a pre-trained network attack model to obtain network attack information corresponding to the log data; wherein the network attack model is pre-trained using a plurality of historical network security log big data;

[0039] A summarizing unit, used to summarize the network attack information through the monitoring and analysis platform, and convert the unstructured text data in the summarized network attack information into a list form to obtain target network attack information;

[0040] The monitoring unit is used to display and monitor the target network attack information.

[0041] Optionally, in the above-mentioned log data monitoring device, the fusion unit includes:

[0042] An extraction unit, used for extracting all attack fields from the attack data;

[0043] A field detection unit, used to detect whether there are multiple fields with similar semantics in all the attack fields;

[0044] A first fusion subunit is used for fusing each of the fields with similar semantics to obtain multiple semantic fields if there are multiple fields with similar semantics in all the attack fields;

[0045] A second fusion subunit is used to fuse multiple values ​​corresponding to each semantic field to obtain a target attack field corresponding to the semantic field;

[0046] First, as a unit, it is used to take the target attack fields corresponding to all the semantic fields as network security log big data.

[0047] Optionally, in the above-mentioned log data monitoring device, the second fusion subunit includes:

[0048] A value detection unit, used for detecting, for each of the semantic fields, whether there are multiple structured values ​​among the multiple values ​​corresponding to the semantic field;

[0049] a screening unit, configured to screen out each of the structured values ​​from the multiple values ​​corresponding to the semantic field if there are multiple structured values ​​among the multiple values ​​corresponding to the semantic field;

[0050] A conversion unit, used to convert the expression form of each of the structured numerical values ​​into a preset form to obtain each target structured numerical value;

[0051] A fusion determination unit, used for performing fusion determination on the remaining values ​​of the semantic field to obtain fusion values ​​corresponding to the semantic field;

[0052] The second unit is used to use each of the target structured values ​​and each of the fused values ​​as a target attack field corresponding to the semantic field.

[0053] Optionally, in the above-mentioned log data monitoring device, it also includes:

[0054] An acquisition unit, used for acquiring multiple historical network security log big data;

[0055] An input unit, used to input each of the historical network security log big data into a network attack model to obtain network attack information corresponding to each of the historical network security log big data;

[0056] A judging unit, configured to judge, for each of the historical network security log big data, whether the network attack information corresponding to the historical network security log big data is actual network attack information;

[0057] A determination unit, configured to determine the network attack model as a trained network attack model if the network attack information corresponding to the historical network security log big data is actual network attack information;

[0058] The adjustment unit is used to adjust the parameters of the network attack model if the network attack information corresponding to the historical network security log big data is not actual network attack information, and return to execute the step of inputting each of the historical network security log big data into the network attack model to obtain the network attack information corresponding to each of the historical network security log big data.

[0059] Optionally, in the above-mentioned log data monitoring device, it also includes:

[0060] A conversion unit, used to convert the network attack information into an alarm signal through a monitoring and analysis platform;

[0061] The sending unit is used to send the alarm signal to the front end through the monitoring and analysis platform.

[0062] A third aspect of the present application provides an electronic device, including:

[0063] Memory and processor;

[0064] Wherein, the memory is used to store programs;

[0065] The processor is used to execute the program, and when the program is executed, it is specifically used to implement a log data monitoring method as described in any one of the above.

[0066] A fourth aspect of the present application provides a method for storing a computer program, which, when executed, is used to implement a log data monitoring method as described in any one of the above.

[0067] The present application provides a method for monitoring log data, which obtains log data collected by a log server, extracts the log data, obtains attack data, and then fuses each attack field in the attack data to obtain network security log big data, and then inputs the network security log big data into a pre-trained network attack model to obtain network attack information corresponding to the log data, wherein the network attack model is pre-trained using multiple historical network security log big data, and then the network attack information is summarized through a monitoring and analysis platform, and the non-structured text data in the summarized network attack information is converted into a list form to obtain target network attack information, and finally the target network attack information is displayed and monitored. Thus, by timely monitoring and alarm feedback of the network attack information identified from the log data, the network attack information can be effectively processed, thereby ensuring the security of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0069] Figure 1 A flow chart of a log data monitoring method provided in an embodiment of the present application;

[0070] Figure 2 A flowchart of a method for acquiring network security log big data provided by another embodiment of the present application;

[0071] Figure 3 A flowchart of a method for determining a target attack field provided in another embodiment of the present application;

[0072] Figure 4 A flowchart of a method for training a network attack model provided in another embodiment of the present application;

[0073] Figure 5 A flowchart of a method for alerting target network attack information provided by another embodiment of the present application;

[0074] Figure 6 A schematic diagram of the structure of a log data monitoring device provided in another embodiment of the present application;

[0075] Figure 7 A schematic diagram of the structure of an electronic device provided in another embodiment of the present application. DETAILED DESCRIPTION

[0076] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0077] In this application, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0078] The present application embodiment provides a method for monitoring log data, such as Figure 1 As shown, the specific steps include:

[0079] S101. Obtain log data collected by a log server.

[0080] It should be noted that before obtaining the log data collected by the log server, it is necessary to unify various network security devices that detect attack information so as to obtain all possible attack information. Therefore, it is necessary to first configure the system log function of each network security device, configure the receiving server to be the same log server, and use the log server as the unified server for receiving log data. Therefore, the log data collected by each network security device will be transmitted to the unified server, that is, the log server. At this time, the system only needs to obtain the log data collected by the log server to realize the identification operation.

[0081] Among them, network security equipment includes but is not limited to intrusion detection system (IDS), web application firewall (WAF) and bastion host, etc.

[0082] S102: Extract log data to obtain attack data.

[0083] It is understandable that after obtaining the log data, it is necessary to first perform preliminary attack identification on the log data, that is, to extract the field information in the log data, mainly extracting the attack type, attack result, victim IP, attack IP and other fields from the field information. Therefore, the extracted attack data will contain attack type, attack result, victim IP, attack IP and other fields.

[0084] S103: Merge each attack field in the attack data to obtain network security log big data.

[0085] Specifically, considering that the attack fields contained in the attack data may have differences in syntax, but the meanings expressed by the semantics are the same, for example, "victim IP" and "attacked IP" have different expressions but the same meaning. Therefore, in order to improve the subsequent recognition efficiency, it is necessary to merge the attack fields with the same meaning in the attack data.

[0086] Optionally, in another embodiment of the present application, another specific implementation of step S103 is as follows: Figure 2 As shown, the following steps are included:

[0087] S201. Extract all attack fields from attack data.

[0088] S202: Detect whether there are multiple fields with similar semantics in all attack fields.

[0089] It should be noted that in terms of field fusion, manual judgment, semantic distance judgment, large language model judgment and other judgment methods can be used to determine whether there are multiple fields with similar semantics in all attack fields. If so, step S203 is executed.

[0090] Optionally, if it does not exist, the attack data is used as network security log big data and step S104 is executed.

[0091] Specifically, the manual judgment method combines manual experience to determine whether each attack field has a semantically similar field. The semantic distance judgment method mainly divides fields with similar semantics into a group by semantic similarity. The large language model judgment method mainly inputs all attack fields into the current large language models such as chatglm and LLAMA. The pre-trained models contained in these large language models will judge and fuse fields with similar semantics.

[0092] S203: Merge each field with similar semantics to obtain multiple semantic fields.

[0093] It should be noted that when there are multiple fields with similar semantics in all attack fields, each field with similar semantics can be merged into one field, that is, a semantic field, according to the semantics.

[0094] S204 , for each semantic field, merge multiple numerical values ​​corresponding to the semantic field to obtain a target attack field corresponding to the semantic field.

[0095] It is understandable that after the field fusion is realized, several values ​​corresponding to each field need to be fused to facilitate subsequent effective attack analysis.

[0096] Optionally, in another embodiment of the present application, another specific implementation of step S204 is as follows: Figure 3 As shown, the following steps are included:

[0097] S301 . For each semantic field, detect whether there are multiple structured values ​​among multiple values ​​corresponding to the semantic field.

[0098] Specifically, in the embodiments of the present application, the fusion of numerical values ​​can be divided into structured numerical fusion and unstructured numerical fusion. Among them, structured numerical values ​​refer to data with clear meaning and format, such as attack time, attacking IP, attacked IP, etc. Unstructured numerical values ​​refer to data without a fixed format and structure, usually expressed as text or descriptive information, such as threat level: low risk, medium risk, high risk, and ultra-high risk, sometimes in the form of low risk, medium risk, high risk, and ultra-high risk, etc., attack type: weak password, code execution, and SQL injection, etc.

[0099] Therefore, in order to better merge the values ​​of the fields, it is optional to detect whether there are multiple structured values ​​in the multiple values ​​corresponding to the semantic field by editing the code. If there are multiple structured values ​​in the multiple values ​​corresponding to the semantic field, step S302 is executed.

[0100] Optionally, if there are no multiple structured values ​​among the multiple values ​​corresponding to the semantic field, it means that the multiple values ​​corresponding to the semantic field are all unstructured values. The multiple values ​​corresponding to the semantic field can be directly fused and determined to obtain the fused values ​​corresponding to the semantic field.

[0101] S302: Filter out each structured value from multiple values ​​corresponding to the semantic field.

[0102] It is understandable that when there are multiple structured values ​​among the multiple values ​​corresponding to the semantic field, in order to flexibly and conveniently process only the structured values, it is optionally possible to filter out each structured value from the multiple values ​​corresponding to the semantic field by means of code.

[0103] S303: Convert the expression form of each structured value into a preset form to obtain each target structured value.

[0104] Specifically, since structured values ​​are data with clear meanings and formats, such values ​​are concentrated on the unification of expression forms. For example, the fusion of attack time mainly involves the matching of time formats. The target structured value can be obtained by converting time formats of different expressions into the same time format.

[0105] S304: Perform fusion judgment on the remaining values ​​of the semantic field to obtain fused values ​​corresponding to the semantic field.

[0106] It is understandable that after filtering out the structured values, only the unstructured values ​​remain in the semantic field. At this time, the unstructured values ​​need to be fused using field fusion methods, i.e., manual judgment, semantic distance judgment, large language model judgment, etc. Specifically, first extract the values ​​with the same semantics from the unstructured values, and then fuse the values ​​with the same semantics into one according to the semantic meaning, and finally obtain the fused values ​​corresponding to the unstructured values.

[0107] S305: Use each target structured value and each fused value as a target attack field corresponding to the semantic field.

[0108] Optionally, each target structured value and each fused value may be integrated together as a target attack field.

[0109] S205. Target attack fields corresponding to all semantic fields are used as network security log big data.

[0110] S104: Input the network security log big data into a pre-trained network attack model to obtain network attack information corresponding to the log data.

[0111] Among them, the network attack model is pre-trained using multiple historical network security log big data.

[0112] It should be noted that the amount of data in the integrated network security log big data is tens of millions, so it is necessary to extract truly effective network attack information from a large amount of data. Among them, truly effective network attack information refers to verified, accurate and practical intelligence or data about network attacks. It can help organizations to understand the current threat situation in a timely manner, take appropriate protective measures, and respond accordingly. Therefore, information about one or two attacks cannot be used as network attack information at will. Effective network attack information should be based on comprehensive data collection, reliable sources and in-depth analysis. Only through continuous monitoring and evaluation of multiple attack events can we form an accurate understanding of the network security situation and formulate corresponding defense strategies. This can avoid wrong decisions caused by one-sided information and improve the overall security protection capabilities of the organization.

[0113] The existing evaluation of whether there is effective network attack information in network security log big data is mainly judged by manual, decision tree algorithm and other interpretable machine learning algorithms. However, this interpretable machine learning algorithm can present the recognition rules from a more objective perspective, but it still relies on manual labeling of data before the model can learn, which may result in the obtained network attack information being inaccurate. Therefore, in the embodiment of the present application, the network security log big data is input into a pre-trained network attack model to obtain the effective network attack information corresponding to the network security log big data. Network attack models such as black box models such as neural networks and large models can extract network attack information from network security big data without exploring their specific rules, reducing the difficulty and pressure of manual identification, and also providing a basis for subsequent precise monitoring of network attacks based on this method.

[0114] Optionally, the present application embodiment provides a method for training a network attack model, such as Figure 4 As shown, the following steps are included:

[0115] S401. Obtain multiple historical network security log big data.

[0116] Optionally, multiple historical network security log big data can be obtained from the data storage location for subsequent network attack model training.

[0117] S402: Input each piece of historical network security log big data into a network attack model to obtain network attack information corresponding to each piece of historical network security log big data.

[0118] Optionally, features can be extracted from historical network security log big data first. For example, the features can be statistical features: traffic peak per unit time, number of abnormal connection attempts, etc., protocol features: analyze the type of protocol used, such as HTTP, FTP, etc., IP features: analyze the characteristics of source IP and target IP, and identify abnormal behavior, time features: analyze the time pattern of the attack, such as whether it occurs during non-working hours, and then input the extracted features into the network attack model for training, which can help identify known attacks, discover new attack patterns, and enhance the organization's network security protection capabilities.

[0119] S403: for each piece of historical network security log big data, determine whether the network attack information corresponding to the historical network security log big data is actual network attack information.

[0120] It should be noted that the network attack information corresponding to each historical network security log big data obtained through the network attack model in step S402 may not be the real network attack information, so it is necessary to further judge the network attack information corresponding to the historical network security log big data. Therefore, after obtaining the network attack information corresponding to the historical network security log big data, it is also necessary to judge whether the network attack information corresponding to the historical network security log big data is the actual network attack information for each historical network security log big data. If the network attack information corresponding to the historical network security log big data is the actual network attack information, it means that the network attack information corresponding to the historical network security log big data is the real network attack information, and then execute step S404. If the network attack information corresponding to the historical network security log big data is not the actual network attack information, it means that the network attack model needs to be iteratively trained, and then execute step S406.

[0121] S404: Determine the network attack model as a trained network attack model.

[0122] S405: Adjust parameters of the network attack model.

[0123] It should be noted that when the network attack information corresponding to the historical network security log big data is not the actual network attack information, it is necessary to return to step S402 until the network attack information corresponding to each historical network security log big data is the actual network attack information.

[0124] S105. Summarize the network attack information through the monitoring and analysis platform, and convert the unstructured text data in the summarized network attack information into a list form to obtain the target network attack information.

[0125] It should be noted that after obtaining effective network attack information, the relevant network attack information needs to be monitored in a timely and accurate manner, that is, the network attack information output by the network attack model can be summarized and displayed through the monitoring and analysis platform, and the json format text in the network attack information can be parsed, that is, the unstructured text data in the network attack information can be converted into a list form, so that the network attack information can be monitored at a glance.

[0126] Optionally, on this basis, the list display content can be personalized and increased or decreased according to the situation of the unit. The monitoring and analysis platform should also support periodic statistics of network attacks for later network attack analysis, disposal and prevention.

[0127] S106. Display and monitor the target network attack information.

[0128] Optionally, after displaying and monitoring the target network attack information, an alarm signal of the target network attack information can also be fed back to the front end so that it can be processed in time. Therefore, another embodiment of the present application provides an alarm method for target network attack information, such as Figure 5 As shown, the specific steps include:

[0129] S501. Convert network attack information into alarm signals through the monitoring and analysis platform.

[0130] It is understandable that in order to push network attack information to security personnel in monitoring and non-monitoring posts more timely, so that security personnel can quickly deal with the network attack information, the network attack information can be converted into alarm signals through the monitoring and analysis platform.

[0131] For example, for monitoring personnel, the monitoring and analysis platform can be configured with alarm tone and other functions according to their roles, so that they can respond to network attacks as soon as they receive the alarm signal. For non-monitoring personnel, a SMS alarm module can be built on the monitoring and analysis platform so that they can also receive alarm information in a timely manner, which is convenient for them to analyze network attacks and further deal with emergencies.

[0132] S502. Send an alarm signal to the front end through the monitoring and analysis platform.

[0133] The present application provides a method for monitoring log data, which obtains log data collected by a log server, extracts the log data, obtains attack data, and then fuses each attack field in the attack data to obtain network security log big data, and then inputs the network security log big data into a pre-trained network attack model to obtain network attack information corresponding to the log data, wherein the network attack model is pre-trained using multiple historical network security log big data, and then the network attack information is summarized through a monitoring and analysis platform, and the non-structured text data in the summarized network attack information is converted into a list form to obtain target network attack information, and finally the target network attack information is displayed and monitored. Thus, by timely monitoring and alarm feedback of the network attack information identified from the log data, the network attack information can be effectively processed, thereby ensuring the security of the system.

[0134] Another embodiment of the present application provides a monitoring device for log data, such as Figure 6 As shown, it includes the following units:

[0135] The data acquisition unit 601 is used to acquire the log data collected by the log server.

[0136] The extraction unit 602 is used to extract the log data to obtain attack data.

[0137] The fusion unit 603 is used to fuse each attack field in the attack data to obtain network security log big data.

[0138] The data input unit 604 is used to input the network security log big data into the pre-trained network attack model to obtain the network attack information corresponding to the log data. The network attack model is pre-trained using a plurality of historical network security log big data.

[0139] The summarizing unit 605 is used to summarize the network attack information through the monitoring and analysis platform, and convert the unstructured text data in the summarized network attack information into a list form to obtain the target network attack information.

[0140] The monitoring unit 606 is used to display and monitor the target network attack information.

[0141] It should be noted that the specific working process of the above-mentioned units in the embodiment of the present application can refer to steps S101 to S106 in the above-mentioned method embodiment, and will not be repeated here.

[0142] Optionally, in a log data monitoring device provided by another embodiment of the present application, the fusion unit 603 includes:

[0143] The extraction unit is used to extract all attack fields from the attack data.

[0144] The field detection unit is used to detect whether there are multiple fields with similar semantics in all attack fields.

[0145] The first fusion subunit is used for fusing each of the fields with similar semantics to obtain multiple semantic fields if there are multiple fields with similar semantics in all attack fields.

[0146] The second fusion subunit is used to fuse multiple numerical values ​​corresponding to each semantic field respectively to obtain a target attack field corresponding to the semantic field.

[0147] The first is used as a unit to treat the target attack fields corresponding to all semantic fields as network security log big data.

[0148] Optionally, in a log data monitoring device provided by another embodiment of the present application, the second fusion subunit includes:

[0149] The value detection unit is used to detect, for each semantic field, whether there are multiple structured values ​​in the multiple values ​​corresponding to the semantic field.

[0150] The screening unit is used to screen out each structured value from the multiple values ​​corresponding to the semantic field if there are multiple structured values ​​among the multiple values ​​corresponding to the semantic field.

[0151] The conversion unit is used to convert the expression form of each structured numerical value into a preset form to obtain each target structured numerical value.

[0152] The fusion determination unit is used to perform fusion determination on the remaining values ​​of the semantic field to obtain the fusion values ​​corresponding to the semantic field.

[0153] The second is used as a unit to use each target structured value and each fused value as a target attack field corresponding to a semantic field.

[0154] Optionally, a log data monitoring device provided in another embodiment of the present application further includes:

[0155] The acquisition unit is used to acquire multiple historical network security log big data.

[0156] The input unit is used to input each historical network security log big data into the network attack model to obtain the network attack information corresponding to each historical network security log big data.

[0157] The judgment unit is used to judge, for each piece of historical network security log big data, whether the network attack information corresponding to the historical network security log big data is actual network attack information.

[0158] The determination unit is used to determine the network attack model as a trained network attack model if the network attack information corresponding to the historical network security log big data is actual network attack information.

[0159] The adjustment unit is used to adjust the parameters of the network attack model if the network attack information corresponding to the historical network security log big data is not the actual network attack information, and return to execute to input each historical network security log big data into the network attack model to obtain the network attack information corresponding to each historical network security log big data.

[0160] Optionally, a log data monitoring device provided in another embodiment of the present application further includes:

[0161] The conversion unit is used to convert network attack information into alarm signals through the monitoring and analysis platform.

[0162] The sending unit is used to send an alarm signal to the front end through the monitoring and analysis platform.

[0163] It should be noted that the specific working process of each unit provided in the above embodiments of the present application can refer to the corresponding steps in the above method embodiments, and will not be repeated here.

[0164] Another embodiment of the present application provides an electronic device, such as Figure 7 As shown, including:

[0165] Memory 701 and processor 702 .

[0166] The memory 701 is used to store programs.

[0167] The processor 702 is used to execute a program. When the program is executed, it is specifically used to implement a log data monitoring method provided by any one of the above embodiments.

[0168] Another embodiment of the present application provides a computer storage medium for storing a computer program. When the computer program is executed, it is used to implement a log data monitoring method provided in any of the above embodiments.

[0169] Computer storage media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0170] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0171] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for monitoring log data, characterized in that: include: Get the log data collected by the log server; Extracting the log data to obtain attack data; Merging each attack field in the attack data to obtain network security log big data; Inputting the network security log big data into a pre-trained network attack model to obtain network attack information corresponding to the log data; wherein the network attack model is pre-trained using a plurality of historical network security log big data; Summarize the network attack information through the monitoring and analysis platform, and convert the unstructured text data in the summarized network attack information into a list form to obtain the target network attack information; The target network attack information is displayed and monitored.

2. The method according to claim 1, characterized in that The fusion of each attack field in the attack data to obtain network security log big data includes: extracting all attack fields from the attack data; Detecting whether there are multiple fields with similar semantics in all the attack fields; If there are multiple fields with similar semantics in all the attack fields, each of the fields with similar semantics is merged to obtain multiple semantic fields; For each of the semantic fields, a plurality of numerical values ​​corresponding to the semantic field are merged to obtain a target attack field corresponding to the semantic field; The target attack fields corresponding to all the semantic fields are used as network security log big data.

3. The method according to claim 2, characterized in that The step of fusing multiple values ​​corresponding to each of the semantic fields to obtain a target attack field corresponding to the semantic field includes: For each of the semantic fields, detecting whether there are multiple structured values ​​among the multiple values ​​corresponding to the semantic field; If there are multiple structured values ​​among the multiple values ​​corresponding to the semantic field, then select each of the structured values ​​from the multiple values ​​corresponding to the semantic field; Converting the expression form of each structured numerical value into a preset form to obtain each target structured numerical value; Performing fusion determination on the remaining values ​​of the semantic field to obtain fused values ​​corresponding to the semantic field; Each of the target structured values ​​and each of the fused values ​​is used as a target attack field corresponding to the semantic field.

4. The method according to claim 1, characterized in that The training method of the network attack model comprises: Obtain multiple historical network security log big data; Input each of the historical network security log big data into the network attack model to obtain the network attack information corresponding to each of the historical network security log big data; For each piece of the historical network security log big data, determine whether the network attack information corresponding to the historical network security log big data is actual network attack information; If the network attack information corresponding to the historical network security log big data is actual network attack information, determining the network attack model as a trained network attack model; If the network attack information corresponding to the historical network security log big data is not actual network attack information, the parameters of the network attack model are adjusted, and the step of inputting each of the historical network security log big data into the network attack model is returned to execute to obtain the network attack information corresponding to each of the historical network security log big data.

5. The method according to claim 1, characterized in that Also includes: Convert the network attack information into an alarm signal through a monitoring and analysis platform; The alarm signal is sent to the front end through the monitoring and analysis platform.

6. A log data monitoring device, characterized in that: include: A data acquisition unit, used to acquire log data collected by the log server; An extraction unit, used to extract the log data to obtain attack data; A fusion unit, used for fusing each attack field in the attack data to obtain network security log big data; A data input unit, used to input the network security log big data into a pre-trained network attack model to obtain network attack information corresponding to the log data; wherein the network attack model is pre-trained using a plurality of historical network security log big data; A summarizing unit, used to summarize the network attack information through the monitoring and analysis platform, and convert the unstructured text data in the summarized network attack information into a list form to obtain target network attack information; The monitoring unit is used to display and monitor the target network attack information.

7. The device according to claim 6, characterized in that The fusion unit comprises: An extraction unit, used for extracting all attack fields from the attack data; A field detection unit, used to detect whether there are multiple fields with similar semantics in all the attack fields; A first fusion subunit is used for fusing each of the fields with similar semantics to obtain multiple semantic fields if there are multiple fields with similar semantics in all the attack fields; A second fusion subunit is used to fuse multiple values ​​corresponding to each semantic field to obtain a target attack field corresponding to the semantic field; First, as a unit, it is used to take the target attack fields corresponding to all the semantic fields as network security log big data.

8. The device according to claim 7, characterized in that The second fusion subunit comprises: A value detection unit, used for detecting, for each of the semantic fields, whether there are multiple structured values ​​among the multiple values ​​corresponding to the semantic field; a screening unit, configured to screen out each of the structured values ​​from the multiple values ​​corresponding to the semantic field if there are multiple structured values ​​among the multiple values ​​corresponding to the semantic field; A conversion unit, used to convert the expression form of each of the structured numerical values ​​into a preset form to obtain each target structured numerical value; A fusion determination unit, used for performing fusion determination on the remaining values ​​of the semantic field to obtain fusion values ​​corresponding to the semantic field; The second unit is used to use each of the target structured values ​​and each of the fused values ​​as a target attack field corresponding to the semantic field.

9. An electronic device, characterized in that: include: Memory and processor; Wherein, the memory is used to store programs; The processor is used to execute the program, and when the program is executed, it is specifically used to implement a log data monitoring method as described in any one of claims 1 to 5.

10. A computer storage medium, characterized in that: Used to store a computer program, which, when executed, is used to implement a log data monitoring method as described in any one of claims 1 to 5.

Images