Host log data threat detection model training method, detection method and device
By constructing multiple original feature graphs and edge-fused feature graphs, and combining them with graph neural networks and machine learning training models, we solve the problem of detecting 0-day attacks and persistent attacks in existing technologies and achieve accurate threat detection in host log data.
Patent Information
- Application Number
- CN202310471898.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-04-27
- Publication Date
- 2025-10-10
- Estimated Expiration
- 2043-04-27
AI Technical Summary
Existing host log threat detection technology has difficulty detecting zero-day attacks and attacks that persist for an extended period of time. Anomaly-based detection methods ignore causal relationships during detection, and storing traceability graph models requires a large amount of storage space.
Construct multiple original feature graphs, fuse the feature graphs through graph neural networks, extract the existence probability of operation types, and use machine learning networks to train models to detect threats in host log data.
It achieves accurate and effective detection of host log data, can identify abnormal operations, reduce storage requirements, and improve the comprehensiveness and accuracy of detection.
Smart Images
Figure CN116662801B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of information security, and in particular to a host log data threat detection model training method and device. BACKGROUND
[0002] Audit logs are usually applied to threat detection. Researchers determine whether an attack is likely to occur according to access records in host logs. In network security protection, the deficiencies of virus detection, alarm analysis and other technologies can be made up. In recent years, as operating systems continue to upgrade, most hosts can record running logs in real time, and threat detection based on audit logs has received more and more attention.
[0003] Existing threat detection technologies for audit logs mainly fall into two categories: misuse-based and anomaly-based.
[0004] Misuse-based threat detection mainly matches attack behaviors of log events through predefined attack models. However, the misuse-based detection method is highly dependent on expert knowledge to predefine attack behavior matching rules due to the semantic gap between low-level audit events and high-level system behaviors, and can only detect known attack behaviors, but cannot detect 0-day attacks.
[0005] Anomaly-based threat detection mainly judges whether a target log event is abnormal by calculating the difference between the target log event and normal log events. The traditional anomaly-based detection method is difficult to effectively associate continuous and long-time attacks when detecting log events, because the causal relationship between logs is not considered. The anomaly-based detection method of constructing a traceability graph mostly detects anomalies in entities in logs. However, intrusion is actually a malicious operation through a control entity to complete an attack, so this coarse-grained detection method that ignores malicious operations affects the accuracy of detection to some extent. On the other hand, because the amount of audit log data is large and increasing, storing and calculating the traceability graph model in memory requires a considerable amount of storage space. SUMMARY
[0006] In view of this, the embodiments of the present application provide a host log data threat detection model training method and device to eliminate or improve one or more defects in the prior art.
[0007] A first aspect of the present application provides a host log data threat detection model training method, which comprises:
[0008] According to the log data corresponding to different time periods in the pre-acquired historical host log data set, a plurality of original feature maps are constructed, wherein the original feature map comprises a plurality of original edges and an operation type corresponding to each original edge.
[0009] According to each of the original feature maps, a corresponding edge fusion feature map is obtained, wherein the edge fusion feature map includes a plurality of aggregation operation edges;
[0010] According to each of the original feature maps, a corresponding existence probability of each of the operation types in each of the aggregation operation edges is obtained;
[0011] According to each of the log data in the historical host log data set and a plurality of corresponding existence probability labels, a preset machine learning network architecture is trained to obtain a corresponding host log data threat detection model.
[0012] In some embodiments of the present application, the plurality of original feature maps are constructed according to the log data corresponding to different time periods in the pre-acquired historical host log data set, comprising:
[0013] According to a preset time window, each of the log data in the host log data set is divided into a plurality of time window log data sets;
[0014] According to each of the time window log data sets, a corresponding original feature map is constructed.
[0015] In some embodiments of the present application, the corresponding edge fusion feature map is obtained according to each of the original feature maps, comprising:
[0016] The single or multiple operation types between each node in each of the original feature maps and another arbitrary node are fused into an aggregation operation edge;
[0017] According to the plurality of aggregation operation edges and a plurality of nodes corresponding to each of the original feature maps, a corresponding edge fusion feature map of each of the original feature maps is constructed.
[0018] In some embodiments of the present application, the existence probability of each of the operation types in each of the aggregation operation edges is obtained according to each of the original feature maps, comprising:
[0019] The node features in each of the original feature maps are extracted;
[0020] Each of the node features in each of the original feature maps and the corresponding edge fusion feature map are input into a graph neural network to update each node in each of the edge fusion feature maps, wherein the node includes a source node and a destination node;
[0021] The features of each of the aggregation operation edges are extracted and a corresponding fusion edge label is set for each of the aggregation operation edges;
[0022] concatenate the updated source node, the destination node and the feature of the aggregation operation edge in each of the edge fusion feature maps to obtain a representation vector;
[0023] input the representation vector and the fusion edge label corresponding to each of the aggregation operation edges in each of the edge fusion feature maps into a fully connected layer model to output a presence probability of each of the operation types in each of the aggregation operation edges.
[0024] In some embodiments of the present application, the training of the preset machine learning network architecture according to each log data in the historical host log data set and the respective corresponding presence probability label to obtain a corresponding host log data threat detection model comprises:
[0025] training the preset machine learning network architecture according to each log data in the historical host log data set and the respective corresponding presence probability label, and based on the preset loss function, calculating a respective loss probability value according to each presence probability and the respective corresponding presence probability label in the training process, and iteratively training the machine learning network architecture based on the loss probability value to generate a host log data threat detection model for outputting a threat presence probability according to a host log data set.
[0026] The second aspect of the present application provides a host log data threat detection method, which comprises:
[0027] obtaining a target host log data set within a preset time length;
[0028] inputting the target host log data set into the host log data threat detection model trained by the host log data threat detection model training method of the first aspect, so that the host log data threat detection model outputs a presence probability corresponding to each operation record in each log data in the target host log data set.
[0029] In some embodiments of the present application, after the target host log data set is inputted into the host log data threat detection model trained by the host log data threat detection model training method of the first aspect, so that the host log data threat detection model outputs a presence probability corresponding to each operation record in each log data in the target host log data set, the method further comprises:
[0030] judging each presence probability, if the presence probability is greater than or equal to a preset security threshold, determining that the operation record corresponding to the presence probability is a normal operation, and if the presence probability is less than the security threshold, determining that the operation record corresponding to the presence probability is an abnormal operation.
[0031] The third aspect of the present application provides a host log data threat detection model training device, the device comprises:
[0032] An original feature map construction module is configured to construct a plurality of original feature maps according to log data corresponding to different time periods in a historical host log data set obtained in advance, wherein the original feature maps comprise a plurality of original edges and operation types corresponding to each original edge;
[0033] An edge fusion feature map construction module is configured to obtain an edge fusion feature map corresponding to each original feature map, wherein the edge fusion feature map comprises a plurality of aggregated operation edges.
[0034] A threat calculation module is configured to obtain an existence probability of each operation type in each aggregated operation edge according to each original feature map.
[0035] A model construction module is configured to train a preset machine learning network architecture according to each log data in the historical host log data set and a plurality of existence probability labels corresponding thereto, to obtain a corresponding host log data threat detection model.
[0036] The fourth aspect of the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the host log data threat detection model training method of the first aspect described above, or implements the host log data threat detection method of the second aspect described above when executing the computer program.
[0037] The fourth aspect of the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the host log data threat detection model training method of the first aspect described above, or implement the host log data threat detection method of the second aspect described above.
[0038] The present application provides a host log data threat detection model training method, comprising: constructing a plurality of original feature maps according to log data corresponding to different time periods in a historical host log data set obtained in advance, wherein the original feature maps comprise a plurality of original edges and operation types corresponding thereto; obtaining an edge fusion feature map corresponding to each original feature map, wherein the edge fusion feature map comprises a plurality of aggregated operation edges; obtaining an existence probability of each operation type in each aggregated operation edge according to each original feature map; and training a preset machine learning network architecture according to the historical host log data set and a plurality of existence probability labels to obtain a host log data threat detection model. The present application can accurately, effectively and comprehensively detect threats in host log data.
[0039] Additional advantages, objects, and features of the application will be set forth in part by the description that follows, and will in part be apparent to those of ordinary skill in the art upon examination of the following or can be learned from practice of the application. The advantages and features of the application can be realized and obtained by means of the instrumentalities and combinations pointed out in the appended claims.
[0040] Those skilled in the art will appreciate that the objects and advantages of the application can be practiced without resorting to the details of the specific implementation described herein, and that the full scope of the application will become more fully apparent and be better understood with reference to the following detailed description. BRIEF DESCRIPTION OF DRAWINGS
[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description, serve to explain the principles of the application. The components in the drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the application. For purposes of clarity and understanding, it is also to be understood that certain portions of the drawings can be exaggerated and others omitted in order to more readily demonstrate and describe the important structures and features of some embodiments of the application. In the drawings:
[0042] Figure 1 A flowchart of a host log data threat detection model training method in an embodiment of the application.
[0043] Figure 2 A flowchart of a host log data threat detection method in another embodiment of the application.
[0044] Figure 3 A structural diagram of a host log data threat detection model training device in another embodiment of the application.
[0045] FIG. 4(a) is a schematic diagram of a Darpa data set in another embodiment of the application.
[0046] FIG. 4(b) is a schematic diagram of a Camflow data set in another embodiment of the application. DETAILED DESCRIPTION
[0047] In order to make the objects, technical solutions and advantages of the application clearer, the following will further describe the application in conjunction with the embodiments and drawings. Here, the illustrative embodiments of the application and their descriptions are used to explain the application, but are not intended to limit the application.
[0048] It should also be noted that, in order to avoid obscuring the application due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the application are shown in the drawings, and other details not closely related to the application are omitted.
[0049] It should be emphasized that the term "comprises / comprising" when used in this specification is taken to specify the presence of stated features, integers, steps or components but does not preclude the presence or addition of one or more other features, integers, steps, components, or groups thereof.
[0050] It should also be noted that, unless otherwise specified, the term "connected" herein can mean directly connected, but can also mean indirectly connected via intervening components.
[0051] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar components, or the same or similar steps.
[0052] The present application will be described in detail with reference to the following embodiments.
[0053] The embodiments of the present application provide a host log data threat detection model training method which can be executed by a host log data threat detection model training device, referring to Figure 1 , the host log data threat detection model training method specifically includes the following contents:
[0054] Step 110: According to the log data corresponding to different time periods in the pre-acquired historical host log data set, a plurality of original feature maps are constructed, and the original feature map includes a plurality of original edges and an operation type corresponding to each original edge.
[0055] Step 120: According to each original feature map, a corresponding edge fusion feature map is obtained, wherein the edge fusion feature map includes a plurality of aggregation operation edges.
[0056] Step 130: According to each original feature map, the existence probability of each operation type in each aggregation operation edge is obtained.
[0057] Step 140: According to each log data in the historical host log data set and the corresponding plurality of existence probability labels, a pre-set machine learning network architecture is trained to obtain a corresponding host log data threat detection model.
[0058] Specifically, a plurality of original feature maps are constructed according to log data corresponding to different time periods in a pre-acquired historical host log data set, the original feature maps including a plurality of original edges and operation types corresponding to each original edge; a corresponding edge fusion feature map is obtained for each original feature map, wherein the edge fusion feature map includes a plurality of aggregation operation edges; the existence probability of each operation type in each aggregation operation edge is obtained according to each original feature map; and a preset machine learning network architecture is trained according to each log data in the historical host log data set and a plurality of existence probability labels corresponding thereto, to obtain a corresponding host log data threat detection model, so that the threats in the host log data can be accurately, effectively and comprehensively detected.
[0059] In the historical host log data set Darpa public host log data set, the host log data is collected by using the Camflow tool. The log format in the Darpa data set is shown in FIG. 4(a): XXXObject represents an entity, XXX represents the type of the entity, for example, NetFlowObject represents an entity of type NetFlow, and the subsequent uuid is the universal unique identifier of the entity; Event represents an event, wherein subject represents the source entity, the subsequent uuid is the unique identifier of the entity, object represents the destination entity, the subsequent uuid is the unique identifier of the entity, type represents the operation type of the source entity on the destination entity in the event, and timestampNanos represents the timestamp of the event. The log format in the Camflow data set is shown in FIG. 4(b): when the type value is Entity, Activities, or Agent, it represents an entity, id is the unique identifier of the entity in camflow, uuid is the universal unique identifier, and object_type represents the entity type; other type values represent an event, from represents the id of the source entity, to represents the id of the destination entity, relation_type represents the operation type of the source entity on the destination entity in the event, and cf:date represents the timestamp of the event.
[0060] In addition, before model training, the historical host log data set also needs to be processed, and the processing method is as follows:
[0061] A uniform six-tuple format <src_uuid, src_type, dst_uuid, dst_type, option_type, timestamp> is used to represent a log entry, and the specific meanings of the elements in the six-tuple are as follows:
[0062] Element 1: src_uuid, representing the uuid of the source entity.
[0063] Element 2: src_type, representing the type of the source entity, such as file, process, socket, etc.
[0064] Element 3: dst_uuid, representing the uuid of the target entity.
[0065] Element 4: dst_type, representing the type of the source entity, such as file, process, socket, etc.
[0066] Element 5: option_type, representing the type of operation between the two entities, such as read, open, create, etc.
[0067] Element 6: timestamp, representing the timestamp of the log.
[0068] For example, the six-tuple <2FDC2573-3DC8-11E8-A5CB-3FA3753A265A, OBJECT_PROCESS, 70434EFC-9030-E251-B090-307CB1E2D224, OBJECT_FILE, EVENT_WRITE, 1523479064067918584> describes the event "At the time of 1523479064067918584, the PROCESS entity with uuid 2FDC2573-3DC8-11E8-A5CB-3FA3753A265A performed a WRITE operation on the FILE entity with uuid 70434EFC-9030-E251-B090-307CB1E2D224".
[0069] To further improve the effectiveness of model training, step 110 includes:
[0070] According to the preset time window, each log data in the host log data set is divided into a plurality of time window log data sets;
[0071] According to each of the time window log data sets, a corresponding original feature map is constructed.
[0072] Specifically, the pre-processed log data according to time is divided: define the time length of the unit time window as t, and obtain the time window T window = [(0, t), (t, 2t),...], according to the timestamp timestamp in each log data six-tuple, each log data is divided into the corresponding time window, thereby obtaining a plurality of time window log data sets divided by time window. For each time window log data set, an original feature map is constructed, and finally an original feature map set is obtained. An original feature map Graw is defined as follows: G = (V, E), where the node set V raw = (V raw , E opt ), where the node set V raw is a set of log data in the time window log data set; the edge set E opt is a set of operation records between any two log data in the time window log data set, represents an operation edge between the source node V src and the destination node V dst , and the more operation types there are between the two nodes, the more operation edges there are between the two nodes (i.e., the out edges of the individual or multiple operation types between each node and any other node), thereby further improving the effectiveness of model training.
[0073] In order to further improve the accuracy of model training, step 120 includes:
[0074] fusing the out edges of the individual or multiple operation types between each node in each of the original feature maps and any other node into an aggregated operation edge;
[0075] constructing an edge fusion feature map corresponding to each of the original feature maps according to the multiple aggregated operation edges and multiple nodes corresponding to each of the original feature maps.
[0076] Specifically, first, the out edges of the individual or multiple operation types between each node in the original feature map and any other node are fused into an aggregated operation edge, and then an edge fusion feature map corresponding to each of the original feature maps is constructed according to the multiple aggregated operation edges and multiple nodes corresponding to each of the original feature maps, thereby further improving the accuracy of model training.
[0077] wherein a six-tuple Z E describes a fused directed edge (i.e., an aggregated operation edge) E: <src_uuid, src_type, dst_uuid, dst_type, edge_feature, edge_type>, and the specific meanings of the elements in the six-tuple are as follows:
[0078] Element 1: src_uuid, representing the uuid of the source node.
[0079] Element 2: src_type, representing the type of the source node, such as file, process, socket, etc.
[0080] Element 3: dst_uuid, representing the uuid of the target node.
[0081] Element 4: dst_type, representing the type of the target node, such as file, process, socket, etc.
[0082] Element 5: edge_feature, representing the feature of the edge.
[0083] Element 6: edge_type, as the label of each aggregated operation edge (i.e., the fusion edge label), representing a vector composed of multiple operation types contained in the fused edge.
[0084] The graph based on edge fusion (edge fusion feature graph) G is defined as follows: G = (V, E), where the node set V is each log data in the time window log data set, and the edge set E represents the existence of operations between two log data.
[0085] It should be noted that the edge set E raw in the original feature graph G raw represents the operation edge (i.e., the original edge) between two entities, and each operation constitutes an edge; the edge set E in the graph based on edge fusion G represents the existence of operations between two log data, and as long as there is an operation between two entities, a edge will be constituted, that is, an aggregated operation edge.
[0086] In order to further improve the comprehensiveness of model training, step 130 includes:
[0087] extracting multiple node features in each of the original feature graphs;
[0088] inputting each of the node features in each of the original feature graphs and the corresponding edge fusion feature graph into a graph neural network to update each node in each of the edge fusion feature graphs, wherein the node includes a source node and a destination node;
[0089] extracting features of each of the aggregated operation edges and setting a corresponding fusion edge label for each of the aggregated operation edges;
[0090] concatenating the updated source node, destination node and feature of each of the aggregated operation edges in each of the edge fusion feature graphs to obtain a representation vector;
[0091] inputting the representation vector and the fusion edge label corresponding to each of the aggregated operation edges in each of the edge fusion feature graphs into a fully connected layer model to output the existence probability of each of the operation types in each of the aggregated operation edges.
[0092] Specifically, first, multiple node features in each original feature graph are extracted, and Z V is defined as a node feature, and the specific node feature is as follows:
[0093] Feature 1: in_degree, representing a vector composed of the in-degrees of the edges of different types of operations associated with the node, for example, assuming that there are read, write, create, open, exec, send_packet, and receive_entity in total, i.e., 7 types of operation in the historical host log data set, node a has 3 write in-edges, 4 read in-edges, and 1 exec in-edge to all the nodes connected to it, and the in-degrees of other types of in-edges are all 0, then the in_degree vector of node a is [4, 3, 0, 0, 1, 0, 0].
[0094] Feature 2: out_degree, representing a vector composed of the out-degrees of the edges of different types of operations associated with the node.
[0095] Feature 3: in_persistent_relative_time, representing the relative time of the in-edge operation of the node, i.e., the total time length of the in-edge operation of the node / the time length of the time window t.
[0096] Feature 4: out_persistent_relative_time, representing the relative time of the out-edge operation of the node, i.e., the total time length of the out-edge operation of the node / the time length of the time window t.
[0097] Feature 5: in_average_time, representing the average time of the in-edge operation of the node, i.e., in_persistent_relative_time / in_degree.
[0098] Feature 6: out_average_time, representing the average time of the out-edge operation of the node, i.e., out_persistent_relative_time / out_degree.
[0099] Node feature Z V The above six features are spliced to form.
[0100] Secondly, the features in the fusion operation edge are extracted as follows:
[0101] Feature 1: persistent_relative_time, representing the relative time of all operation types contained in the edge, i.e., the total time length of all operation types contained in the edge / the time length of the unit time window t.
[0102] Feature 2: average_time represents the average time of the operation types contained in the edge, i.e. persistent_relative_time / number of operation edges contained in the edge.
[0103] Feature 3: src_type represents the entity type of the source node, such as file, process, socket, etc.
[0104] Feature 4: dst_type represents the entity type of the target node, such as file, process, socket, etc.
[0105] The features of the aggregated operation edge edge_feature are composed of the above-mentioned four features.
[0106] Then use edge_type as the label of the aggregated operation edge (i.e. the fusion edge label), which represents the vector composed of the multiple operation types contained in the fused edge.
[0107] For example, assuming that there are read, write, create, open, exec, send_packet, receive_entity, a total of 7 operation types in the historical host log data set, and all operation edges between the source node a and the destination node b are fused into an edge e(a, b), there are a total of 3 write edges, 4 read edges, and 1 exec edge in the original edge, and the number of edges of other operation types is 0, then the edge_type of the fused edge e(a, b) is [1, 1, 0, 0, 1, 0, 0].
[0108] For example, the six-tuple <2FDC2573-3DC8-11E8-A5CB-3FA3753A265A, OBJECT_PROCESS, 70434EFC-9030-E251-B090-307CB1E2D224, OBJECT_FILE, [0.86, 0.1075], [1, 1, 0, 0, 1, 0, 0]> describes the fused edge “in the unit time window, the PROCESS entity with uuid 2FDC2573-3DC8-11E8-A5CB-3FA3753A265A performs WRITE operation, READ operation and EXEC operation on the FILE entity with uuid 70434EFC-9030-E251-B090-307CB1E2D224, the total time length of all operations accounts for 86% of the unit time window time length, and the average time length of each specific operation accounts for 10.75% of the unit time window time length”.
[0109] Finally, the Z in each of the extracted original structure diagrams is extracted V The node features and each edge-based fusion graph (i.e., a fusion feature graph) are input into a GraphSAGE (graph neural network) algorithm (a known algorithm) to update the representation of each node in each fusion feature graph. The updated representation of each source node and destination node at both ends of each aggregation operation edge is spliced with the feature vector edge_feature of the edge to obtain a representation vector of each aggregation operation edge. The representation vector of each aggregation operation edge and the label of the aggregation operation edge are input into an MLP model (a fully connected layer model) to output the existence probability of each operation type in each edge.
[0110] To further optimize the model, step 140 includes:
[0111] The preset machine learning network architecture is trained according to each log data in the historical host log data set and each corresponding existence probability label, and based on a preset loss function, a loss probability value corresponding to each existence probability and each corresponding existence probability label is calculated in the training process, and the machine learning network architecture is iteratively trained based on the loss probability value to generate a host log data threat detection model for outputting a threat existence probability according to a host log data set.
[0112] Specifically, the preset machine learning network architecture is trained according to each log data in the historical host log data set and each corresponding existence probability label, and based on a preset loss function, a loss probability value corresponding to each existence probability and each corresponding existence probability label is calculated in the training process, and the machine learning network architecture is iteratively trained based on the loss probability value to generate a host log data threat detection model for outputting a threat existence probability according to a host log data set, thereby further optimizing the model.
[0113] The embodiment of the present application also provides a host log data threat detection method which can be executed by a host log data threat detection device, and the method includes:
[0114] Step 210: obtaining a target host log data set within a preset time length.
[0115] Step 220: inputting the target host log data set into the host log data threat detection model trained by the host log data threat detection model training method mentioned in the foregoing embodiments, so that the host log data threat detection model outputs an existence probability corresponding to each operation record in each log data in the target host log data set.
[0116] Specifically, first, a target host log data set within a preset time length is obtained. The target host log data set is input into the host log data threat detection model trained by the host log data threat detection model training method mentioned in the foregoing embodiments, so that the host log data threat detection model outputs the existence probability of each operation record in each log data in the target host log data set, thereby being able to accurately, effectively and comprehensively detect threats in the host log data.
[0117] To further improve the accuracy of anomaly detection, after step 220, the following steps are further included:
[0118] Each of the existence probabilities is judged. If the existence probability is greater than or equal to a preset security threshold, the operation record corresponding to the existence probability is determined to be a normal operation. If the existence probability is less than the security threshold, the operation record corresponding to the existence probability is determined to be an abnormal operation.
[0119] Specifically, for each existence probability, if the existence probability is greater than or equal to a preset security threshold ε e , the operation type of the edge contained in the prediction is the operation type of the edge. If the predicted operation type is the same as the labeled operation type, the prediction is correct, and the operation type is a normal operation. Otherwise, the prediction is incorrect. For example, the operation types include read, write, create, open, exec, send_packet, and receive_entity, a total of 7 operation types, the prediction probability vector result is [0.8, 0.6, 0.1, 0.1, 0.01, 0.14, 0.35], and the threshold ε e is 0.5. Therefore, the predicted operation type is [1, 1, 0, 0, 0, 0, 0]. The operation type in the label is [1, 1, 0, 0, 0, 0, 0], and the predicted operation type is the same as the labeled operation type, indicating that the operation type of the edge is correctly predicted. The correctly predicted edge is regarded as a normal edge, and the operation record corresponding to the edge in the target host log data is a normal operation. The incorrectly predicted edge is regarded as an abnormal edge, and the operation record corresponding to the edge in the target host log data is an abnormal operation, and an alarm is issued.
[0120] From the software aspect, the application further provides a host log data threat detection model training device for executing all or part of the host log data threat detection model training method. Referring to FIG. 4, the host log data threat detection model training device specifically includes the following contents:
[0121] An original feature map construction module 10 is configured to construct a plurality of original feature maps according to log data corresponding to different time periods in a historical host log data set obtained in advance. The original feature maps include a plurality of original edges and operation types corresponding to each of the original edges.
[0122] An edge fusion feature map construction module 20 is configured to obtain a respective edge fusion feature map from each of the original feature maps, wherein the edge fusion feature map includes a plurality of aggregated operation edges;
[0123] A threat calculation module 30 is configured to obtain a presence probability of each of the operation types in each of the aggregated operation edges according to each of the original feature maps;
[0124] A model construction module 40 is configured to train a preset machine learning network architecture according to each of the log data in the historical host log data set and the corresponding plurality of presence probability labels, to obtain a host log data threat detection model.
[0125] The embodiments of the host log data threat detection model training apparatus provided in the present application can be specifically used to execute the processing procedures of the embodiments of the host log data threat detection model training method described above, and thus the functions thereof will not be repeated here, and the detailed descriptions of the embodiments of the host log data threat detection model training method described above can be referred to.
[0126] The present application provides a host log data threat detection model training apparatus, the method executed by the apparatus comprising: constructing a plurality of original feature maps according to log data corresponding to different time periods in a historical host log data set obtained in advance, wherein each original feature map includes a plurality of original edges and corresponding operation types; obtaining a respective edge fusion feature map from each of the original feature maps, wherein the edge fusion feature map includes a plurality of aggregated operation edges; obtaining a presence probability of each of the operation types in each of the aggregated operation edges according to each of the original feature maps; and training a preset machine learning network architecture according to the historical host log data set and a plurality of presence probability labels, to obtain a host log data threat detection model. The present application can accurately, effectively and comprehensively detect threats in host log data.
[0127] The present application also provides a host log data threat detection apparatus for executing all or part of the contents of the host log data threat detection method described above, and the embodiments of the host log data threat detection apparatus can be specifically used to execute the processing procedures of the embodiments of the host log data threat detection method described above, and thus the functions thereof will not be repeated here, and the detailed descriptions of the embodiments of the host log data threat detection method described above can be referred to.
[0128] The embodiments of the present application also provide an electronic device, for example, a central server, which can include a processor, a memory, a receiver and a transmitter. The processor is configured to execute the host log data threat detection model training method mentioned in the above embodiments or the host log data threat detection method mentioned in the above embodiments. The processor and the memory can be connected by a bus or other means, for example, by the bus connection. The receiver can be connected to the processor and the memory by wired or wireless means.
[0129] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination thereof.
[0130] The memory is a non-transitory computer readable storage medium, which can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the program instructions / modules corresponding to the host log data threat detection model training method or the host log data threat detection method mentioned in the above embodiments. The processor executes various functions and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implements the host log data threat detection model training method or the host log data threat detection method mentioned in the above embodiments.
[0131] The memory can include a program storage area and a data storage area. The program storage area can store an operating system and at least one application required by a function. The data storage area can store data created by the processor and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged with respect to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.
[0132] The one or more modules are stored in the memory and, when executed by the processor, perform the host log data threat detection model training method in the embodiments or the host log data threat detection method mentioned in the foregoing embodiments.
[0133] In some embodiments of the present application, the user equipment can include a processor, a memory, and a transceiver which can include a receiver and a transmitter, the processor, the memory, the receiver and the transmitter can be connected through a bus system, the memory is used to store computer instructions, and the processor is used to execute the computer instructions stored in the memory to control the transceiver to transceive signals.
[0134] As an implementation manner, the functions of the receiver and the transmitter in the present application can be implemented by a transceiving circuit or a transceiving dedicated chip, and the processor can be implemented by a dedicated processing chip, a processing circuit or a general-purpose chip.
[0135] As another implementation manner, the server provided by the embodiments of the present application can be implemented by using a general-purpose computer. That is, program codes for implementing the functions of the processor, the receiver and the transmitter are stored in the memory, and the general-purpose processor implements the functions of the processor, the receiver and the transmitter by executing the codes in the memory.
[0136] The embodiments of the present application also provide a computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of the foregoing host log data threat detection model training method or the host log data threat detection method mentioned in the foregoing embodiments. The computer readable storage medium can be a tangible storage medium such as a random access memory (RAM), an internal memory, a read only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable memory disk, a CD-ROM, or any other form of storage medium known in the technical field.
[0137] Those of ordinary skill in the art should understand that the example components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether the implementation is in hardware or software depends on the specific application and design constraints imposed on the overall system. Skilled artisans can employ different elements in different combinations to implement the described functions, but such elements do not depart from the scope of the present application. When implemented in hardware, the elements of the present application can be implemented as an electronic circuit, an application specific integrated circuit (ASIC), suitable firmware, a plug-in, a functional card, or the like. When implemented in software, the elements of the present application are program or code segments to perform the required tasks. The program or code segments can be stored in a machine readable medium or transmitted by a data signal carried in a carrier wave over a transmission medium or communication link.
[0138] It is to be understood that the application is not limited to particular configurations and processes described herein and shown in the drawings, which can vary. For the sake of brevity and clarity, detailed descriptions of well-known methods and apparatuses will not be repeated. In the above embodiments, several specific steps are described and illustrated in order to provide a thorough understanding of the method processes. However, the method processes according to the application can be performed in a number of different specific sequences, and steps can be modified, combined, and / or separated into different steps, and / or other steps can be added, without departing from the spirit and scope of the application.
[0139] In the present application, features described and / or illustrated with respect to one embodiment can be used in the same or a similar way in one or more other embodiments and / or combined with or substituted for features of other embodiments.
[0140] The above description is merely illustrative of the application, and is not intended to limit the scope of the application. Various modifications and changes can be made by persons of ordinary skill in the art, which should be included in the scope of the application as defined by the following claims.
Claims
1. A host log data threat detection model training method, characterized in that: include: Constructing multiple original feature graphs based on log data corresponding to different time periods in a pre-acquired historical host log data set, wherein the original feature graphs include multiple original edges and operation types corresponding to each of the original edges; Obtaining a corresponding edge fusion feature map according to each of the original feature maps, wherein the edge fusion feature map includes a plurality of aggregation operation edges; Obtaining, according to each of the original feature graphs, an existence probability of each of the operation types in each of the aggregated operation edges; A preset machine learning network architecture is trained based on each log data in the historical host log data set and a plurality of corresponding existence probability labels to obtain a corresponding host log data threat detection model; The obtaining, according to each of the original feature graphs, the existence probability of each of the operation types in each of the aggregated operation edges includes: Extracting multiple node features from each of the original feature graphs; Inputting each of the node features in each of the original feature maps and the corresponding edge fusion feature maps into a graph neural network to update each of the nodes in each of the edge fusion feature maps, wherein the node includes a source node and a destination node; Extracting features of each of the aggregation operation edges and setting corresponding fusion edge labels for each of the aggregation operation edges; splicing the updated source node, the destination node, and the features of each aggregation operation edge in each edge fusion feature graph to obtain a representation vector; The representation vectors and the fused edge labels corresponding to each of the aggregated operation edges in each of the edge fusion feature graphs are input into a fully connected layer model to output the existence probability of each of the operation types in each of the aggregated operation edges.
2. The host log data threat detection model training method according to claim 1, characterized in that: The method constructs multiple original feature graphs based on the log data corresponding to different time periods in the pre-acquired historical host log data set, including: Dividing each log data in the host log data set into a plurality of time window log data sets according to a preset time window; The corresponding original feature graphs are constructed according to the respective time window log data sets.
3. The host log data threat detection model training method according to claim 1, characterized in that: The step of obtaining the corresponding edge fusion feature maps according to the original feature maps includes: Merge the outgoing edges corresponding to the single or multiple operation types between each node in each of the original feature graphs and another arbitrary node into an aggregated operation edge; According to the multiple aggregation operation edges and multiple nodes corresponding to each of the original feature maps, the edge fusion feature maps corresponding to each of the original feature maps are respectively constructed.
4. The host log data threat detection model training method according to claim 1, characterized in that: The method of training a preset machine learning network architecture based on each log data and its corresponding existence probability label in the historical host log data set to obtain a corresponding host log data threat detection model includes: A preset machine learning network architecture is trained according to each log data in the historical host log data set and each corresponding existence probability label, and based on a preset loss function, the corresponding loss probability value is calculated according to each existence probability and each corresponding existence probability label during the training process, and the machine learning network architecture is iteratively trained based on the loss probability value to generate a host log data threat detection model for outputting the threat existence probability according to the host log data set.
5. A host log data threat detection method, characterized in that: include: Obtain the target host log data set within a preset time length; The target host log data set is input into a host log data threat detection model pre-trained based on the host log data threat detection model training method described in any one of claims 1-4, so that the host log data threat detection model outputs the existence probability corresponding to each operation record in each log data in the target host log data set.
6. The log data threat detection method according to claim 5, characterized in that: After inputting the target host log data set into a host log data threat detection model pre-trained based on the host log data threat detection model training method according to any one of claims 1 to 4, so that the host log data threat detection model outputs the existence probability corresponding to each operation record in each log data in the target host log data set, the method further includes: Each of the existence probabilities is judged. If the existence probability is greater than or equal to the preset safety threshold, the operation record corresponding to the existence probability is determined to be a normal operation. If the existence probability is less than the safety threshold, the operation record corresponding to the existence probability is determined to be an abnormal operation.
7. A host log data threat detection model training device, characterized in that: include: An original feature graph construction module is used to construct multiple original feature graphs based on the log data corresponding to different time periods in the pre-acquired historical host log data set. The original feature graphs include multiple original edges and the operation type corresponding to each original edge; An edge fusion feature map construction module is configured to obtain corresponding edge fusion feature maps according to each of the original feature maps, wherein the edge fusion feature map includes a plurality of aggregation operation edges; a threat calculation module, configured to obtain, according to each of the original feature graphs, an existence probability of each of the operation types in each of the aggregated operation edges; A model building module is used to train a preset machine learning network architecture based on each log data in the historical host log data set and the corresponding multiple existence probability labels to obtain a corresponding host log data threat detection model; The obtaining, according to each of the original feature graphs, the existence probability of each of the operation types in each of the aggregated operation edges includes: Extracting multiple node features from each of the original feature graphs; Inputting each of the node features in each of the original feature maps and the corresponding edge fusion feature maps into a graph neural network to update each of the nodes in each of the edge fusion feature maps, wherein the node includes a source node and a destination node; Extracting features of each of the aggregation operation edges and setting corresponding fusion edge labels for each of the aggregation operation edges; splicing the updated source node, the destination node, and the features of each aggregation operation edge in each edge fusion feature graph to obtain a representation vector; The representation vectors and the fused edge labels corresponding to each of the aggregated operation edges in each of the edge fusion feature graphs are input into a fully connected layer model to output the existence probability of each of the operation types in each of the aggregated operation edges.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the host log data threat detection model training method according to any one of claims 1 to 4, or implements the log data threat detection method according to any one of claims 5-6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the host log data threat detection model training method as described in any one of claims 1 to 4, or implements the log data threat detection method as described in any one of claims 5-6.