A network data graphing and threat ad hoc analysis method

Through real-time sniffing and data modeling, the graphical and threat improper analysis of network data without relying on graph databases is realized, and the problems of low efficiency and high threshold of data analysis in the existing technology are solved, real-time graphical analysis and threat prompts are realized. .

CN118673185BActive Publication Date: 2024-11-12科来网络技术股份有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411147618.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-21
Publication Date
2024-11-12
Estimated Expiration
2044-08-21

AI Technical Summary

Technical Problem

The existing technology cannot meet the real-time analysis needs in the data graph database, the data analysis efficiency is low, the homologous node aggregation is difficult, the data usage threshold is high, and the network threat cannot be sensed in real time.

Method used

Through the traffic probe, data modeling and graph construction are carried out in real time, and the same-original nodes and relationships are aggregated to realize graphical analysis of zero codes, and threat analysis is carried out in combination with preset alert rules.

Benefits of technology

Real-time graphical analysis of graph-free databases is realized, which reduces technical thresholds, improves data mining efficiency and intuitive network risk prompt capabilities, and facilitates attack timing and link sorting.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118673185B_ABST
    Figure CN118673185B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for network data graphing and ad hoc threat analysis, comprising: S1: data capture; S2: data modeling, modeling the network data obtained in step S1 according to its data characteristics, and expressing the data characteristics as a model that can be recognized and converted by a program; S3: data processing and graph construction, standardizing the data obtained in step S1, processing the data into a general type of structured data, and constructing a graph for the structured data based on the data model configured in step S2; S4: graph aggregation, aggregating the graph nodes or graph relationships of the same source in the graph obtained in step S3; S5: ad hoc threat analysis, based on the entire graphed graph of the corresponding batch of structured data obtained in step S4, correlating and analyzing and displaying the alarms triggered by the network traffic through preset alarm rules, or threat logs from other sources with the graph nodes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of network security and data graphic structure, and in particular relates to a network data graphic and threat ad hoc analysis method. Background Art

[0002] Import the data into a traditional graph database, such as Neo4j. Then filter out some data logic information through cql statements; see the published patent: A big data storage method based on Neo4j graph database—CN108170847B.

[0003] For the above technical solution, it is necessary to rely on a graph database and then store all the data in the database, but the aggregation of many homologous nodes in the data cannot be met; the logical combing of different data or different dimensions of the same data requires redesign and functional development; the overall efficiency of data analysis is low and cannot meet the needs of real-time analysis; when using data, professional experience in using cql statements is required, and the threshold for using technical products is high.

[0004] The graph data obtained in step 1 of the above technical solution is static and one-time. It is impossible to perceive the immediate threat situation in real time. The data is too monotonous and not rich enough in content, making it inconvenient to conduct data mining. Summary of the invention

[0005] The purpose of the present invention is to overcome the problems of the prior art and disclose a method for graphing network data and ad hoc threat analysis, which is independent of conventional graph databases and can simultaneously perform real-time analysis and intuitively display the internal logical relationship graph of data.

[0006] The object of the present invention is achieved through the following technical solutions:

[0007] A network data graphing and threat ad hoc analysis method, the network data graphing and threat ad hoc analysis method comprising the following steps:

[0008] S1: Data capture, real-time sniffing through traffic probes, and extracting network data packet feature information; or network structured data from other sources;

[0009] S2: Data modeling: Based on the network data obtained in step S1, modeling is performed according to its data characteristics, and the data features are expressed as a model that can be recognized and converted by the program;

[0010] S3: Data processing and graph construction,

[0011] The data acquired in step S1 is standardized and processed into structured data of a common type, and a graph of the structured data is constructed based on the data model configured in step S2;

[0012] S4: Graph aggregation, aggregating the homologous graph nodes or graph relationships in the graph obtained in step S3 to reduce the number of graph nodes and graph relationships, thereby realizing the graphing of the shape and shadow network data.

[0013] According to a preferred embodiment, S5: ad hoc threat analysis, based on the entire graphical map of the corresponding batch of structured data obtained in step S4, the alarms triggered by the network traffic through preset alarm rules, or threat logs from other sources are correlated and analyzed and displayed with the map nodes.

[0014] According to a preferred embodiment, the data modeling method in step S2 includes:

[0015] S21: Create a graph node model, and select the column to be analyzed in the data structure of step S1 as the key value of the graph node model;

[0016] S24: Create a graph relationship model, and complete the pointing and belonging relationship association of each graph node in the graph node model through the graph relationship model.

[0017] According to a preferred embodiment, S22: configure the style and preset operations for the graph node model created in step S21, wherein the style is configured to enable subsequent businesses to personalize the display of graph nodes generated according to the graph node model, and the preset operations are used to implement business processing for the corresponding batches of graph nodes.

[0018] According to a preferred embodiment, the data modeling method in step S2 further includes: S23: adding graph node model attributes, so that the graph nodes generated according to the graph node model can express more attribute information.

[0019] According to a preferred embodiment, in step S3, constructing a graph of structured data includes:

[0020] Read a row of structured data records, and extract the value of the specified column from the structured data as the graph node key value according to the configured graph node models;

[0021] Based on the configured graph relationship model, a graph relationship is established for all generated graph nodes, thereby converting a row of structured records into a graph structure type. Then, the process is looped over all the data obtained in step S1 to complete the structural conversion of all data and realize the construction of a complete graph.

[0022] According to a preferred embodiment, the graph aggregation in step S4 includes:

[0023] S41: Based on the graph nodes and graph relations obtained in step S3, extract the key values ​​of the corresponding graph nodes and graph relations, and form a hash index;

[0024] S42: If it is found that the currently generated graph node / graph relationship key value already has a hash index, that is, the graph node / graph relationship already has a homologous object, then the condition for data aggregation exists;

[0025] S43: Aggregate objects of the same origin.

[0026] According to a preferred implementation, step S43 specifically includes: aggregating the attributes in each homologous object respectively, and finally forming each corresponding homologous object into one object, so as to reduce the number of graph nodes and graph relationships.

[0027] According to a preferred embodiment, step S5 of ad hoc threat analysis includes: matching the graph nodes of the graph obtained in S4 with the threat source and purpose, instantly marking the graph data in time series, and changing the color or style of the corresponding node to remind the user of system risks.

[0028] According to a preferred embodiment, in step S5 threat ad hoc analysis: for the hit node, it can be known that a network security risk has occurred, and the corresponding node attribute is immediately modified, and the threat unique identifier and occurrence time are recorded;

[0029] The styles of the corresponding nodes and node relationship edges are modified to indicate that a security risk has occurred at the node. Based on the unique threat identifier and occurrence time, the user can observe the occurrence of the risk and the risk propagation path in real time, and perform data mining through the unique threat identifier.

[0030] The aforementioned main scheme of the present invention and its further options can be freely combined to form multiple schemes, all of which are schemes that can be adopted and claimed for protection by the present invention. After understanding the scheme of the present invention, those skilled in the art can understand that there are multiple combinations based on the prior art and common knowledge, all of which are technical schemes to be protected by the present invention, and they are not exhaustively listed here.

[0031] Beneficial effects of the present invention:

[0032] The network data graphing and threat ad hoc analysis method of the present invention performs data modeling based on the constraints and integrity of the structured data itself, and then graphs the structured data. This method can achieve flexible and fast graph analysis of different dimensions of the same batch of data with zero code, while traditional graph databases are relatively bulky and some functions need to be reprogrammed to implement.

[0033] The network data graphing and threat ad hoc analysis method of the present invention relies on the graphed network graph combined with the real-time triggered network attack or network threat data, which can more intuitively and quickly prompt network risks. It is also more convenient to sort out the attack sequence and attack link, as well as to directly collect evidence and mine on the graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1It is a flowchart of an invented network data graphing and threat ad hoc analysis method. DETAILED DESCRIPTION

[0035] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0036] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0037] refer to Figure 1 As shown, a method for network data graphing and ad hoc threat analysis is shown in the figure, and the method for network data graphing and ad hoc threat analysis includes the following steps.

[0038] Step S1: Data capture. The traffic probe sniffs in real time and extracts network data packet feature information; or other source network structured data can be involved. This step lays the data foundation for subsequent data modeling.

[0039] Step S2: Based on the network data obtained in step 1, model the data characteristics and express the data features as a model that can be recognized and converted by the program.

[0040] The specific method of data modeling is as follows:

[0041] a) Create a graph node model: Select the columns to be analyzed in the data structure in step S1 as the key value of the model, that is, one or several columns of structured data are used to build the graph node model. Subsequent businesses generate graph nodes based on these graph node models.

[0042] b) Configure the style and specific operations for the graph node model in step a). The style is configured so that subsequent businesses can display the graph nodes generated based on this graph node model in a personalized way, and the specific operations are used to perform business processing on this batch of graph nodes, such as batch analysis, misjudgment identification, data mining, etc.

[0043] c) Adding attributes to the graph node model: This step is done after the above steps a) and b) are completed. In order to enrich the connotation of the graph node model, the graph nodes generated based on this graph node model can express more information. That is, the generated graph nodes are allowed to carry other field information in addition to their own display values.

[0044] For example, a communication node can carry information such as the communication initiation time, duration, and communication traffic in addition to the communication IP. It also allows the aggregation of specified attributes, such as the accumulation of communication traffic in multiple structured records, the earliest communication time, and so on.

[0045] d) Create a graph relationship model: After completing the above steps a), b), and c), the configuration of the graph node model is completed. At this time, the system can convert the structured data obtained in step S1 into several graph nodes based on the configured graph node model. However, the relationship between these graph nodes cannot be expressed yet, and graph node relationships are required to unite all graph nodes. Therefore, the graph relationship model needs to be configured. Subsequently, the graph relationship is generated based on the graph relationship model.

[0046] The graph relationship model depends on the graph node model, that is, it depends on the above steps a), b), and c). After the graph node model is configured, the graph relationship model is configured. The graph relationship model depends on at least two graph node models, that is, the graph relationship model is a kind of establishment of the direction and belonging of the graph node model.

[0047] The specific method of establishing the graph relationship model is: perform the above steps a), b), and c) multiple times (at least 2 times) to obtain two or more graph node models. Determine the direction and ownership between the two graph node models. For example, after forming two graph node models A and B, A->B (A points to B) or B->A (B points to A) or A<->B (A points to B, and B also points to A). This direction and ownership relationship between graph node models is the graph relationship model. The key value of the graph relationship model is the key-value pair of the graph node model that the relationship points to and belongs to.

[0048] The graph relationship model is similar to the graph node model and allows the existence of attributes belonging to the graph relationship model. The reason and configuration method are the same as step c).

[0049] Step S3: Data processing and map construction.

[0050] The data obtained in step S1 is standardized and processed into a common type, such as JSON type, or a standardized functional interface is implemented to customize and extend other standard data types. This step provides a standard data format for subsequent graph construction.

[0051] According to the data model configured in step S2 above, start to build a graph for structured data: the specific method is as follows:

[0052] i. Read a row of structured data records (processed into a standard format), and extract the value of the specified column from the structured data as the graph node key value according to all the configured graph node models. How many graph nodes can be generated from a row of data depends on how many degree node models are configured, but as long as there are two graph node models.

[0053] ii. After process i is completed, at least 2 graph nodes will be generated. At this time, according to the graph relationship model configured in process d of step 2, a graph relationship is established for all generated graph nodes. At this moment, a row of structured records has been converted into a graph structure type. Subsequently, all data obtained in step 1 are looped and executed to complete the structure conversion of all data.

[0054] Step S4: Graph aggregation.

[0055] After completing all the processes in step S3, the graphing process of a row of structured data can be completed. That is, multiple graph nodes and graph node relationships can be obtained through a row of structured data.

[0056] However, in data flow analysis and processing, there may be nodes with the same key value and graph relationships with the same key value. In this case, these graph nodes or graph relationships of the same source need to be aggregated to reduce the number of graph nodes and graph relationships.

[0057] Specific aggregation method:

[0058] 1) In the data analysis process, according to all the processes of step S3, multiple graph nodes and graph relationships are obtained, and the key values ​​of the generated graph nodes and graph relationships are extracted to form a hash index.

[0059] 2) In the subsequent data analysis process, if it is found that the hash index already exists for the key value of the currently generated graph node / graph relationship, that is, the graph node / graph relationship already has a homologous object, so the conditions for data aggregation exist.

[0060] 3) Aggregate homologous objects, that is, aggregate the attributes in the object. At this time, according to the configuration of step c) in step S2, aggregate homologous attributes (homologous attributes are attributes with the same attribute name) and finally form an object. All graph nodes and graph relationships are performed accordingly.

[0061] Step S5: Ad hoc analysis.

[0062] After the above steps S1 to S4, the processing of structured data to graph structure has been completed, and all graph nodes and graph relationship data will be obtained, so as to obtain the entire graph of the batch of structured data. On this basis, the alarms triggered by the preset alarm rules of network traffic or threat logs from other sources are correlated with the graph nodes for analysis and display.

[0063] Specifically, by matching the threat source and purpose, S4 obtains the graph nodes of the graph, instantly marks the graph data in time series, and changes the color or style of the corresponding node to remind the user of system risks.

[0064] Whether the IP pairs in the threat log hit the graph structure data, for the nodes that hit, it can be known that a network security risk has occurred, and the node attributes are modified immediately, the threat unique identifier and occurrence time are recorded, and the style of the network node and node relationship edge can be modified to identify the node that has a security risk, and based on the threat unique identifier and occurrence time. It is convenient for users to observe the occurrence of risks and the risk propagation path in real time. And convenient data mining can be carried out through the threat unique identifier.

[0065] The network data graphing and threat ad hoc analysis method of the present invention relies on the graphed network graph combined with the real-time triggered network attack or network threat data, which can more intuitively and quickly prompt network risks. It is also more convenient to sort out the attack sequence and attack link, as well as to directly collect evidence and mine on the graph.

[0066] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A network data graphing and threat ad hoc analysis method, characterized in that: The network data graphing and threat ad hoc analysis method comprises the following steps: S1: Data capture, real-time sniffing through traffic probes, and extracting network data packet feature information; or network structured data from other sources; S2: Data modeling: Based on the network data obtained in step S1, modeling is performed according to its data characteristics, and the data features are expressed as a model that can be recognized and converted by the program; The data modeling method in step S2 includes: S21: Create a graph node model, and select the column to be analyzed in the data structure of step S1 as the key value of the graph node model; S22: configuring a style and preset operations for the graph node model created in step S21, wherein the style is configured so that subsequent businesses can display graph nodes generated according to the graph node model in a personalized manner, and the preset operations are used to implement business processing for corresponding batches of graph nodes; S23: adding graph node model attributes, so that the graph nodes generated according to the graph node model can express more attribute information; S24: creating a graph relationship model, and completing the pointing and belonging relationship association of each graph node in the graph node model through the graph relationship model; S3: Data processing and graph construction, The data acquired in step S1 is standardized and processed into structured data of a common type, and a graph of the structured data is constructed based on the data model configured in step S2; S4: Graph aggregation, aggregating the graph nodes or graph relationships of the same source in the graph obtained in step S3 to reduce the number of graph nodes and graph relationships, thereby realizing the graphing of the shape and shadow network data; The graph aggregation in step S4 includes: S41: Based on the graph nodes and graph relations obtained in step S3, extract the key values ​​of the corresponding graph nodes and graph relations, and form a hash index; S42: If it is found that the currently generated graph node / graph relationship key value already has a hash index, that is, the graph node / graph relationship already has a homologous object, then the condition for data aggregation exists; S43: Aggregate objects of the same origin; Step S43 specifically includes: aggregating the attributes in each homologous object respectively, and finally forming each corresponding homologous object into one object to reduce the number of graph nodes and graph relationships.

2. The network data graphing and threat ad hoc analysis method according to claim 1, characterized in that: It also includes S5: ad hoc threat analysis, based on the entire graphical map of the corresponding batch structured data obtained in step S4, the alarms triggered by the network traffic through preset alarm rules, or threat logs from other sources are correlated and analyzed and displayed with the map nodes.

3. The network data graphing and threat ad hoc analysis method according to claim 1, characterized in that: In step S3, constructing a graph of structured data includes: Read a row of structured data records, and extract the value of the specified column from the structured data as the graph node key value according to the configured graph node models; Based on the configured graph relationship model, a graph relationship is established for all generated graph nodes, thereby converting a row of structured records into a graph structure type. Then, the process is looped over all the data obtained in step S1 to complete the structural conversion of all data and realize the construction of a complete graph.

4. The network data graphing and threat ad hoc analysis method according to claim 2, characterized in that: Step S5 of ad hoc threat analysis includes: matching the graph nodes of the graph obtained in S4 with the threat source and purpose, marking the graph data in time series immediately, and changing the color or style of the corresponding node to remind the user of system risks.

5. The network data graphing and threat ad hoc analysis method according to claim 4, characterized in that: Step S5: Threat ad hoc analysis: For the hit nodes, it is known that a network security risk has occurred, and the corresponding node attributes are modified immediately to record the unique threat identifier and occurrence time; The styles of the corresponding nodes and node relationship edges are modified to indicate that a security risk has occurred at the node. Based on the unique threat identifier and occurrence time, the user can observe the occurrence of the risk and the risk propagation path in real time, and perform data mining through the unique threat identifier.