Communication topology information modeling method and device based on collaborative flow monitoring

By preprocessing and merging network traffic data and generating a materialized database view, the problem of low data retrieval efficiency of relational databases in communication graph data is solved, efficient single-table query is realized, and multi-dimensional query requirements for collaborative traffic monitoring is met.

CN119030879AActive Publication Date: 2024-11-26NAT IND INFORMATION SECURITY DEV RES CENT
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411029574.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-26
Estimated Expiration
2044-07-30

AI Technical Summary

Technical Problem

In the prior art, the retrieval efficiency of communication graph data based on relational database modeling is low and cannot meet the efficient query requirements of collaborative traffic monitoring, especially when the data volume is large.

Method used

By cached the collected network traffic data into memory, preprocessing and merging, data objects that meet database requirements are generated, and database materialized view processing is performed on the communication link table, reducing the dimensions into single table query, improving retrieval efficiency.

Benefits of technology

It realizes efficient communication topology information modeling, improves query efficiency, meets the multi-dimensional graph query needs, and solves the efficiency problem of traditional relational databases under large-scale data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119030879A_ABST
    Figure CN119030879A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of computers, and discloses a communication topology information modeling method and device based on collaborative flow monitoring, and the method comprises the steps: caching collected network flow data into a memory, and carrying out the preprocessing of the network flow data, and obtaining a network flow attribute corresponding to the network flow data; combining the network flow data with the same network flow attribute acquired in a preset time period in real time to obtain combined network flow data, and analyzing the combined network flow data to obtain initial network flow data; carrying out modeling processing on the initial network flow data, generating a data object meeting a database requirement, storing the data object into a database, and generating a communication link table; and performing database materialized view processing on the communication link table to generate a database physical view. By applying the method, the retrieval efficiency can be effectively improved, the efficient query requirement is met, and the multi-dimensional graph query requirement is met.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and in particular to a communication topology information modeling method based on collaborative traffic monitoring. Background Art

[0002] With the expansion of the scale of the dual-network integration of industrial control networks and office networks, how to ensure the real-time display and retrieval of collaborative traffic network communication information is a challenge. When the nodes in the collaborative traffic monitoring system reach a certain scale, the use of traditional relational databases to model traffic data usually cannot meet the requirements of real-time and efficient data retrieval.

[0003] In order to solve the above problems, professional third-party graph databases are usually used to store graph data. However, third-party graph databases usually take up a large amount of resource space and their insertion efficiency for large-volume communication data is significantly inferior to that of relational databases. In addition, although third-party graph databases are professional graph data carriers, their support for professional communication data is not sufficient. Therefore, there are currently methods of using relational databases to model graph data, but their query efficiency is still a flaw.

[0004] In view of this, how to provide a solution for modeling communication graph data based on relational database to meet the efficient query requirements of collaborative traffic monitoring and at the same time meet the multi-dimensional graph query needs has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] In view of this, an embodiment of this specification provides a communication topology information modeling method based on collaborative traffic monitoring. One or more embodiments of this specification also involve a communication topology information modeling device based on collaborative traffic monitoring, a computing device, and a computer-readable storage medium to solve the problem of retrieval efficiency of communication graph data modeling based on relational database in the prior art, and reduce complex statistical queries and logical processing to single table queries.

[0006] According to a first aspect of an embodiment of this specification, a communication topology information modeling method based on collaborative traffic monitoring is provided, comprising:

[0007] The collected network flow data is cached in a memory, and the network flow data is preprocessed to obtain network flow attributes corresponding to the network flow data, wherein the network flow data includes: industrial control network flow data and intranet flow data;

[0008] Merging the network traffic data with the same network traffic attributes collected within a preset time period in real time to obtain merged network traffic data, and parsing and processing the merged network traffic data to obtain initial network traffic data, wherein the initial network traffic data includes: communication node data, communication link data and device application data;

[0009] Modeling the initial network traffic data to generate data objects that meet database requirements, and storing the data objects in the database to generate a communication link table, wherein the data objects include: communication node objects, communication link objects, and device application objects;

[0010] Database materialized view processing is performed on the communication link table to generate a database physical view.

[0011] According to a second aspect of an embodiment of this specification, a communication topology information modeling device based on collaborative traffic monitoring is provided, comprising:

[0012] The collection module is configured to cache the collected network flow data into a memory, and pre-process the network flow data to obtain network flow attributes corresponding to the network flow data, wherein the network flow data includes: industrial control network flow data and intranet flow data;

[0013] A merging module is configured to merge the network traffic data with the same network traffic attribute collected within a preset time period in real time to obtain merged network traffic data, and parse and process the merged network traffic data to obtain initial network traffic data, wherein the initial network traffic data includes: communication node data, communication link data and device application data;

[0014] A modeling module is configured to perform modeling processing on the initial network traffic data, generate data objects that meet the database requirements, store the data objects in the database, and generate a communication link table, wherein the data objects include: communication node objects, communication link objects, and device application objects;

[0015] The materialization module is configured to perform database materialized view processing on the communication link table to generate a database physical view.

[0016] According to a third aspect of an embodiment of this specification, a computer device is provided, including:

[0017] Memory and processor;

[0018] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned communication topology information modeling method based on collaborative traffic monitoring are implemented.

[0019] According to a fourth aspect of an embodiment of the present specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned communication topology information modeling method based on collaborative traffic monitoring are implemented.

[0020] The present application provides a communication topology information modeling method based on collaborative traffic monitoring, comprising: first, caching the collected network traffic data into a memory, and preprocessing the network traffic data to obtain network traffic attributes corresponding to the network traffic data, wherein the network traffic data includes: industrial control network traffic data and intranet traffic data; then, merging the network traffic data with the same network traffic attributes collected within a preset time period in real time to obtain merged network traffic data, and parsing the merged network traffic data to obtain initial network traffic data, wherein the initial network traffic data includes: communication node data, communication link data and device application data; secondly, modeling the initial network traffic data to generate data objects that meet database requirements, and storing the data objects in the database to generate a communication link table, wherein the data objects include: communication node objects, communication link objects and device application objects; finally, performing database materialized view processing on the communication link table to generate a database physical view.

[0021] The communication topology information modeling method based on collaborative traffic monitoring provided in the embodiment of the present application is applied. First, the traffic data of the industrial control network and the office intranet engine are collected, and the traffic data belonging to the same link are merged; then the collected data is processed and abstractly modeled as a communication object that can be entered into the database; secondly, based on the communication object, a communication link table is generated; finally, the communication link table is obtained by performing database materialized view processing on the communication link table, and a single-table query of the topology information is realized, thereby effectively improving the retrieval efficiency to meet the requirements of efficient query, while meeting the multi-dimensional graph query requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flow chart of a communication topology information modeling method based on collaborative traffic monitoring provided in an embodiment of the present application;

[0023] Figure 2 It is a schematic diagram of the process structure of a communication topology information modeling method based on collaborative traffic monitoring provided in an embodiment of the present application;

[0024] Figure 3 It is a structural schematic diagram of a communication topology information modeling device based on collaborative traffic monitoring provided in an embodiment of the present application;

[0025] Figure 4It is a structural block diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0027] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0028] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0029] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0030] Currently, there are two common basic model structures for the relationship database data model used to display topological graphs: the first model structure includes: communication source, communication destination, communication direction (with server-client logo), other communication information, etc.; the second model structure includes: communication server, communication client, communication direction, other communication information, etc. The advantages of these two common data models are that they are suitable for communication log records, the information is relatively intuitive, their structure is close to the data collection structure, and they are easy to parse and store. When using a graph database to store communication graph data, its basic structure includes: communication nodes and directed relationships. Other detailed information is included in the node and relationship objects, so it only plays a role in filtering data when querying graph data.

[0031] The most obvious disadvantage of the existing relational database communication graph data model is that the graph query efficiency is poor and it is only suitable for displaying data in a table format. This is because the topology graph data ultimately needs to summarize the directed relationship between the communication nodes. The two common models are slow in summarizing communication nodes and directed relationships. Because the nodes at both ends of the communication are the same but the sessions are different, the column stored in the database for the same node is uncertain. In order to comprehensively and correctly count the complex and diverse communication data, multiple queries, joint queries, or query the data into the program memory and then summarize and calculate through the program code. In addition, the uncertainty of the columns increases the complexity of the final directional summary statistics between nodes. This complexity is easily reflected in the communication data of devices that are mutually source and destination or mutually server and client in the communication; on the other hand, due to the existence of multiple IP devices in the communication network, if these two models want to calculate the directed relationship between multiple IP assets, they must perform an associated query on the device data to obtain the directed relationship. For relational databases, single-table query is the preferred efficiency, especially when the data volume is large. The efficiency gap is obvious.

[0032] Therefore, an embodiment of the present application proposes a communication topology information modeling method based on collaborative traffic monitoring, which is used to solve the problem of retrieval efficiency of communication graph data modeling based on relational database, and reduce complex statistical queries and logical processing to single-table queries.

[0033] In the present application, a method for modeling communication topology information based on collaborative traffic monitoring is provided. The present specification also relates to a device for modeling communication topology information based on collaborative traffic monitoring, a computer device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0034] See also Figure 1 , Figure 1 1 is a flow chart of a communication topology information modeling method based on collaborative traffic monitoring provided by an embodiment of the present application. Figure 1 As shown, the specific steps include the following steps.

[0035] Step S102: Cache the collected network traffic data into the memory, and pre-process the network traffic data to obtain network traffic attributes corresponding to the network traffic data, wherein the network traffic data includes: industrial control network traffic data and intranet traffic data.

[0036] Step S104: real-time merge the network traffic data with the same network traffic attributes collected within a preset time period to obtain merged network traffic data, and parse and process the merged network traffic data to obtain initial network traffic data, wherein the initial network traffic data includes: communication node data, communication link data and device application data.

[0037] Step S106: Modeling the initial network traffic data to generate data objects that meet database requirements, and storing the data objects in the database to generate a communication link table, wherein the data objects include: communication node objects, communication link objects and device application objects.

[0038] In the embodiment of the present application, it also includes: sending the obtained communication link table to the user end for display through the retrieval interface.

[0039] Step S108: performing database materialized view processing on the communication link table to generate a database physical view.

[0040] In the embodiment of the present application, the method further includes: in response to receiving a communication topology information query request sent by a client, and monitoring that the communication topology information query request triggers a call to a topology interface instruction, starting a view refresh switch, refreshing the database physical view in real time, and generating target communication topology information. Because the database physical view is only useful when the user is querying the topology, the embodiment of the present application uses a switch to control the view refresh, and when a user calls a topology-related interface, the view refresh switch is enabled to find a balance between performance and data real-time, thereby improving data effectiveness while ensuring a high query rate of the materialized view, reducing the high computing pressure of the database due to frequent refreshes, and reducing energy consumption.

[0041] It should be noted that the database here is the PostgreSQL database.

[0042] The embodiment itself performs data persistence processing on the data obtained after abstract modeling, that is, stores the modeled objects in a graph database.

[0043] In an embodiment of the present application, the network traffic data is preprocessed to obtain network traffic attributes corresponding to the network traffic data, including: obtaining data feature information corresponding to each communication link in the network traffic data, wherein the data feature information is a unique identifier for each communication link, and the data feature information includes: server IP, server mac, server port, client IP, client mac, client port, link session direction and application layer protocol; merging network traffic data corresponding to communication links with the same data feature information, and using the data feature information corresponding to the communication links as the network traffic attributes corresponding to the network traffic data.

[0044] In actual applications, the server IP, server MAC, server port, client IP, client MAC, client port, link session direction, and application layer protocol are used as unique identifiers of the link, and other attributes are combined and calculated for the same link.

[0045] After obtaining the target traffic data, the communication node object, communication link object and device application object are obtained by parsing the target traffic data. The communication node object and communication link object are used for the subsequent construction of the communication link table and the display of the communication link diagram; the device application object is used to analyze the application program of the device.

[0046] In the embodiment of the present application, the corresponding relationship between the encapsulated object and the original data is: the communication link is composed of the application data of each device, wherein the device application data is composed of the data of each communication device.

[0047] Specifically, the main structure of the object encapsulated by the original data includes: communication equipment, device application and communication topology data, among which the device application data is used to analyze and record the communication status of the application installed on the device; the communication topology data is used for preliminary analysis of the back-link object.

[0048] After completing the modeling of the communication object, the modeled communication object is stored in the graph database. The specific implementation is as follows.

[0049] In actual applications, if we look at the direction of topological display, the three core fields of storing the device ID and communication direction at both ends of the communication can display the topological relationship diagram. In order to achieve a richer display effect and support the display of communication details, the embodiment of the present application selects the device ID, ip, mac, port, communication direction, and communication protocol at both ends of the communication as the joint unique key to aggregate the storage link data. The storage of the communication direction uses -1, 0, and 1 to mark the direction, abandoning the server-client concept of the direction in the source data, so that it only represents the direction. It should be noted that the direction here represents the direction of the device in column 1 to the device in column 2, which needs to be calculated by the source data.

[0050] The storing of the data object into a database and generating a communication link table comprises: in response to the communication node object not existing in the database, adding the communication device corresponding to the communication node object into the database; in response to the communication link object not existing in the database, adding the communication link data corresponding to the communication link object into the database based on the insert update syntax of the database.

[0051] In the embodiment of the present application, based on the insert update syntax of the database, the communication link data corresponding to the communication link object is added to the database, including: adding the communication link object to the database by merging, inserting, modifying, etc. In practical applications, if the communication device object does not exist, it is newly added to the database; for the communication link object, the insert update syntax ON CONFL I CT DO UPDATE of the database is used to merge the insertion and modification steps to improve the processing efficiency, and the joint unique key of the topology table is used to ensure the uniqueness of the same link data in the database.

[0052] Specifically, adding the communication link data corresponding to the communication link object to a database includes:

[0053] Using the joint unique key corresponding to the communication link table, determine that there are no other communication link objects identical to the communication link object in the database, wherein the joint unique key is determined based on the server IP, server mac, server port, client IP, client mac, client port, link session direction and application layer protocol.

[0054] More specifically, storing data objects that meet data requirements in a database includes: obtaining the first communication device ID and the second communication device ID at both ends of the communication link corresponding to the communication node object carried by the data object; determining the warehousing form corresponding to the first communication device ID and the second communication device ID based on a preset warehousing rule, wherein the preset warehousing rule is a rule that determines that the same communication link corresponding to the first communication device and the second communication device should remain consistent when warehousing based on the first communication device ID information and the second communication device ID information at both ends of the communication link; based on the warehousing form, storing the data object in the database.

[0055] In practical applications, storing more detailed data requires more consideration of query efficiency. To improve the statistical and query efficiency of topological data and avoid inefficient queries such as multiple queries and union queries, the link data is compared with the device IDs at both ends of the communication when entering the database, and the final storage form of the data is determined according to the specified storage rules to ensure that the link data with the same devices at both ends of the communication are in the same column when entering the database, so as to improve the efficiency of data aggregation calculation. It should be noted that the storage rules here can be the lexicographic order of the device ID or other easy-to-calculate sorting methods, and the direction represents the direction of the communication device corresponding to column 1 and the communication device corresponding to column 2, which can be calculated based on the source data.

[0056] Furthermore, after the modeled objects are stored in the database, a retrieval interface is provided for display on the web front end.

[0057] In the embodiment of the present application, performing database materialized view processing on the communication link table to generate a database physical view includes:

[0058] Determine the first communication device ID, the second communication ID and the communication direction based on the communication node objects and the communication link objects corresponding to each communication link in the communication link table;

[0059] Determine a topology direction by summarizing each of the communication directions in the communication link table;

[0060] A database physical view is generated based on the first communication device ID, the second communication device ID and the topology direction.

[0061] In order to improve the retrieval efficiency, a database materialized view is made for the communication link table. The communication link table stores each link session, which is the details of each direction line in the topology diagram. The direction line itself needs to be summarized and counted from the communication link table. The communication direction is stored as -1, 0, and 1, which makes it possible to calculate the final communication direction through view summary, because (-1) + 1 = 0, which means that one-way communications with different directions are combined into two-way communications, avoiding the situation where a large amount of data needs to be stored in memory to calculate the communication direction.

[0062] In an embodiment of the present application, the topology direction is determined by summarizing each of the communication directions in the communication link table, including: summarizing each of the communication directions in the communication link table through a sum function to determine a first topology direction; summarizing each of the communication directions in the communication link table through a counting function to determine a second topology direction; determining the topology direction based on the first topology direction and the second topology direction, wherein, when the topology direction is zero, the topology direction is determined to be bidirectional communication, and when the topology direction is non-zero, the topology direction is determined to be unidirectional communication.

[0063] Therefore, the aggregated calculation of the final direction is obtained by calculating the comparison value between the summation function sum(direction) and the counting function count(direction).

[0064] If abs(sum(direction)) < count(direction), that is, if the absolute value of the sum of directions is less than the number of directions, the final communication direction of the device is two-way 0.

[0065] In addition, if sum(direction) < 0, the final communication direction is -1, which is one-way communication.

[0066] If sum(direction) > 0, the final communication direction is 1, which is one-way communication.

[0067] Through this algorithm, the topological direction can be directly calculated from the database.

[0068] The query efficiency of the materialized view is high while the data timeliness is low. To improve the data timeliness, the view needs to be refreshed when the source data changes. However, frequent view refreshing will increase the database calculation pressure and is also prone to waste. Especially when the data traffic is large, it will cause a large performance consumption.

[0069] By applying the method provided in the embodiment of the present application, the real-time traffic data of the industrial control network and the office intranet reported by the traffic collection engine is modeled in the form of graph data, so as to realize the function of viewing the real-time network communication topology diagram; the communication graph data is scientifically modeled based on the relational database, and the modeled data is persisted to the database, so as to solve the problem of drawing and retrieving efficiency of traffic topology diagram based on relational database. The method provided by the embodiment of the present application specifically includes the following beneficial effects: 1. Real-time drawing of the network communication topology diagram of the industrial control network and the office intranet is convenient for timely discovery of network risks and troubleshooting of network problems; 2. Using relational data to store topological data avoids the extremely low efficiency of entering large quantities of traffic data into graph databases such as neo4j, and scientifically designs data models to balance data entry efficiency and topological query efficiency; 3. Network nodes in the communication topology diagram support merging, support multi-IP devices, and are close to the real network environment; 4. All complex and diverse communication information between communication nodes is saved, and information such as IP, MAC, port, protocol, PPS, BPS, alarm, etc. can be viewed and retrieved in real time, and the information is more detailed; 5. Due to the detailed information storage and the support provided by the relational database (Postgres) for special functions of IP and MAC data types, efficient query support is provided for the display of topological diagrams from special perspectives; 6. Topological data is stored in the form of table (TABLE) + materialized view (MATERIALIZED VIEW), taking into account performance overhead and query efficiency.

[0070] See also Figure 2 , Figure 2 Schematic diagram of the process structure of a communication topology information modeling method based on collaborative traffic monitoring provided by an embodiment of the present application. Figure 2 As shown, the communication topology information modeling method provided in the embodiment of the present application is: first, industrial control network process collection and office network traffic collection are performed; then the traffic data is aggregated and cached in the memory, and the same links are merged at the same time; secondly, the traffic data in the cache is pulled every minute; secondly, the device data, the device application data and the topology data are parsed; secondly, the data objects are added in batches or matched and updated into the database; thirdly, it is determined whether the topological physical view of the database needs to be refreshed, and if so, the physical view is refreshed; if not, the parsing is confirmed to be complete.

[0071] Corresponding to the above method embodiment, this specification also provides a communication topology information modeling device embodiment based on collaborative traffic monitoring, Figure 3 Schematic diagram of a communication topology information modeling device based on collaborative traffic monitoring provided by an embodiment of the present application. Figure 3 As shown, the device comprises:

[0072] The collection module 302 is configured to cache the collected network flow data into the memory, and pre-process the network flow data to obtain the network flow attributes corresponding to the network flow data, wherein the network flow data includes: industrial control network flow data and intranet flow data;

[0073] The merging module 304 is configured to merge the network traffic data with the same network traffic attribute collected within a preset time period in real time to obtain merged network traffic data, and parse and process the merged network traffic data to obtain initial network traffic data, wherein the initial network traffic data includes: communication node data, communication link data and device application data;

[0074] The modeling module 306 is configured to perform modeling processing on the initial network traffic data, generate data objects that meet the database requirements, store the data objects in the database, and generate a communication link table, wherein the data objects include: communication node objects, communication link objects, and device application objects;

[0075] The materialization module 308 is configured to perform database materialized view processing on the communication link table to generate a database physical view.

[0076] In an optional embodiment, the acquisition module 302 is further configured to:

[0077] Obtain data feature information corresponding to each communication link in the network traffic data, wherein the data feature information is a unique identifier of each communication link, and the data feature information includes: server IP, server MAC, server port, client IP, client MAC, client port, link session direction and application layer protocol;

[0078] The network flow data corresponding to the communication links with the same data characteristic information are merged, and the data characteristic information corresponding to the communication links is used as the network flow attribute corresponding to the network flow data.

[0079] In an optional embodiment, the modeling module 306 is further configured to:

[0080] In response to the communication node object not existing in the database, adding the communication device corresponding to the communication node object to the database;

[0081] In response to the communication link object not existing in the database, the communication link data corresponding to the communication link object is added to the database based on the insert update syntax of the database.

[0082] In an optional embodiment, the modeling module 306 is further configured to:

[0083] Using the joint unique key corresponding to the communication link table, determine that there are no other communication link objects identical to the communication link object in the database, wherein the joint unique key is determined based on the server IP, server mac, server port, client IP, client mac, client port, link session direction and application layer protocol.

[0084] In an optional embodiment, the modeling module 306 is further configured to:

[0085] Acquire a first communication device ID and a second communication device ID at both ends of a communication link corresponding to the communication node object carried by the data object;

[0086] Based on the preset warehousing rules, determine the warehousing forms corresponding to the first communication device ID and the second communication device ID, wherein the preset warehousing rules are rules for determining that the same communication link corresponding to the first communication device and the second communication device is kept consistent when entering the warehousing according to the first communication device ID information and the second communication device ID information at both ends of the communication link;

[0087] Based on the storage form, the data object is stored in a database.

[0088] In an optional embodiment, the materialization module 308 is further configured to:

[0089] Determine the first communication device ID, the second communication ID and the communication direction based on the communication node objects and the communication link objects corresponding to each communication link in the communication link table;

[0090] Determine a topology direction by summarizing each of the communication directions in the communication link table;

[0091] A database physical view is generated based on the first communication device ID, the second communication device ID and the topology direction.

[0092] In an optional embodiment, the materialization module 308 is further configured to:

[0093] Summarizing the communication directions in the communication link table through a sum function to determine a first topology direction;

[0094] Summarizing each of the communication directions in the communication link table through a counting function to determine a second topology direction;

[0095] Based on the first topology direction and the second topology direction, a topology direction is determined, wherein when the topology direction is zero, the topology direction is determined to be bidirectional communication, and when the topology direction is non-zero, the topology direction is determined to be unidirectional communication.

[0096] In an optional embodiment, the modeling module 306 is further configured to:

[0097] The communication link table obtained is sent to the user end for display through the retrieval interface.

[0098] In an optional embodiment, the device further includes:

[0099] The refresh module is configured to respond to receiving a communication topology information query request sent by a client, and detect that the communication topology information query request triggers a topology interface instruction, start a view refresh switch, and refresh the database physical view in real time.

[0100] By using the device provided in the embodiment of the present application, the real-time traffic data of the industrial control network and the office intranet reported by the traffic collection engine is modeled in the form of graph data, so as to realize the function of viewing the real-time network communication topology map; the communication graph data is scientifically modeled based on the relational database, and the modeled data is persisted to the database, so as to solve the problem of drawing and retrieving efficiency of traffic topology map based on relational database.

[0101] The above is a schematic scheme of a communication topology information modeling device based on collaborative flow monitoring in this embodiment. It should be noted that the technical scheme of the communication topology information modeling device based on collaborative flow monitoring and the technical scheme of the communication topology information modeling method based on collaborative flow monitoring belong to the same concept. For details not described in detail in the technical scheme of the communication topology information modeling device based on collaborative flow monitoring, please refer to the description of the technical scheme of the communication topology information modeling method based on collaborative flow monitoring.

[0102] Figure 4 4 is a block diagram of a computing device 400 provided in an embodiment of the present application. The components of the computing device 400 include but are not limited to a memory 410 and a processor 420. The processor 420 is connected to the memory 410 via a bus 430, and the database 450 is used to store data.

[0103] The computing device 400 also includes an access device 440 that enables the computing device 400 to communicate via one or more networks 460. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 440 may include one or more of any type of network interface, wired or wireless (e.g., a network interface card (NIC)), such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0104] In one embodiment of the present specification, the above components of the computing device 400 and Figure 4 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 4 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0105] The computing device 400 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 400 may also be a mobile or stationary server.

[0106] The processor 420 is used to execute the following computer executable instructions, which, when executed by the processor, implement the steps of the above-mentioned communication topology information modeling method based on collaborative traffic monitoring.

[0107] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the communication topology information modeling method based on collaborative traffic monitoring belong to the same concept. For details not described in detail in the technical scheme of the computing device, please refer to the description of the technical scheme of the communication topology information modeling method based on collaborative traffic monitoring.

[0108] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned communication topology information modeling method based on collaborative traffic monitoring.

[0109] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the communication topology information modeling method based on collaborative traffic monitoring belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be found in the description of the technical scheme of the communication topology information modeling method based on collaborative traffic monitoring.

[0110] An embodiment of the present specification also provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned communication topology information modeling method based on collaborative traffic monitoring.

[0111] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the communication topology information modeling method based on collaborative traffic monitoring belong to the same concept, and the details not described in detail in the technical scheme of the computer program can be found in the description of the technical scheme of the communication topology information modeling method based on collaborative traffic monitoring.

[0112] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0113] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0114] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0115] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0116] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A communication topology information modeling method based on collaborative traffic monitoring, characterized in that: include: The collected network flow data is cached in a memory, and the network flow data is preprocessed to obtain network flow attributes corresponding to the network flow data, wherein the network flow data includes: industrial control network flow data and intranet flow data; Merging the network traffic data with the same network traffic attributes collected within a preset time period in real time to obtain merged network traffic data, and parsing and processing the merged network traffic data to obtain initial network traffic data, wherein the initial network traffic data includes: communication node data, communication link data and device application data; Modeling the initial network traffic data to generate data objects that meet database requirements, and storing the data objects in the database to generate a communication link table, wherein the data objects include: communication node objects, communication link objects, and device application objects; Database materialized view processing is performed on the communication link table to generate a database physical view.

2. The method according to claim 1, characterized in that Preprocessing the network traffic data to obtain network traffic attributes corresponding to the network traffic data includes: Obtain data feature information corresponding to each communication link in the network traffic data, wherein the data feature information is a unique identifier of each communication link, and the data feature information includes: server IP, server MAC, server port, client IP, client MAC, client port, link session direction and application layer protocol; The network flow data corresponding to the communication links with the same data characteristic information are merged, and the data characteristic information corresponding to the communication links is used as the network flow attribute corresponding to the network flow data.

3. The method according to claim 1, characterized in that: The storing of the data object into a database and generating a communication link table comprises: In response to the communication node object not existing in the database, adding the communication device corresponding to the communication node object to the database; In response to the communication link object not existing in the database, the communication link data corresponding to the communication link object is added to the database based on the insert update syntax of the database.

4. The method according to claim 3, characterized in that The step of adding the communication link data corresponding to the communication link object to a database comprises: Using the joint unique key corresponding to the communication link table, determine that there are no other communication link objects identical to the communication link object in the database, wherein the joint unique key is determined based on the server IP, server mac, server port, client IP, client mac, client port, link session direction and application layer protocol.

5. The method according to claim 1, characterized in that The storing the data object in a database comprises: Acquire a first communication device ID and a second communication device ID at both ends of a communication link corresponding to the communication node object carried by the data object; Based on the preset warehousing rules, determine the warehousing forms corresponding to the first communication device ID and the second communication device ID, wherein the preset warehousing rules are rules for determining that the same communication link corresponding to the first communication device and the second communication device is kept consistent when entering the warehousing according to the first communication device ID information and the second communication device ID information at both ends of the communication link; Based on the storage form, the data object is stored in a database.

6. The method according to claim 1, characterized in that The performing database materialized view processing on the communication link table to generate a database physical view includes: Determine the first communication device ID, the second communication ID and the communication direction based on the communication node objects and the communication link objects corresponding to each communication link in the communication link table; Determine a topology direction by summarizing each of the communication directions in the communication link table; A database physical view is generated based on the first communication device ID, the second communication device ID and the topology direction.

7. The method according to claim 6, characterized in that The determining the topology direction by summarizing each of the communication directions in the communication link table includes: Summarizing the communication directions in the communication link table through a sum function to determine a first topology direction; Summarizing each of the communication directions in the communication link table through a counting function to determine a second topology direction; Based on the first topology direction and the second topology direction, a topology direction is determined, wherein when the topology direction is zero, the topology direction is determined to be bidirectional communication, and when the topology direction is non-zero, the topology direction is determined to be unidirectional communication.

8. The method according to claim 1, characterized in that After storing the data object in the database, the method further includes: The communication link table obtained is sent to the user end for display through the retrieval interface.

9. The method according to claim 1, characterized in that: The method further comprises: In response to receiving a communication topology information query request sent by a client, and monitoring that the communication topology information query request triggers a topology interface call instruction, a view refresh switch is started to refresh the database physical view in real time.

10. A communication topology information modeling device based on collaborative traffic monitoring, characterized in that: include: The collection module is configured to cache the collected network flow data into a memory, and pre-process the network flow data to obtain network flow attributes corresponding to the network flow data, wherein the network flow data includes: industrial control network flow data and intranet flow data; A merging module is configured to merge the network traffic data with the same network traffic attribute collected within a preset time period in real time to obtain merged network traffic data, and parse and process the merged network traffic data to obtain initial network traffic data, wherein the initial network traffic data includes: communication node data, communication link data and device application data; A modeling module is configured to perform modeling processing on the initial network traffic data, generate data objects that meet the database requirements, store the data objects in the database, and generate a communication link table, wherein the data objects include: communication node objects, communication link objects, and device application objects; The materialization module is configured to perform database materialized view processing on the communication link table to generate a database physical view.

Citation Information

Patent Citations

  • Communication topology information modeling method based on flow data

    CN110011830A

  • Construction method capable of dynamic network atlas database reflecting network multi-dimensional resources

    CN111130842A

  • Wireless network analysis method and device, electronic equipment and storage medium

    CN115002823A

  • Database-based hierarchical information knowledge enhancement method and device

    CN117493578A

  • Data warehouse framework for high performance reporting

    US20230177049A1