Network target range flow data visualization method and system based on dynamic configuration and real-time rendering

Abstract

The invention discloses a network target range flow data visualization method and system based on dynamic configuration and real-time rendering. The method comprises the following steps: configuring network equipment and information thereof; performing field verification on the acquired flow data between the network devices, and filtering out invalid data; merging a plurality of pieces of single data containing timestamps and specific data contents into a time period with a set time interval, and storing the data in each time period through a time bucket array; for the data in each time bucket, performing deep screening on the traffic data according to different network equipment types, and regarding the same data of the network equipment and the target network equipment as repeated connection data; and according to the count of the connection lines, matching the color corresponding to the configured times to determine the color of the connection lines, and rendering the connection lines between the network devices. According to the invention, the problem of lagging display at the front end of the network range is effectively relieved, and the user experience is improved. By dynamically displaying the network traffic data, the transparency and real-time performance of network operation and maintenance are enhanced.

Description

Technical Field

[0001] The present invention relates to network range traffic visualization, and in particular to a network range traffic data visualization method and system based on dynamic configuration and real-time rendering. Background Art

[0002] The network target range is a platform that simulates actual combat environments to deepen security testing and drills. The real-time traffic data of network devices in the network target range is displayed in a visual interface. It can monitor network traffic in real time, flow trends between network devices, quickly identify abnormal traffic patterns, evaluate network traffic conditions, and optimize network configuration.

[0003] At present, the visualization display of traffic data between network devices in the network target range has the following shortcomings: the data is too scattered, not aggregated data, or a single network topology diagram is displayed, and the traffic effect cannot be displayed in real time; the traffic data between network devices needs to be displayed according to the color scheme of the network target range demonstrator, and real-time switching cannot be achieved; when the amount of data is too large, the front end is prone to lag, resulting in a poor user experience; when affected by network fluctuations, some invalid data will lead to incorrect display effects. Summary of the invention

[0004] Purpose of the invention: In view of the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a method and system for visualizing network target range traffic data based on dynamic configuration and real-time rendering. According to the traffic data between network devices pushed in real time between network target ranges, the data is cleaned, aggregated, weighted and dynamically colored to achieve a visual display of the data flow path.

[0005] Technical solution: To achieve the above-mentioned invention object, the present invention adopts the following technical solution:

[0006] A network range traffic data visualization method based on dynamic configuration and real-time rendering includes the following steps:

[0007] Network device configuration: configure network devices and their information;

[0008] Data cleaning: Perform field verification on the traffic data between network devices and filter out invalid data;

[0009] Data aggregation: Customized time intervals, multiple single data containing timestamps and specific data content are merged into a time period with a set time interval, and the data in each time period is stored in a time bucket array;

[0010] Data weighting: Dynamically pull new data at set time intervals. For the data in each time bucket, the data with the same starting and target network devices are considered as duplicate connection data.

[0011] Dynamic color matching: According to the connection count, the color corresponding to the configured number of times is matched to determine the color of the connection and render the connection between network devices.

[0012] Preferably, in the data cleaning step, it is determined whether the traffic data includes the starting network device, the target network device and the time when the data traffic occurs. If one of them is missing, it is considered to be invalid data.

[0013] Preferably, in the data aggregation step, the time interval is in milliseconds, the key of each time period is the start time of the time period, and the value is the set of all data in the time period.

[0014] Preferably, in the data cleaning step, the traffic is also intelligently detected, and the detected abnormal traffic is recorded and filtered; in the data aggregation step, the traffic data in the time bucket is deeply screened according to the data logic processing rules defined in the network device configuration step.

[0015] Preferably, in the data weighting step, the front end maintains a connection count map: a JavaScript Map is used to store the key of the connection between each network device and its number of occurrences, or the combination of the starting point and the end point and its number of occurrences; the data within the pulled time period is looped through, and when a connection that is the same as the current network device and the target connected network device is found, the count of the connection is updated.

[0016] Preferably, in the dynamic color matching step, different colors are set for different numerical ranges. When the number does not exceed the first threshold, the color is light gray, when the number is between the first threshold and the second threshold, the color is orange, and when the number exceeds the second threshold, the color is red.

[0017] A network range traffic data visualization system based on dynamic configuration and real-time rendering, including:

[0018] Network device configuration module: used to configure network devices and their information;

[0019] The data cleaning module is used to perform field verification on the traffic data between network devices and filter out invalid data;

[0020] The data aggregation module is used for custom time intervals to aggregate multiple single data items containing timestamps and specific data content into time periods with the set time intervals, and store the data in each time period through the time bucket array;

[0021] The data weighting module is used to dynamically pull new data according to the set time interval. For the data in each time bucket, the data that is the same between the starting network device and the target network device is regarded as duplicate connection data;

[0022] The dynamic color matching module is used to determine the color of the connection according to the count of the connection, match the color corresponding to the configured number of times, and render the connection between network devices.

[0023] Preferably, in the data cleaning module, intelligent detection of traffic is also performed, and detected abnormal traffic is recorded and filtered; in the data aggregation module, in-depth screening of traffic data in the time bucket is also performed according to the data logic processing rules defined in the network device configuration module.

[0024] Preferably, in the data weighting module, the front end maintains a connection count map: a JavaScript Map is used to store the key of the connection between each network device and its number of occurrences, or the combination of the starting point and the end point and its number of occurrences; the data within the pulled time period is looped through, and when a connection that is the same as the current network device and the target connected network device is found, the count of the connection is updated.

[0025] A computer program product includes a computer program / instruction, which, when executed by a processor, implements the steps of a network range traffic data visualization method based on dynamic configuration and real-time rendering.

[0026] Beneficial effects: Compared with the prior art, the present invention has the following advantages: Through real-time data cleaning, aggregation, weighting and dynamic color matching, the present invention effectively alleviates the problem of freezing when displaying large amounts of data on the front end of the network range, and improves the user experience. At the same time, by dynamically displaying network traffic data, the transparency and real-time nature of network operation and maintenance are enhanced, which helps to timely discover and solve potential network problems and improve the stability and security of network equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of a flow chart of an embodiment of the present invention.

[0028] Figure 2 A schematic diagram of rendering traffic between network devices according to an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments.

[0030] The embodiment of the present invention discloses a method for visualizing network range traffic data based on dynamic configuration and real-time rendering, comprising the following steps:

[0031] Network device configuration: used to configure network devices (switches, routers, firewalls, etc.) and their information;

[0032] Data cleaning: Perform field verification on the traffic data between network devices and filter out invalid data;

[0033] Data aggregation: Customized time intervals, multiple single data containing timestamps and specific data content are merged into a time period with a set time interval, and the data in each time period is stored in a time bucket array;

[0034] Data weighting: Dynamically pull new data at set time intervals. For the data in each time bucket, the data with the same starting and target network devices are considered as duplicate connection data.

[0035] Dynamic color matching: According to the connection count, the color corresponding to the configured number of times is matched to determine the color of the connection and render the connection between network devices.

[0036] The method of the embodiment of the present invention can be widely used in real-time traffic display of network equipment in a network target range. By displaying the traffic data of network equipment in the network target range in real time, users can better understand the network operation status, timely discover and solve potential network problems, and ensure the stability and security of network equipment.

[0037] More specifically, see Figure 1 The specific implementation process example of the network target range traffic data visualization method based on dynamic configuration and real-time rendering is as follows.

[0038] Step S1: Configure network devices and their information. In this step, users can flexibly define, configure, dynamically update and delete various types of devices in the network environment, including but not limited to switching devices (such as switches), routing devices (such as routers), security devices (such as firewalls) and terminal devices (such as computers, mobile phones, etc.). Users can specify a unique name for each device, clearly identify the type, and enter basic information such as device IP address, port number, MAC address, etc., and can further configure data filtering logic specific to the device type. The details are as follows:

[0039] Device IP address: The network address of the device, used for network communication and data routing.

[0040] Port number: For devices such as switches and routers, port numbers are used to identify physical or logical interfaces on the device that connect different network devices or network segments.

[0041] MAC address: The physical address of a device, used to uniquely identify the device within a local area network.

[0042] Traffic data logic processing configuration: To optimize the data aggregation of the S3 step, users can customize the traffic display logic for the device type. For example, when the switch focuses on a specific port or VLAN traffic, port and vlan can be configured as statistical keywords; when the firewall focuses on malicious traffic (such as DDoS), ddos ​​can be configured as a keyword. All keywords are directly mapped to the return field of each traffic data, ensuring that data aggregation is both accurate and efficient.

[0043] This information constitutes the complete configuration file of the device, providing important basic data support for subsequent network traffic data cleaning, processing, and network operation and maintenance.

[0044] Step S2: Cleaning of traffic data between network devices. In this step, not only basic traffic data cleaning can be performed to ensure the integrity and accuracy of the data, but also a machine learning algorithm can be further introduced to automatically identify abnormal traffic data.

[0045] In some embodiments, the basic data cleaning steps are as follows:

[0046] Receiving data: The network range backend sends the traffic data between network devices to the frontend in real time and continuously via websocket.

[0047] Field check: The front end checks whether each flow data contains the starting network device, the target network device, the time when the data flow occurred (timestamp), and the necessary network device information fields.

[0048] Type verification: Confirm whether the starting and target network device types belong to the pre-configured device type list (such as switching devices, routing devices, security devices, terminal devices, etc.). If not in the list, it is considered invalid data and filtered directly.

[0049] Integrity check: If the data flow occurrence time or network device information field is missing, it is also regarded as invalid data and filtered directly.

[0050] In some embodiments, the abnormal traffic detection steps are as follows:

[0051] Algorithm selection: After data cleaning, introduce unsupervised learning algorithms for anomaly detection. Algorithms can use isolation forest or local outlier factor (LOF).

[0052] Feature selection: Select features for anomaly detection based on the characteristics of traffic data. These features may include traffic size, number of packets, duration, source / destination IP addresses, port numbers, etc.

[0053] Model training: Use historical traffic data to train the Isolation Forest or LOF model. During the training process, the model will learn the distribution characteristics of normal traffic.

[0054] Anomaly Identification: Apply the trained model to detect anomalies in real-time traffic data. For example, Isolation Forest evaluates the degree of anomaly by calculating the distance from a data point to its nearest isolation point; LOF identifies anomalies by calculating the local density deviation of each data point.

[0055] Result processing: For detected abnormal traffic, the system can record the abnormal details, generate alarms, or perform other processing measures according to business needs, such as blocking connections, recording logs, etc.

[0056] Data output: Prepare the effective traffic data that has been cleaned and passed anomaly detection to facilitate subsequent data aggregation.

[0057] Step S3: Traffic data aggregation processing between network devices, the main process is as follows: when the network target range backend returns the traffic between network devices to the front end through websocket, when the traffic is too large and the amount of data is too large, the front end directly rendering these data may cause the page to freeze or crash. In order to alleviate this problem, the present invention adopts a data aggregation method, that is, aggregating data according to a custom time interval, and can also process traffic data according to a custom traffic data logic, and then render the aggregated data.

[0058] In this embodiment, the following identifiers are first defined: t1: a custom time interval, in milliseconds. n1: a single piece of data, an object containing a timestamp and specific data content.

[0059] Merge multiple n1 data into time periods with t1 as intervals according to timestamps. We can process data in a time bucket manner and create a time bucket array to store data in each time period. The key of each time period can be the start time of the time period (rounded down to a multiple of t1), and the value is the set of all data in the time period. Assuming T is the timestamp of n1 (milliseconds), and t1 is a custom time interval (milliseconds), then n1 belongs to the time period time_bucket_k. The formula for data processing is as follows: In this way, we can pull new data regularly according to the time interval of t1 to achieve preliminary data integration.

[0060] In addition, the data in the time bucket can be further screened according to the logical processing rules defined in step S1. For example, for switches, users can set port and vlan as keywords, and the system only retains data containing these keywords, accurately reflecting the traffic of specific ports or VLANs. For firewalls, configure the ddos ​​keyword to automatically classify and aggregate malicious traffic data, such as DDoS attacks, to ensure accurate display of information and traffic data of related network devices. Then, based on the aggregated and specific logically processed data, the connections between the data of network devices are rendered.

[0061] Step S4: Dynamically generate the color matching of the traffic data connection between network devices. The traffic data between network devices first needs to be weighted. Through the above method, each time_bucket_k array stores the traffic data between network devices at that time period. After that, based on data aggregation, if the same data is found between the starting network device and the target network device, these are repeated connection data. We can adjust the color of the connection according to the number of occurrences. This involves weighted calculation. After calculation, the count of each device connection is obtained, and the color is set based on the count.

[0062] The implementation steps are as follows: Maintain a connection count map: Use JavaScript Map to store the key (or the combination of the starting point + the end point) of each connection between network devices and its occurrence count. Update count: Loop through the time_bucket_k array, and when a connection that is the same as the current network device and the target network device is found, update the count of the connection. Then calculate the color, and determine the color of the connection based on the connection count and the color corresponding to the predefined background configuration number.

[0063] Then each time_bucket_k will be updated to the following data structure:

[0064]

[0065]

[0066] The sample code for data weighting processing is as follows:

[0067]

[0068] / / Clear connectionCounts

[0069] connectionCounts.clear();

[0070] return result;

[0071] }

[0072] / / Color mapping example

[0073] function getColorBasedOnCount(count){

[0074] / / Weight is less than or equal to 200

[0075] if(count<=200){

[0076] return '#CCCCCC'; / / light color

[0077] / / The weight is greater than 200 and less than or equal to 500

[0078] }else if(count<=500){

[0079] return '#FF8000'; / / orange

[0080] / / The weight is greater than 500

[0081] }else{

[0082] return '#FF0000'; / / red

[0083] }

[0084] }

[0085] Step S5: Dynamic drawing of traffic data between network devices. Finally, we will dynamically pull new data through the above steps at the defined time interval t1, and after cleaning, aggregation, weighted processing, and dynamic color matching, we will get the final data that needs to be rendered. At this point, data processing is completed, and users can observe the traffic flow between network devices in real time, so as to better monitor and analyze the network. An example of the traffic flow between network devices is shown in the figure below. Figure 2 shown.

[0086] The network range traffic data visualization method based on dynamic configuration and real-time rendering described in the embodiment of the present invention has the following advantages:

[0087] Strong real-time performance: Traffic data between network devices is obtained in real time through WebSocket, and is processed and displayed immediately, ensuring that users can understand the current status of network devices in a timely manner and facilitate timely discovery of potential problems.

[0088] Data cleaning ensures accuracy: Strict field verification is performed on the acquired data to filter out invalid data, avoiding misjudgment or incorrect display due to incomplete or erroneous data, and improving the accuracy and reliability of the data.

[0089] Data aggregation improves performance: When the amount of data is large, the data aggregation method is used to merge the data at custom time intervals, reducing the amount of data rendered on the front end, effectively alleviating page freezes or crashes, and improving user experience and system performance.

[0090] Dynamic color matching enhances visibility: Dynamically adjust the color of the connection according to the number of times the flow data between network devices appears, making high or abnormal flow easier to identify, enhancing data visibility and user understanding.

[0091] Flexibility and scalability: The solution dynamically configures network devices and custom data processing logic, and the data processing process (cleaning, aggregation, weighting, drawing) is relatively independent and modular, which is convenient for flexible adjustment and expansion according to actual needs. For example, you can easily modify the color mapping rules or add new data processing steps.

[0092] Based on the same inventive concept, an embodiment of the present invention discloses a network range traffic data visualization system based on dynamic configuration and real-time rendering, including: a network device configuration module: used to configure network devices and their information; a data cleaning module, used to perform field verification on the acquired traffic data between network devices and filter out invalid data; a data aggregation module, used to customize the time interval, merge multiple single data containing timestamps and specific data content into a time period with a set time interval, and store the data in each time period through a time bucket array; a data weighting module, used to dynamically pull new data according to the set time interval, for the data in each time bucket, the data that is the same as the starting network device and the target network device is regarded as repeated connection data; a dynamic color matching module, used to determine the color of the connection according to the count of the connection, matching the color corresponding to the configured number of times, and rendering the connection between network devices.

[0093] In some embodiments, the data cleaning module also performs intelligent detection on the traffic, and records and filters the detected abnormal traffic.

[0094] In some embodiments, the data aggregation module also performs in-depth screening of traffic data in the time bucket according to the data logic processing rules defined in the network device configuration module.

[0095] In some embodiments, in the data weighting module, the front end maintains a connection count map: uses JavaScript's Map to store the key of the connection between each network device and its number of occurrences, or the combination of the starting point and the end point and its number of occurrences; loops through the data within the pulled time period, and when a connection that is the same as the current network device and the target connection network device is found, the count of the connection is updated.

[0096] An embodiment of the present invention also discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of a method for visualizing network range traffic data based on dynamic configuration and real-time rendering.

Claims

1. A network range traffic data visualization method based on dynamic configuration and real-time rendering, characterized in that: The steps include: Network device configuration: configure network devices and their information; Data cleaning: Perform field verification on the traffic data between network devices and filter out invalid data; Data aggregation: Customized time intervals, multiple single data containing timestamps and specific data content are merged into a time period with a set time interval, and the data in each time period is stored in a time bucket array; Data weighting: Dynamically pull new data at set time intervals. For the data in each time bucket, the data with the same starting and target network devices are considered as duplicate connection data. Dynamic color matching: According to the connection count, the color corresponding to the configured number of times is matched to determine the color of the connection and render the connection between network devices.

2. The network range traffic data visualization method based on dynamic configuration and real-time rendering according to claim 1 is characterized in that: In the data cleaning step, it is determined whether the traffic data contains the starting network device, the target network device and the time when the data traffic occurs. If one of them is missing, it is considered to be invalid data.

3. The network range traffic data visualization method based on dynamic configuration and real-time rendering according to claim 1 is characterized in that: In the data aggregation step, the time interval is in milliseconds, the key of each time period is the start time of the time period, and the value is the collection of all data in the time period.

4. The network range traffic data visualization method based on dynamic configuration and real-time rendering according to claim 1 is characterized in that: In the data cleaning step, the traffic is also intelligently detected, and the detected abnormal traffic is recorded and filtered; In the data aggregation step, the traffic data in the time bucket is deeply screened according to the data logic processing rules defined in the network device configuration step.

5. The network range traffic data visualization method based on dynamic configuration and real-time rendering according to claim 1 is characterized in that: In the data weighting step, the front end maintains a connection count map: JavaScript's Map is used to store the key of the connection between each network device and its number of occurrences, or the combination of the starting point and the end point and its number of occurrences; the data within the pulled time period is looped through, and when a connection that is the same as the current network device and the target connected network device is found, the count of the connection is updated.

6. The network range traffic data visualization method based on dynamic configuration and real-time rendering according to claim 1 is characterized in that: In the dynamic color matching step, different colors are set for different value ranges. When the number does not exceed the first threshold, the color is light gray, when the number is between the first threshold and the second threshold, the color is orange, and when the number exceeds the second threshold, the color is red.

7. A network range traffic data visualization system based on dynamic configuration and real-time rendering, characterized in that: include: Network device configuration module: used to configure network devices and their information; The data cleaning module is used to perform field verification on the traffic data between network devices and filter out invalid data; The data aggregation module is used for custom time intervals to aggregate multiple single data items containing timestamps and specific data content into time periods with the set time intervals, and store the data in each time period through the time bucket array; The data weighting module is used to dynamically pull new data according to the set time interval. For the data in each time bucket, the data that is the same between the starting network device and the target network device is regarded as duplicate connection data; The dynamic color matching module is used to determine the color of the connection according to the count of the connection, match the color corresponding to the configured number of times, and render the connection between network devices.

8. The network range traffic data visualization system based on dynamic configuration and real-time rendering according to claim 7 is characterized in that: In the data cleaning module, traffic is also intelligently detected, and abnormal traffic detected is recorded and filtered; In the data aggregation module, the traffic data in the time bucket is deeply screened according to the data logic processing rules defined in the network device configuration module.

9. The network range traffic data visualization system based on dynamic configuration and real-time rendering according to claim 7 is characterized in that: In the data weighting module, the front end maintains a connection count map: JavaScript's Map is used to store the key of the connection between each network device and its number of occurrences, or the combination of the starting point and the end point and its number of occurrences; the data within the pulled time period is looped through, and when a connection that is the same as the current network device and the target connected network device is found, the count of the connection is updated.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by the processor, the steps of a method for visualizing network range traffic data based on dynamic configuration and real-time rendering according to any one of claims 1 to 6 are implemented.

Images