Network data flow relation graph generation method and device, electronic equipment and readable medium

Abstract

The embodiment of the invention discloses a network data flow relation graph generation method and device, electronic equipment and a readable medium. A specific embodiment of the method comprises the following steps: acquiring preset network data packet capturing information and network data packet filtering information; capturing each initial network data packet among the servers, and filtering each initial network data packet in parallel based on the network data packet filtering information; analyzing the target network data packet to obtain data packet analysis information corresponding to the target network data packet; based on the obtained analysis information of each data packet, generating information of each communication connection edge between the servers; based on the preset network data packet capture information, storing the information of each communication connection side to a preset database; and in response to the detected data flow query request information sent by the user side, generating a network data flow relation graph corresponding to the at least one piece of communication connection side information. According to the embodiment, waste of computing power resources of a computer is reduced.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a method, device, electronic device, and readable medium for generating a network data flow relationship graph. Background Art

[0002] Network data flow relationship diagram generation is a technology for generating a relationship diagram of the communication relationship between servers. At present, when generating a relationship diagram of the communication relationship between servers, the commonly used method is to capture and analyze large-capacity network data packets by switching packet capture tools and analysis tools to generate a relationship diagram of the communication relationship between servers.

[0003] However, when the above method is used to generate a relationship diagram of the communication relationship between servers, the following technical problems often occur:

[0004] The packet capture tool is used to capture the large amount of network data packets between the servers, and then the analysis tool is switched to parse the large amount of network data packets to generate a relationship diagram of the communication relationship between the servers. There may be compatibility issues between different packet capture tools and analysis tools, resulting in additional steps such as data format conversion, data import and export, resulting in a waste of computer computing resources. At the same time, switching analysis tools may require reallocation of resources. If the allocation is improper or the resources are insufficient, it may cause the system to crash.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose a method, device, electronic device and computer-readable medium for generating a network data flow relationship diagram to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a method for generating a network data flow relationship graph, the method comprising: obtaining preset network data packet capture information and network data packet filtering information; based on the above-mentioned preset network data packet capture information, capturing each initial network data packet between each server, and based on the above-mentioned network data packet filtering information, filtering each initial network data packet in parallel to obtain each target network data packet; for each target network data packet among the above-mentioned target network data packets, parsing and processing the above-mentioned target network data packet to obtain data packet parsing information corresponding to the above-mentioned target network data packet; based on the obtained data packet parsing information, generating each communication connection side information between the above-mentioned servers, wherein each communication connection side information among the above-mentioned communication connection side information represents the data flow relationship between two servers corresponding to the above-mentioned communication connection side information among the above-mentioned servers; based on the above-mentioned preset network data packet capture information, storing the above-mentioned each communication connection side information into a preset database; in response to detecting the data flow query request information sent by the user end, querying at least one communication connection side information corresponding to the above-mentioned data flow query request information from the above-mentioned preset database, and generating a network data flow relationship graph corresponding to the above-mentioned at least one communication connection side information.

[0009] In a second aspect, some embodiments of the present disclosure provide a network data flow relationship diagram generating device, the device comprising: an acquisition unit, configured to acquire preset network data packet capture information and network data packet filtering information; a parallel filtering unit, configured to capture each initial network data packet between each server based on the above preset network data packet capture information, and to filter each initial network data packet in parallel based on the above network data packet filtering information to obtain each target network data packet; a parsing unit, configured to parse each target network data packet among the above target network data packets to obtain data packet parsing information corresponding to the above target network data packet; a first generating unit, configured to generate a plurality of network packets based on the above preset network data packet capture information and to obtain the data packet parsing information corresponding to the above target network data packet. Based on the obtained parsing information of each data packet, each communication connection side information between the above-mentioned servers is generated, wherein each communication connection side information in the above-mentioned communication connection side information represents the data flow relationship between two servers corresponding to the above-mentioned communication connection side information in the above-mentioned servers; the storage unit is configured to store the above-mentioned each communication connection side information in a preset database based on the above-mentioned preset network data packet capture information; the second generating unit is configured to query at least one communication connection side information corresponding to the above-mentioned data flow query request information from the above-mentioned preset database in response to detecting the data flow query request information sent by the user terminal, and generate a network data flow relationship graph corresponding to the above-mentioned at least one communication connection side information.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0012] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the network data flow relationship diagram generation method of some embodiments of the present disclosure, the waste of computer computing resources and the risk of system crash are reduced. Specifically, the reasons for the waste of computer computing resources and the increased risk of system crash are: the network data packets with large data volume between each server are captured by the packet capture tool, and then the network data packets with large data volume are switched to the analysis tool to parse and generate the relationship diagram of the communication relationship between the servers. There may be compatibility problems between different packet capture tools and analysis tools, resulting in additional steps such as data format conversion, data import and export, resulting in waste of computer computing resources. At the same time, switching the analysis tool may require reallocation of resources. If the allocation is improper or the resources are insufficient, the system may crash. Based on this, the network data flow relationship diagram generation method of some embodiments of the present disclosure first obtains preset network data packet capture information and network data packet filtering information. Thus, the preset network data packet capture information for capturing each initial network data packet between each server and the network data packet filtering information for filtering the initial network data packet can be obtained. Afterwards, based on the above-mentioned preset network data packet capture information, each initial network data packet between each server is captured, and based on the above-mentioned network data packet filtering information, each initial network data packet is filtered in parallel to obtain each target network data packet. Thus, in the process of capturing each initial network data packet between each server, the captured data packets can be filtered at the same time, reducing the subsequent parsing of data packets that do not require parsing, thereby reducing the occupation of system resources. Then, for each target network data packet in the above-mentioned target network data packets, the above-mentioned target network data packet is parsed and processed to obtain the data packet parsing information corresponding to the above-mentioned target network data packet. Thus, after capturing the target network data packet, the parsing of the target network data packet can be achieved without switching tools. Then, based on the obtained parsing information of each data packet, each communication connection side information between the above-mentioned servers is generated, wherein each communication connection side information in the above-mentioned communication connection side information represents the data flow relationship between two servers corresponding to the above-mentioned communication connection side information in the above-mentioned servers. Thus, each communication connection side information representing the data flow relationship between each server can be generated. Then, based on the above-mentioned preset network data packet capture information, the above-mentioned each communication connection side information is stored in a preset database. Then, in response to detecting the data flow query request information sent by the user end, at least one communication connection side information corresponding to the above-mentioned data flow query request information is queried from the above-mentioned preset database, and a network data flow relationship graph corresponding to the above-mentioned at least one communication connection side information is generated. Thus, a network data flow relationship diagram corresponding to the query request information sent by the user can be automatically generated.Because before generating the network data flow relationship diagram, the network data packets are captured and filtered in parallel to obtain each target network data packet. After capturing the target network data packet, there is no need to switch tools, and the target network data packet can be parsed, reducing the waste of computer computing resources caused by switching tool compatibility issues. At the same time, the reallocation of resources caused by switching tools is reduced, thereby reducing system crashes caused by uneven resource allocation. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0014] Figure 1 is a flow chart of some embodiments of the method for generating a network data flow relationship diagram according to the present disclosure;

[0015] Figure 2 It is a schematic diagram of the structure of some embodiments of the network data flow relationship diagram generating device according to the present disclosure;

[0016] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0017] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0018] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0019] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0020] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0021] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0022] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0023] Figure 1 The process 100 of some embodiments of the network data flow relationship diagram generation method according to the present disclosure is shown. The network data flow relationship diagram generation method comprises the following steps:

[0024] Step 101, obtaining preset network data packet capture information and network data packet filtering information.

[0025] In some embodiments, the execution subject (e.g., a computing device) of the network data flow relationship diagram generation method can obtain preset network data packet capture information and network data packet filtering information. The preset network data packet capture information includes a database host identifier (e.g., a database host IP), a database host connection port (e.g., a database host connection TCP port), a database user name, a database password, and a network interface name for which network data packets need to be captured. The network data packet filtering information can represent preset filtering conditions for filtering data packets. For example, the network data packet filtering information can be "capture all data packets with a source IP address of 192.168.1.100". In practice, the execution subject can obtain preset network data packet capture information and network data packet filtering information from a preset configuration file. The execution subject can be a server for network monitoring, data packet capture, and analysis.

[0026] Step 102, based on the preset network data packet capture information, each initial network data packet between each server is captured, and based on the network data packet filtering information, each initial network data packet is filtered in parallel to obtain each target network data packet.

[0027] In some embodiments, the above-mentioned execution entity can capture each initial network data packet between each server based on the above-mentioned preset network data packet capture information, and filter each initial network data packet in parallel based on the above-mentioned network data packet filtering information to obtain each target network data packet.

[0028] In some optional implementations of some embodiments, the execution subject may capture each initial network data packet between each server based on the preset network data packet capture information, and filter each initial network data packet in parallel based on the network data packet filtering information to obtain each target network data packet through the following steps:

[0029] The first step is to capture the initial network data packets between each server in real time based on the preset network data packet capture information, and store the captured initial network data packets into a preset queue. In practice, first, the above-mentioned execution subject can determine the network interface name included in the above-mentioned preset network data packet capture information as the target network interface name. Then, the above-mentioned execution subject can monitor the data link layer of the network interface corresponding to the above-mentioned target network interface name by executing the preset code. When the data packet is on the network interface, the above-mentioned execution subject can store the data packet passing through as the initial network data packet into the preset queue. The above-mentioned preset code can be a programming language code written by a developer for monitoring the data link layer of the network interface corresponding to the above-mentioned target network interface name and storing the data packet passing through the network interface as the initial network data packet into the preset queue.

[0030] In the second step, based on the preset queue, perform the following filtering steps:

[0031] In the first sub-step, in response to determining that the preset queue is not empty, a dequeue operation is performed on the preset queue to obtain an initial network data packet to be filtered.

[0032] The second sub-step is to filter the initial network data packet to be filtered based on the network data packet filtering information to obtain a filtered network data packet. In practice, in response to determining that the initial network data packet to be filtered meets the filtering condition represented by the network data packet filtering information, the execution subject may determine the initial network data packet to be filtered as a filtered network data packet. For example, the filtering condition may be "the source IP address is 192.168.1.100".

[0033] The third sub-step is to perform the above filtering step again based on the preset queue after the dequeue operation.

[0034] The third step is to determine each obtained filtered network data packet as each target network data packet.

[0035] Step 103: For each target network data packet in each target network data packet, the target network data packet is parsed to obtain data packet parsing information corresponding to the target network data packet.

[0036] In some embodiments, the execution subject may parse each of the target network data packets to obtain data packet parsing information corresponding to the target network data packet.

[0037] In some optional implementations of some embodiments, the execution subject may parse the target network data packet through the following steps to obtain data packet parsing information corresponding to the target network data packet:

[0038] The first step is to parse the network layer of the target network data packet to obtain the source server address information and the target server address information. In practice, the execution subject can call a preset parsing library (for example, Python's Scapy library parsing) to parse the network layer of the target network data packet to obtain the source server address information and the target server address information. The source server address information can be the IP address of the server that sends the target network data packet. The target server address information can be the IP address of the server that receives the target network data packet.

[0039] The second step is to parse the transport layer of the target network data packet to obtain the source port identifier and the destination port identifier. In practice, the execution subject can call a preset parsing library (for example, Python's Scapy library parsing) to parse the transport layer of the target network data packet to obtain the source port identifier and the destination port identifier. Among them, the source port identifier can be the port number of the port used by the sender (i.e., the source server) to send the data packet. The destination port identifier can be the port number of the port used by the receiver (i.e., the destination server) to receive the data packet.

[0040] The third step is to determine the source server address information, the target server address information, the source port identifier and the target port identifier as data packet parsing information corresponding to the target network data packet.

[0041] Step 104: Generate information about each communication connection between each server based on the obtained parsing information of each data packet.

[0042] In some embodiments, the execution subject may generate each communication connection side information between the servers based on the obtained data packet parsing information. Each communication connection side information in the communication connection side information represents the data flow relationship between two servers corresponding to the communication connection side information in the servers. The data flow relationship may represent the existence of a communication relationship between the servers.

[0043] In some optional implementations of some embodiments, the execution subject may generate the communication connection edge information between the servers based on the obtained parsing information of each data packet through the following steps:

[0044] In the first step, each source server address information and each target server address information included in the above-mentioned data packet parsing information are determined as each initial network device address identifier.

[0045] The second step is to deduplicate the above-mentioned initial network device address identifiers to obtain the identifiers of each network device node. In practice, the above-mentioned execution subject can deduplicate the initial network device address identifiers through a deduplication algorithm based on a Bloom filter to obtain the identifiers of each network device node. For example, the identifiers of each network device node can be "192.168.1.1, 192.168.1.2, 192.168.1.3".

[0046] In the third step, for each of the above network device node identifiers, perform the following steps:

[0047] In the first sub-step, at least one data packet parsing information including the network device node identifier among the above-mentioned data packet parsing information is determined as at least one target data packet parsing information.

[0048] The second sub-step, for each target data packet parsing information in the at least one target data packet parsing information, the source server address information or the target server address information in the target data packet parsing information that is different from the network device node identifier is determined as an associated network device node identifier corresponding to the network device node identifier. As an example, the network device node identifier may be "192.168.1.1". The target data packet parsing information may be "source server address information: 192.168.1.1, target server address information: 192.168.1.2, source port identifier: 80, target port identifier: 20". Then the associated network device node identifier may be "192.168.1.2".

[0049] The third sub-step is to perform deduplication processing on the obtained associated network device node identifiers to obtain deduplication associated network device node identifiers. In practice, the above execution subject can perform deduplication processing on the obtained associated network device node identifiers by using a hash deduplication algorithm to obtain deduplication associated network device node identifiers.

[0050] The fourth sub-step is to generate each initial communication connection edge information based on the network device node identifier, each deduplication-associated network device node identifier and the at least one target data packet parsing information. Each initial communication connection edge information in the initial communication connection edge information includes connection edge endpoint information and connection edge attribute information, and the connection edge endpoint information includes the network device node identifier and a deduplication-associated network device node identifier;

[0051] The fourth step is to perform deduplication processing on the obtained initial communication connection side information to obtain the communication connection side information between each server. In practice, the above execution subject can perform deduplication processing on the obtained initial communication connection side information to obtain the communication connection side information between each server by using a deduplication algorithm based on a Bloom filter.

[0052] In some optional implementations of some embodiments, the execution subject may generate each initial communication connection edge information based on the network device node identifier, each deduplication associated network device node identifier and the at least one target data packet parsing information through the following steps:

[0053] In the first step, for each of the above-mentioned deduplicated associated network device node identifiers, perform the following steps:

[0054] In a first sub-step, the deduplicated associated network device node identifier and the network device node identifier are determined as connection edge endpoint information.

[0055] In a second sub-step, a target data packet parsing information including the deduplication associated network device node identifier in the at least one target data packet parsing information is determined as the screening data packet parsing information.

[0056] The third sub-step is to determine the source port identifier and the destination port identifier included in the above-mentioned screening data packet parsing information as the connection edge attribute information corresponding to the above-mentioned connection edge endpoint information.

[0057] The fourth sub-step is to determine the connection edge endpoint information and the connection edge attribute information as the initial communication connection edge information.

[0058] Step 105, based on the preset network data packet capture information, store each communication connection edge information into a preset database.

[0059] In some embodiments, the execution subject may store the above-mentioned communication connection side information into a preset database based on the above-mentioned preset network data packet capture information. In practice, the execution subject may connect to the preset database through the database host identifier, database host connection port, database user name, and database password included in the preset network data packet capture information. Afterwards, the execution subject may store the above-mentioned communication connection side information into the connected preset database.

[0060] Step 106, in response to detecting the data flow query request information sent by the user end, querying at least one communication connection edge information corresponding to the data flow query request information from a preset database, and generating a network data flow relationship graph corresponding to the at least one communication connection edge information.

[0061] In some embodiments, the execution subject may, in response to detecting the data flow query request information sent by the user end, query at least one communication connection edge information corresponding to the data flow query request information from the preset database, and generate a network data flow relationship graph corresponding to the at least one communication connection edge information. The data flow query request information includes server address information. In practice, the execution subject may convert at least one communication connection edge information into an edge list through a preset function. Then, the execution subject may construct a graph object by calling the NetworkX library, and add nodes and edges corresponding to the edge list to the constructed graph object. Afterwards, the execution subject may render the graph object after adding nodes and edges through a preset graph layout algorithm (such as force-directed layout, circular layout, etc.), and obtain a rendered image as a network data flow relationship graph. The preset function may be a custom function responsible for converting at least one communication connection edge information into an edge list format acceptable to NetworkX. The user end may be a device (such as a desktop, a laptop, etc.) used by a person (such as a network administrator, a system administrator, etc.) who queries the data flow relationship between servers. The server address information may represent the IP address of the server whose data flow relationship needs to be queried.

[0062] In some optional implementations of some embodiments, the execution subject may query at least one communication connection edge information corresponding to the data flow query request information from the preset database through the following steps:

[0063] The communication connection side information including the server address information is queried from the preset database to obtain at least one communication connection side information corresponding to the data flow query request information.

[0064] In the process of adopting technical solutions to solve the problems mentioned in the background technology, the following problems are often accompanied:

[0065] When the generated network data flow relationship diagram is transmitted to the user end, the network condition of the user end may be unstable or poor. When the user is in an unstable network environment, if the uncompressed network data flow relationship diagram is directly transmitted, the user may need to wait for a long time for the network data flow relationship diagram to load, or may even be unable to view the network data flow relationship diagram due to transmission failure, resulting in a poor user experience.

[0066] In the face of the above technical problems, the inventors decided to adopt the following solutions:

[0067] In some optional implementations of some embodiments, after generating the network data flow relationship graph corresponding to the at least one communication connection edge information for each push information in the push information set, the execution subject may further perform the following steps:

[0068] The first step is to perform bandwidth detection on the network of the user terminal to obtain bandwidth detection information corresponding to the user terminal. In practice, the execution subject can perform bandwidth detection on the network of the user terminal through the PGM (Probe Gap Model) available bandwidth measurement method to obtain bandwidth detection information corresponding to the user terminal. For example, the bandwidth detection information can be "50Mbps".

[0069] The second step is to perform delay detection on the network of the user terminal to obtain network delay detection information corresponding to the user terminal. In practice, the execution subject may perform delay detection on the network of the user terminal by ping test technology to obtain network delay detection information corresponding to the user terminal. Optionally, the execution subject may perform delay detection on the network of the user terminal by a delay measurement method based on TCPTraceroute to obtain network delay detection information corresponding to the user terminal. For example, the network delay detection information may be "60s".

[0070] The third step is to detect the packet loss rate of the network of the user terminal and obtain the packet loss rate detection information corresponding to the user terminal. In practice, the execution subject can detect the packet loss rate of the network of the user terminal through a preset network packet loss test tool (for example, ATKKPING (network packet loss test tool)) to obtain the packet loss rate detection information corresponding to the user terminal. For example, the packet loss rate detection information can be "30%".

[0071] The fourth step is to generate a network status score corresponding to the user terminal based on the bandwidth detection information, the network delay detection information and the packet loss rate detection information. In practice, first, the execution subject may determine the ratio of the bandwidth detection information to the first preset value as the first value. The first preset value may be the preset maximum bandwidth. Then, the execution subject may determine the ratio of the network delay detection information to the second preset value as the second initial value. After that, the execution subject may determine the difference between the preset value and the second initial value as the second value. The second preset value may be the preset maximum delay. The preset value may be 1. Then, the execution subject may determine the ratio of the packet loss rate detection information to the third preset value as the third initial value. After that, the execution subject may determine the difference between the preset value and the third initial value as the third value. The third preset value may be the preset maximum packet loss rate. Finally, the execution subject may determine the average of the first value, the second value and the third value as the network status score.

[0072] In a fifth step, in response to determining that the network status score is greater than a preset threshold, the network data flow relationship diagram is sent to the user terminal so that the user terminal can display the network data flow relationship diagram.

[0073] Step 6: In response to determining that the network status score is less than or equal to a preset threshold, the following dynamic compression steps are performed on the network data flow relationship graph:

[0074] The first sub-step is to generate network status level information based on the above network status score. In practice, the above execution entity can obtain a preset network status mapping table from a preset database. The above network status mapping table contains various network level mapping relationship information. Each network level mapping relationship information in the above network level mapping relationship information includes a network status score interval and level information. For example, the above network status score can be 0.4. The above network level mapping relationship information can be "{0.1 is greater than or equal to and less than 0.3, extremely poor}, {0.3 is greater than or equal to and less than 0.5, poor}". The above "extremely poor, poor" is the level information. Then the network status level information can be "poor".

[0075] The second sub-step is to determine the compression algorithm information based on the above network status level information. In practice, the above execution entity can query the compression algorithm information corresponding to the above network status level information from the preset compression algorithm information mapping relationship table from the preset database. The above preset compression algorithm information mapping relationship table contains various mapping relationship information. Each mapping relationship information in the above mapping relationship information can represent the correspondence between a network status level information and a compression algorithm information. The above compression algorithm information can represent a compression algorithm. For example, the mapping relationship information can be "{poor, JPEG compression algorithm}". If the network status level information is "poor", the determined compression algorithm information can be information characterizing the JPEG compression algorithm.

[0076] The third sub-step is to compress the network data flow relationship graph based on the determined compression algorithm information to obtain a compressed network data flow relationship graph. The compressed network data flow relationship graph may be a compressed network data flow relationship graph.

[0077] The fourth sub-step is to send the compression algorithm information and the compressed network data flow relationship diagram to the user terminal so that the user terminal can display the network data flow relationship diagram.

[0078] The above technical solution and its related contents, as an inventive point of the embodiment of the present disclosure, solve the technical problem of "poor user experience". The factors leading to poor user experience are often as follows: when the generated network data flow relationship graph is transmitted to the user end, the network condition of the user end may be unstable or poor. When the user end is in an unstable network environment, if the uncompressed network data flow relationship graph is directly transmitted, the user may need to wait for a long time for the network data flow relationship graph to load, or may even be unable to view the network data flow relationship graph due to transmission failure, resulting in poor user experience. If the above factors are solved, the effect of improving user experience can be achieved. In order to achieve this effect, first, the bandwidth of the network of the above user end is detected to obtain bandwidth detection information corresponding to the above user end. Thus, the bandwidth of the user end can be detected. Then, the delay of the network of the above user end is detected to obtain network delay detection information corresponding to the above user end. Thus, the delay of the network of the user end can be detected. Next, the packet loss rate of the network of the above user end is detected to obtain packet loss rate detection information corresponding to the above user end. Thus, the packet loss rate of the network of the user end can be detected. Afterwards, based on the bandwidth detection information, the network delay detection information and the packet loss rate detection information, a network status score corresponding to the user terminal is generated. Thus, a network status score representing the network status of the user terminal can be generated. Afterwards, in response to determining that the network status score is greater than a preset threshold, the network data flow relationship diagram is sent to the user terminal so that the user terminal can display the network data flow relationship diagram. Thus, when the network status of the user terminal is good, that is, the network status score is greater than the preset threshold, the network data flow relationship diagram can be sent to the user terminal. Next, in response to determining that the network status score is less than or equal to the preset threshold, the following dynamic compression steps are performed on the network data flow relationship diagram: Step 1, based on the network status score, network status level information is generated. Thus, network status level information for determining compression algorithm information can be obtained. Step 2, based on the network status level information, compression algorithm information is determined. Thus, compression algorithm information for compressing the network data flow relationship diagram can be obtained. Afterwards, based on the determined compression algorithm information, the network data flow relationship diagram is compressed to obtain a compressed network data flow relationship diagram. Thus, the network data flow relationship diagram can be compressed to reduce the amount of data in the transmission of the network data flow relationship diagram. Then, the compression algorithm information and the compressed network data flow relationship diagram are sent to the user terminal so that the user terminal can display the network data flow relationship diagram. Thus, a compressed network data flow relationship diagram with a smaller amount of data can be sent to the user terminal. Also, before sending the network data flow relationship diagram to the user terminal, the network status of the user terminal is detected and a network status score is obtained.When the user-side network status score is small, that is, the user-side network is poor or unstable, the compression algorithm information suitable for the current network status is determined according to the user-side network status score to compress the transmitted network data flow relationship graph, thereby reducing the amount of data transmitted when transmitting the network data flow relationship graph, improving the transmission speed, and reducing the waiting time for users to load the network data flow relationship graph. At the same time, compressing the transmitted network data flow relationship graph improves the tolerance for network fluctuations and transmission errors during data transmission, reduces the number of transmission failures, and improves the user experience.

[0079] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the network data flow relationship diagram generation method of some embodiments of the present disclosure, the waste of computer computing resources and the risk of system crash are reduced. Specifically, the reasons for the waste of computer computing resources and the increased risk of system crash are: the network data packets with large data volume between each server are captured by the packet capture tool, and then the network data packets with large data volume are switched to the analysis tool to parse and generate the relationship diagram of the communication relationship between the servers. There may be compatibility problems between different packet capture tools and analysis tools, resulting in additional steps such as data format conversion, data import and export, resulting in waste of computer computing resources. At the same time, switching the analysis tool may require reallocation of resources. If the allocation is improper or the resources are insufficient, the system may crash. Based on this, the network data flow relationship diagram generation method of some embodiments of the present disclosure first obtains preset network data packet capture information and network data packet filtering information. Thus, the preset network data packet capture information for capturing each initial network data packet between each server and the network data packet filtering information for filtering the initial network data packet can be obtained. Afterwards, based on the above-mentioned preset network data packet capture information, each initial network data packet between each server is captured, and based on the above-mentioned network data packet filtering information, each initial network data packet is filtered in parallel to obtain each target network data packet. Thus, in the process of capturing each initial network data packet between each server, the captured data packets can be filtered at the same time, reducing the subsequent parsing of data packets that do not require parsing, thereby reducing the occupation of system resources. Then, for each target network data packet in the above-mentioned target network data packets, the above-mentioned target network data packet is parsed and processed to obtain the data packet parsing information corresponding to the above-mentioned target network data packet. Thus, after capturing the target network data packet, the parsing of the target network data packet can be achieved without switching tools. Then, based on the obtained parsing information of each data packet, each communication connection side information between the above-mentioned servers is generated, wherein each communication connection side information in the above-mentioned communication connection side information represents the data flow relationship between two servers corresponding to the above-mentioned communication connection side information in the above-mentioned servers. Thus, each communication connection side information representing the data flow relationship between each server can be generated. Then, based on the above-mentioned preset network data packet capture information, the above-mentioned each communication connection side information is stored in a preset database. Then, in response to detecting the data flow query request information sent by the user end, at least one communication connection side information corresponding to the above-mentioned data flow query request information is queried from the above-mentioned preset database, and a network data flow relationship graph corresponding to the above-mentioned at least one communication connection side information is generated. Thus, a network data flow relationship diagram corresponding to the query request information sent by the user can be automatically generated.Because before generating the network data flow relationship diagram, the network data packets are captured and filtered in parallel to obtain each target network data packet. After capturing the target network data packet, there is no need to switch tools, and the target network data packet can be parsed, reducing the waste of computer computing resources caused by switching tool compatibility issues. At the same time, the reallocation of resources caused by switching tools is reduced, thereby reducing system crashes caused by uneven resource allocation.

[0080] Further references Figure 2 As an implementation of the methods shown in the figures, the present disclosure provides some embodiments of a network data flow relationship graph generation device, and these device embodiments are Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0081] like Figure 2 As shown, the network data flow relationship diagram generating device 200 of some embodiments includes: an acquisition unit 201, a parallel filtering unit 202, a parsing unit 203, a first generating unit 204, a storage unit 205 and a second generating unit 206. Among them, the acquisition unit 201 is configured to acquire preset network data packet capture information and network data packet filtering information; the parallel filtering unit 202 is configured to capture each initial network data packet between each server based on the above preset network data packet capture information, and to filter each initial network data packet in parallel based on the above network data packet filtering information to obtain each target network data packet; the parsing unit 203 is configured to parse each target network data packet among the above target network data packets to obtain data packet parsing information corresponding to the above target network data packet; the first generating unit 204 is configured to, based on the obtained data packet parsing information, Generate each communication connection side information between the above-mentioned servers, wherein each communication connection side information in the above-mentioned communication connection side information represents the data flow relationship between two servers corresponding to the above-mentioned communication connection side information in the above-mentioned servers; the storage unit 205 is configured to store the above-mentioned each communication connection side information in a preset database based on the above-mentioned preset network data packet capture information; the second generation unit 206 is configured to query at least one communication connection side information corresponding to the above-mentioned data flow query request information from the above-mentioned preset database in response to detecting the data flow query request information sent by the user terminal, and generate a network data flow relationship graph corresponding to the above-mentioned at least one communication connection side information.

[0082] It is understood that the units described in the device 200 are similar to those described in the reference Figure 1Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 200 and the units contained therein, and will not be described in detail here.

[0083] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0084] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0085] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0086] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the functions defined in the method of some embodiments of the present disclosure are executed.

[0087] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0088] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0089] The computer-readable medium may be included in the electronic device; or may exist separately without being incorporated into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains preset network data packet capture information and network data packet filtering information; based on the preset network data packet capture information, captures each initial network data packet between each server, and based on the network data packet filtering information, filters each initial network data packet in parallel to obtain each target network data packet; for each target network data packet among the target network data packets, parses and processes the target network data packet to obtain data packet parsing information corresponding to the target network data packet; based on the obtained data packet parsing information, generates each communication connection side information between the servers, wherein each communication connection side information among the communication connection side information represents the data flow relationship between two servers corresponding to the communication connection side information among the servers; based on the preset network data packet capture information, stores the each communication connection side information into a preset database; in response to detecting the data flow query request information sent by the user end, queries at least one communication connection side information corresponding to the data flow query request information from the preset database, and generates a network data flow relationship graph corresponding to the at least one communication connection side information.

[0090] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0091] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0092] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be arranged in a processor, for example, may be described as: a processor includes an acquisition unit, a parallel filtering unit, a parsing unit, a first generation unit, a storage unit, and a second generation unit. The names of these units do not constitute limitations on the units themselves in certain cases, for example, the acquisition unit may also be described as a "unit for acquiring preset network data packet capture information and network data packet filtering information".

[0093] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0094] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of technical features, but should also cover other technical solutions formed by any combination of technical features or their equivalent features without departing from the inventive concept. For example, a technical solution formed by replacing the features with (but not limited to) the technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A method for generating a network data flow relationship diagram, comprising: Obtain preset network packet capture information and network packet filtering information; Based on the preset network data packet capture information, each initial network data packet between each server is captured, and based on the network data packet filtering information, each initial network data packet is filtered in parallel to obtain each target network data packet; For each target network data packet in the target network data packets, parsing the target network data packet to obtain data packet parsing information corresponding to the target network data packet; Generate each communication connection side information between each server based on each obtained data packet parsing information, wherein each communication connection side information in each communication connection side information represents a data flow relationship between two servers in each server corresponding to the communication connection side information; Based on the preset network data packet capture information, storing the information of each communication connection edge in a preset database; In response to detecting data flow query request information sent by the user end, querying at least one communication connection edge information corresponding to the data flow query request information from the preset database, and generating a network data flow relationship graph corresponding to the at least one communication connection edge information.

2. The method according to claim 1, wherein: The parsing of the target network data packet to obtain data packet parsing information corresponding to the target network data packet includes: Parsing the network layer of the target network data packet to obtain source server address information and target server address information; Parsing the transport layer of the target network data packet to obtain a source port identifier and a target port identifier; The source server address information, the target server address information, the source port identifier and the target port identifier are determined as data packet parsing information corresponding to the target network data packet.

3. The method according to claim 1, wherein: The data stream query request information includes server address information; and the querying of at least one communication connection edge information corresponding to the data stream query request information from the preset database includes: The communication connection side information including the server address information is queried from the preset database to obtain at least one communication connection side information corresponding to the data flow query request information.

4. The method according to claim 1, wherein: The filtering of each initial network data packet in parallel based on the network data packet filtering information to obtain each target network data packet includes: Based on the preset network data packet capture information, the initial network data packets between the servers are captured in real time, and the captured initial network data packets are stored in a preset queue; Based on the preset queue, the following filtering steps are performed: In response to determining that the preset queue is not empty, performing a dequeue operation on the preset queue to obtain an initial network data packet to be filtered; Based on the network data packet filtering information, filtering the initial network data packet to be filtered to obtain a filtered network data packet; Based on the preset queue after the dequeue operation, the filtering step is repeated; The obtained filtered network data packets are determined as target network data packets.

5. The method according to claim 1, wherein: The generating of each communication connection edge information between each server based on each obtained data packet parsing information includes: Determine each source server address information and each target server address information included in each data packet parsing information as each initial network device address identifier; Deduplication processing is performed on each of the initial network device address identifiers to obtain each network device node identifier; For each of the network device node identifiers, perform the following steps: Determine at least one data packet parsing information including the network device node identifier among the respective data packet parsing information as at least one target data packet parsing information; For each target data packet parsing information in the at least one target data packet parsing information, determining the source server address information or the target server address information in the target data packet parsing information that is different from the network device node identifier as an associated network device node identifier corresponding to the network device node identifier; Deduplication processing is performed on each obtained associated network device node identifier to obtain each deduplication associated network device node identifier; Based on the network device node identifier, the respective deduplication-associated network device node identifiers and the at least one target data packet parsing information, each initial communication connection edge information is generated, wherein each initial communication connection edge information in the respective initial communication connection edge information includes connection edge endpoint information and connection edge attribute information, and the connection edge endpoint information includes the network device node identifier and a deduplication-associated network device node identifier; The obtained initial communication connection side information is deduplicated to obtain the communication connection side information between the servers.

6. The method according to claim 5, wherein: The generating each initial communication connection edge information based on the network device node identifier, each deduplication associated network device node identifier and the at least one target data packet parsing information comprises: For each of the deduplicated associated network device node identifiers, perform the following steps: Determine the deduplicated associated network device node identifier and the network device node identifier as connection edge endpoint information; Determine a target data packet parsing information including the deduplication associated network device node identifier in the at least one target data packet parsing information as the screening data packet parsing information; Determine the source port identifier and the destination port identifier included in the parsed information of the screening data packet as the connection edge attribute information corresponding to the connection edge endpoint information; The connection edge endpoint information and the connection edge attribute information are determined as initial communication connection edge information.

7. A network data flow relationship diagram generating device, comprising: An acquisition unit, configured to acquire preset network data packet capture information and network data packet filtering information; A parallel filtering unit is configured to capture each initial network data packet between each server based on the preset network data packet capture information, and to filter each initial network data packet in parallel based on the network data packet filtering information to obtain each target network data packet; A parsing unit is configured to parse each of the target network data packets to obtain data packet parsing information corresponding to the target network data packet; A first generating unit is configured to generate each communication connection side information between each server based on each obtained data packet parsing information, wherein each communication connection side information in the each communication connection side information represents a data flow relationship between two servers corresponding to the communication connection side information in each server; A storage unit configured to store the information of each communication connection edge into a preset database based on the preset network data packet capture information; The second generating unit is configured to, in response to detecting data flow query request information sent by the user terminal, query at least one communication connection edge information corresponding to the data flow query request information from the preset database, and generate a network data flow relationship graph corresponding to the at least one communication connection edge information.

8. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images