Network information detection method and system based on dynamic graph stream triangle counting, terminal and storage medium

Abstract

The application relates to the technical field of data detection, and discloses a network information detection method and system based on dynamic graph stream triangle counting, a terminal and a storage medium. The method comprises the following steps: obtaining network information to be detected, performing graph definition to obtain an original dynamic graph stream, and inputting the original dynamic graph stream into a constructed distributed architecture; distributing and processing each vertex of the original dynamic graph stream through the distributed architecture to obtain a node distribution result, and performing triangle sampling to obtain a target sampling result; performing summary processing on all triangle sampling information in the target sampling result to obtain a triangle counting result, and obtaining a network information detection result according to the triangle counting result. According to the application, the vertices are dynamically distributed to fixed working nodes, the three edges of each triangle belong to the same working node, the demand for cross-node communication is eliminated, the sampling probability of the triangle is improved, the uniqueness and accuracy of sampling are ensured, and therefore the accuracy and efficiency of the network information detection result are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data detection, and particularly relates to a network information detection method and system based on dynamic graph stream triangle counting, a terminal and a computer readable storage medium. BACKGROUND

[0002] In the real world, graphs (a graph is a mathematical object used to represent objects (i.e., vertices) and relationships (i.e., edges) between objects) are widely used in various fields because graphs provide an effective way to represent complex relationships and structures. Nowadays, a large amount of data is often modeled and analyzed in the form of graphs, including social media networks, collaboration and co-authorship networks, transportation networks, and various types of biological networks. In the era of big data, there are severe memory demand challenges in researching and analyzing these large-scale networks, and in this context, a common coping strategy is to use sampling techniques to analyze these networks to obtain approximate analysis results. Triangle counting is a core problem in the field of network science, and triangle counting is a high-complexity but important graph statistic. There are usually a large number of triangles in real-world networks, while randomly generated networks rarely have this feature. In real-world social networks and graph data mining application scenarios, triangle sampling counting plays a key role, for example, detecting fake accounts (real users' social relationships usually form natural triangles, while fake accounts' relationship chains often present a star structure, i.e., a central node connects a large number of isolated nodes, and the triangle density is significantly lower), detecting Web spam (natural triangle structures have power-law distribution characteristics, and spam websites often link to each other to improve search engine rankings, forming dense but abnormal triangle structures), and the like.

[0003] However, existing technologies use the Triest-FD and ThinkD algorithms to estimate the number of triangles. Triest-FD is an efficient algorithm for estimating the global and local number of triangles in fully dynamic graph flows (i.e., where edges are continuously inserted or deleted), achieving unbiased estimation with fixed memory. ThinkD is an efficient triangle counting method for fully dynamic graph flows, significantly improving the probability of capturing triangle structures by updating the estimator before sampling edges. However, both Triest-FD and ThinkD algorithms have their drawbacks. Triest-FD relies on a global random sampling strategy, resulting in insufficient distributed scalability and limited adaptability to dynamic flows. In a distributed environment, simply extending it to independent sampling across multiple nodes can lead to repeated sampling or omission of triangles. Furthermore, due to a lack of awareness of graph structure, replacement decisions are based solely on randomness, neglecting vertex relationships, leading to a reduced triangle sampling probability in high-density subgraphs. The ThinkD algorithm suffers from incomplete optimization of sampling probability and low utilization of heterogeneous computing resources. In distributed streams, the difference in computing power among worker nodes causes some nodes to become bottlenecks. Because no load balancing strategy is designed, high-speed nodes have to wait for low-speed nodes to complete sampling, which reduces overall efficiency.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The main objective of this invention is to provide a network information detection method, system, terminal, and storage medium based on dynamic graph flow triangle counting. This invention aims to solve the problems of low sampling rate, inability to reduce algorithm estimation error, poor load balancing, and inability to improve running speed through distributed expansion steps in the existing dynamic graph flow triangle counting technology, resulting in low accuracy and efficiency of network information detection results.

[0006] To achieve the above objectives, the present invention provides a network information detection method based on dynamic graph flow triangle counting, the method comprising the following steps:

[0007] Obtain the network information to be detected, define the graph based on the network information to obtain the original dynamic graph flow, and input the original dynamic graph flow into the constructed distributed architecture;

[0008] The distributed architecture allocates each vertex of the original dynamic graph flow according to the vertex hash allocation strategy to obtain the node allocation result, and performs triangle sampling based on the node allocation result to obtain the target sampling result;

[0009] The distributed architecture is used to aggregate all the triangle sampling information in the target sampling result to obtain triangle counting result, and network information detection result is obtained according to the triangle counting result.

[0010] Optionally, the network information detection method based on dynamic graph stream triangle counting, wherein the distributed architecture comprises a master node, a working node set and an aggregation node, and the node distribution result comprises a first distribution result, a second distribution result and a third distribution result.

[0011] The distributed architecture is used to distribute each vertex of the original dynamic graph stream according to a vertex hash distribution strategy to obtain a node distribution result, and the method specifically comprises the following steps.

[0012] An initial graph is generated according to the original dynamic graph stream, all triangles of the initial graph are obtained, and a vertex calculation result is obtained by calculating the hash value of each vertex of the triangles through the master node.

[0013] If there is a first triangle with three vertices having the same hash value in the vertex calculation result, the three vertices of the first triangle are mapped to the working node set through the master node to obtain a first mapping result, and the first triangle is distributed to the working node set according to the first mapping result to obtain a first distribution result.

[0014] If there is a second triangle with two vertices having the same hash value in the vertex calculation result, the two vertices of the second triangle having the same hash value are mapped to the working node set through the master node to obtain a second mapping result, and the second triangle is distributed to the working node set according to the second mapping result to obtain a second distribution result.

[0015] If there is a third triangle with three vertices having different hash values in the vertex calculation result, the third triangle is mapped to the working node set through the master node to obtain a third mapping result, and the third triangle is distributed to the working node set according to the third mapping result to obtain a third distribution result.

[0016] Optionally, the network information detection method based on dynamic graph stream triangle counting, wherein if there is a first triangle with three vertices having the same hash value in the vertex calculation result, the three vertices of the first triangle are mapped to the working node set through the master node to obtain a first mapping result, and the first triangle is distributed to the working node set according to the first mapping result to obtain a first distribution result, and the method specifically comprises the following steps.

[0017] If the vertex calculation result contains a first triangle with three vertices having the same hash value, the master node maps the three vertices of the first triangle to target worker nodes of the worker node set to obtain a first mapping result;

[0018] The master node obtains three edges of the first triangle, and distributes all the edges to the target worker nodes according to the first mapping result to obtain a first distribution result.

[0019] Optionally, the network information detection method based on dynamic graph stream triangle counting, wherein if the vertex calculation result contains a second triangle with two vertices having the same hash value, the master node maps the two vertices of the second triangle having the same hash value to the worker node set to obtain a second mapping result, and distributes nodes to the second triangle according to the second mapping result to obtain a second distribution result, specifically including:

[0020] If the vertex calculation result contains a second triangle with two vertices having the same hash value, the master node maps the two vertices of the second triangle having the same hash value to a first worker node in the worker node set, and maps the remaining vertices of the second triangle to a second worker node in the worker node set to obtain a second mapping result.

[0021] The master node obtains an edge formed by the two vertices of the second triangle having the same hash value, and distributes the edge to the first worker node according to the second mapping result, and distributes the remaining two edges of the second triangle to the first worker node and the second worker node according to the second mapping result to obtain a second distribution result.

[0022] Optionally, the network information detection method based on dynamic graph stream triangle counting, wherein if the vertex calculation result contains a third triangle with three vertices having different hash values, the master node maps the third triangle to the worker node set to obtain a third mapping result, and distributes nodes to the third triangle according to the third mapping result to obtain a third distribution result, specifically including:

[0023] If the vertex calculation result contains a third triangle with three vertices having different hash values, the master node maps the three vertices of the third triangle to all worker nodes in the worker node set to obtain a third mapping result.

[0024] The master node obtains three edges of the third triangle, and distributes all the edges to all worker nodes in the worker node set according to the third mapping result to obtain a third distribution result.

[0025] Optionally, the network information detection method based on dynamic graph stream triangle counting, wherein the triangle sampling according to the node distribution result to obtain a target sampling result specifically comprises:

[0026] a target worker node for obtaining the first distribution result, and the target worker node performs triangle sampling on the original dynamic graph stream according to a random pairing sampling method to obtain a first sampling result;

[0027] a first worker node and a second worker node for obtaining the second distribution result, and the first worker node and the second worker node perform triangle sampling on the original dynamic graph stream according to a random pairing sampling method to obtain a second sampling result;

[0028] all worker nodes for obtaining the third distribution result, and all the worker nodes perform triangle sampling on the original dynamic graph stream according to a random pairing sampling method to obtain a third sampling result;

[0029] updating the initial graph according to the first sampling result, the second sampling result and the third sampling result to obtain a target graph, and obtaining a target sampling result according to the target graph.

[0030] Optionally, the network information detection method based on dynamic graph stream triangle counting, wherein the distributed architecture is used to aggregate and process all triangle sampling information in the target sampling result to obtain a triangle counting result, and the triangle counting result is used to obtain a network information detection result, and the method specifically comprises:

[0031] the aggregation node is used to aggregate and process all triangle sampling information in the target sampling result to obtain a triangle counting result, and the triangle counting result is used for numerical calculation to obtain a global triangle number estimate value and a local triangle number estimate value of each vertex;

[0032] optimizing the global triangle number estimate value and all the local triangle number estimate values to obtain a target global triangle number value and a plurality of target local triangle number values, and the target global triangle number value and all the target local triangle number values are used to obtain a network information detection result.

[0033] Optionally, the network information detection method based on dynamic graph stream triangle counting, wherein the network information detection system based on dynamic graph stream triangle counting comprises:

[0034] an information processing module, configured to obtain network information to be detected, perform graph definition according to the network information to be detected to obtain an original dynamic graph stream, and input the original dynamic graph stream into the constructed distributed architecture;

[0035] a node distribution module, configured to perform distribution processing on each vertex of the original dynamic graph stream according to a vertex hash distribution strategy through the distributed architecture, to obtain a node distribution result, and perform triangle sampling according to the node distribution result, to obtain a target sampling result;

[0036] a triangle counting module, configured to perform summary processing on all triangle sampling information in the target sampling result through the distributed architecture, to obtain a triangle counting result, and obtain a network information detection result according to the triangle counting result.

[0037] In addition, to achieve the above object, the present application also provides a terminal, wherein the terminal comprises a memory, a processor, and a network information detection program based on dynamic graph stream triangle counting stored in the memory and capable of running on the processor, and the network information detection program based on dynamic graph stream triangle counting implements the steps of the network information detection method based on dynamic graph stream triangle counting when executed by the processor.

[0038] In addition, to achieve the above object, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a network information detection program based on dynamic graph stream triangle counting, and the network information detection program based on dynamic graph stream triangle counting implements the steps of the network information detection method based on dynamic graph stream triangle counting when executed by a processor.

[0039] In the present application, network information to be detected is acquired, graph definition is performed according to the network information to be detected, an original dynamic graph stream is obtained, and the original dynamic graph stream is input to a distributed architecture constructed; each vertex of the original dynamic graph stream is distributed through the distributed architecture according to a vertex hash distribution strategy, a node distribution result is obtained, and triangle sampling is performed according to the node distribution result, to obtain a target sampling result; all triangle sampling information in the target sampling result is summarized through the distributed architecture, a triangle counting result is obtained, and a network information detection result is obtained according to the triangle counting result. In the present application, vertices are dynamically distributed to fixed working nodes, it is ensured that three edges of each triangle belong to the same working node, the demand for cross-node communication is eliminated, the sampling probability of a triangle is improved, the uniqueness and accuracy of sampling are ensured, and thus the accuracy and efficiency of the network information detection result are improved. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 is a flow chart of a preferred embodiment of the network information detection method based on dynamic graph stream triangle counting of the present application;

[0041] Figure 2is a schematic diagram of work node distribution when the hash values of three vertices of a triangle are the same in the preferred embodiment of the present application;

[0042] Figure 3 is a schematic diagram of work node distribution when the hash values of two vertices of a triangle are the same in the preferred embodiment of the present application;

[0043] Figure 4 is a schematic diagram of work node distribution when the hash values of three vertices of a triangle are all different in the preferred embodiment of the present application;

[0044] Figure 5 is a structure diagram of the preferred embodiment of the network information detection system based on dynamic graph stream triangle counting of the present application;

[0045] Figure 6 is a structure diagram of the preferred embodiment of the terminal of the present application. DETAILED DESCRIPTION

[0046] To make the objectives, technical solutions, and advantages of the present application clearer and more explicit, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0047] It should be noted that if the embodiments of the present application involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative positional relationship, movement condition, etc. between components in a certain posture (as shown in the drawings), and if the certain posture changes, the directional indications also change accordingly.

[0048] In addition, if the embodiments of the present application involve descriptions of “first”, “second”, etc., the descriptions of “first”, “second”, etc. are only for description purposes and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features with “first” and “second” can explicitly or implicitly include at least one of the features. In addition, the technical solutions of the various embodiments can be combined with each other, but must be based on the realization of a person of ordinary skill in the art, and when the combination of technical solutions contradicts each other or cannot be realized, it should be considered that the combination of technical solutions does not exist and is not within the protection scope claimed by the present application.

[0049] The network information detection method based on dynamic graph stream triangle counting in the preferred embodiment of the present application, as shown in Figure 1 The network information detection method based on dynamic graph stream triangle counting includes the following steps:

[0050] In step S10, the network information to be detected is acquired, a graph definition is performed according to the network information to be detected, an original dynamic graph stream is obtained, and the original dynamic graph stream is input to the constructed distributed architecture.

[0051] Specifically, in order to solve the problem that the sampling rate of the triangle counting of the prior art dynamic graph stream is low, the algorithm estimation error cannot be reduced, the load balancing is poor, the running speed cannot be improved through distributed expansion steps, and the accuracy and efficiency of the network information detection result are low, the present application provides a network information detection method based on dynamic graph stream triangle counting. The network information detection based on dynamic graph stream triangle counting is realized through a VHADS algorithm (Vertex hashing assignment-based distributed sampling, vertex hashing assignment-based distributed sampling algorithm for large-scale dynamic graph stream). The VHADS algorithm uniquely allocates each triangle to a working node through a reasonable vertex hashing allocation strategy, ensures that each triangle is only responsible for sampling by one working node, the working node is responsible for sampling the allocated triangle, and then sends the corresponding sampling result to an aggregation node. The aggregation node is responsible for aggregating the triangle sampling information from all working nodes and performing corresponding processing to obtain the estimated value of the global triangle number and the local triangle number in the entire original dynamic graph stream, which not only improves the sampling probability but also reduces the algorithm estimation error. At the same time, the load balancing is realized through distributed sampling, and the running speed is improved. The specific processing process is as follows. In the embodiment of the present application, the detection of false accounts is taken as an example. Because the social relationship of a real user usually forms a natural triangle, and the relationship chain of a false account often presents a star structure, i.e., a center node connects a large number of isolated nodes, the triangle density is significantly low, and through high-precision sampling, it can be quickly identified whether a newly inserted edge forms an abnormal triangle pattern, so as to effectively identify a botnet and reduce the number of false accounts.

[0052] Firstly, the network information to be detected is acquired, a graph definition is performed according to the network information to be detected, and an original dynamic graph stream is obtained. The triangle counting problem in the dynamic stream graph refers to a data form in which the graph structure dynamically changes over time. By introducing a time dimension, the life cycle events such as addition, update and deletion of nodes and edges in the graph structure are recorded. The process of graph definition is to define the original dynamic graph stream as a sequence of edge operations , wherein is the first arriving edge operation, is the second arriving edge operation, is the th arriving edge operation. The initial graph is obtained through the original dynamic graph stream , and the number of triangles in the initial graph The edges are generated step by step through the arriving operations, where... and These are the sets of vertices and edges in the initial graph, respectively. For each edge... It connects two different nodes, namely ;symbol In order to be in Add or delete edges at any time ,in, This represents changes in the initial graph. In this invention, it is stipulated that only two operations can be performed on the initial graph: adding new edges and deleting existing edges. In the initial graph, for three vertices... , and If there exists an edge between any two vertices, then it is called a vertex. Form a triangle. Use symbols. To represent the set of global triangles in the initial graph, that is, all triangles in the initial graph; in addition, the symbol Represent each vertex The set of local triangles, that is, the set containing the vertices. All triangles. By using symbols To describe the set of triangles and symbols added to the original dynamic graph stream. To describe the set of triangles deleted from the original dynamic graph stream, where, defined For at any time Or a set of triangles that have already been added to the initial graph, the corresponding expression is:

[0053] ;

[0054] in, For time identifiers, For a moment The set of global triangles in the initial graph. for The set of global triangles in the initial graph at time; by introducing time identifiers to distinguish triangles with the same vertex composition but added to the graph at different times.

[0055] definition For at any time Or the set of triangles previously deleted from the initial graph, the corresponding expression is:

[0056] ;

[0057] Similarly, time identifiers are introduced to distinguish triangles that share the same vertices but were deleted at different times. Likewise, when considering each vertex in the initial graph... At that time, adopt to represent the added triangle set containing vertex and to represent the deleted triangle set containing vertex . It should be noted that a triangle with the same vertex composition can be added multiple times or deleted multiple times. Then, the original dynamic graph stream is input to the constructed distributed architecture, wherein the distributed architecture includes a master node, a working node set (including a plurality of working nodes), and an aggregation node, and the core of the distributed architecture is a data distribution technology driven by a vertex hash allocation strategy.

[0058] In step S20, each vertex of the original dynamic graph stream is allocated by the distributed architecture according to the vertex hash allocation strategy, a node allocation result is obtained, and triangle sampling is performed according to the node allocation result to obtain a target sampling result.

[0059] Specifically, after the original dynamic graph stream is input to the constructed distributed architecture, each vertex of the original dynamic graph stream needs to be allocated by the distributed architecture according to the vertex hash allocation strategy. Specifically, all triangles of an initial graph are obtained, and a hash value of each vertex of all the triangles is calculated by the master node to obtain a vertex calculation result. In this embodiment of the application, an edge insertion operation is taken as an example, because the edge insertion operation is consistent with the edge deletion operation. The purpose of calculating the vertex hash value is that, for an insertion edge and a deletion edge with the same vertex composition, the hash value of the vertex is also necessarily the same, so these edges will follow the same vertex hash allocation strategy. Further analysis shows that, for an added triangle and a deleted triangle composed of the same vertex, if the arrival order of the three edges is consistent, then the added triangle and the deleted triangle are necessarily mapped to the same working node. When a given original dynamic graph stream is given, the vertex hash allocation strategy of the VHADS algorithm can ensure that each triangle in the original dynamic graph stream is uniquely mapped to a working node and can only be sampled by the working node. For each edge in the original dynamic graph stream , at most, is stored in two working nodes, because when the hash value of the vertex is not equal to the hash value of the vertex , the edge may be stored in the sample graph of the working node corresponding to the hash value of the vertex and the working node corresponding to the hash value of the vertex . For each triangle in the original dynamic graph stream, the VHADS algorithm can ensure that only the sample graph of one working node can constitute the triangle by the vertex hash allocation strategy. The hash value combination of the three vertices of the triangle contains three modes, which are that the hash values of the three vertices are the same, the hash values of two vertices are the same, and the hash values of the three vertices are all different.

[0060] If the three vertices of a triangle have the same hash value, such as... Figure 2 As shown, if the vertex calculation results contain a first triangle (i.e., a triangle with three vertices having the same hash value), then... ,in, , and If all three vertices are vertices, then the master node maps the three vertices of the first triangle to the target worker node (i.e., the triangle) in the worker node set. All three vertices of the triangle are mapped to the same worker node 1 (i.e., the target worker node), resulting in the first mapping result. The three edges of the first triangle are obtained through the master node, and all edges are assigned to the target worker node according to the first mapping result, resulting in the first assignment result. Therefore, only worker node 1 can sample the triangle; other worker nodes cannot simultaneously have triangles in their local memory. The sampling probability of the three sides of the triangle is 0. Therefore, when the triangle... When the hash values ​​of the three vertices are the same, VHADS can ensure that the triangle is uniquely mapped to a worker node.

[0061] For two vertices of a triangle to have the same hash value, such as Figure 3 As shown, if the vertex calculation results contain two vertices with the same hash value, then a second triangle (i.e., a triangle) is formed. ,in, , and (All are vertices). The master node maps two vertices with the same hash value in the second triangle to the first working node (e.g., working node 2) in the working node set, and maps the remaining vertices of the second triangle to the second working node (e.g., working node 1) in the working node set, obtaining a second mapping result. The master node obtains the edge formed by the two vertices with the same hash value in the second triangle, and according to the second mapping result, assigns the edge to the first working node. The master node then assigns the remaining two edges in the second triangle to both the first and second working nodes, obtaining a second allocation result. At this point, the master node will assign the edges... and edge Broadcasting is performed to worker nodes, and edges may appear simultaneously in the sample graphs of both worker node 1 and worker node 2. and edge Due to the vertex and vertex The hash values ​​of all edges are 2. It will only be sent to worker node 2. Therefore, only worker node 2's local memory may contain triangles simultaneously. The triangle has 3 edges, meaning only worker node 2 can sample it, while worker node 1's local memory can only contain the triangle at most at any given time. The triangle has two edges. Therefore, even when two of the three vertices of a triangle have the same hash value, VHADS can still ensure that the triangle is uniquely mapped to a specific worker node.

[0062] For a triangle whose three vertices have different hash values, such as Figure 4 As shown, if the vertex calculation results contain a third triangle (i.e., a triangle) where the hash values ​​of all three vertices are different, then... ,in, , and If all three vertices are vertices, then the master node maps the three vertices of the third triangle to all worker nodes in the worker node set, obtaining the third mapping result; the master node obtains the three edges of the third triangle, and according to the third mapping result, all edges are assigned to all worker nodes in the worker node set, obtaining the third assignment result; at this time, the triangle... Which worker node an edge is mapped to depends on the arrival times of the three edges. Without loss of generality, assume that the edge arrival times satisfy... (in, For the edge Arrival time, For the edge Arrival time, For the edge (Arrival time), the master node will set the triangle The three edges are broadcast to all worker nodes, and the worker nodes sample the edges based on the vertex hash values. On the edges... Before reaching it, only the sample graph of worker node 3 may contain edges simultaneously. and edge When on the side Upon arrival, only worker node 3 may simultaneously contain triangles in its local memory. With three edges, at most two triangles can appear simultaneously in the local memory of worker node 1 and worker node 2. The triangle has two edges. Therefore, even when the hash values ​​of the three vertices of the triangle are different, VHADS can still ensure that the triangle is uniquely mapped to a certain worker node.

[0063] Afterwards, the target working node obtaining the first allocation result performs triangle sampling on the original dynamic graph stream according to a random pairing sampling method to obtain a first sampling result; the first working node and the second working node obtaining the second allocation result perform triangle sampling on the original dynamic graph stream according to the random pairing sampling method to obtain a second sampling result; all the working nodes obtaining the third allocation result perform triangle sampling on the original dynamic graph stream according to the random pairing sampling method to obtain a third sampling result; the initial graph is updated according to the first sampling result, the second sampling result and the third sampling result to obtain a target graph, and a target sampling result is obtained according to the target graph. The application ensures that each triangle edge belongs to the same node by distributing the vertexes to the fixed working nodes through the hash function, eliminates the cross-node communication demand, improves the triangle sampling probability, ensures the sampling uniqueness and accuracy, and significantly reduces the estimation error. The non-uniform hash is designed according to the vertex degree to prevent the high-density vertexes from excessively concentrating on a single node, and the complex balance is realized by avoiding the calculation tilt through the parallelization to further improve the triangle sampling efficiency.

[0064] In step S30, all the triangle sampling information in the target sampling result is aggregated and processed through the distributed architecture to obtain a triangle counting result, and a network information detection result is obtained according to the triangle counting result.

[0065] Specifically, after obtaining the target sampling result, all the triangle sampling information in the target sampling result is aggregated and processed through the aggregation node to obtain a triangle counting result, and the triangle counting result is numerically calculated to obtain a global triangle number estimation value and a local triangle number estimation value of each vertex. The global triangle number estimation value and all the local triangle number estimation values are optimized to obtain a target global triangle number value and a plurality of target local triangle number values. The optimization process corresponding to the global triangle number estimation value is:

[0066] ;

[0067] Wherein, is a global absolute percentage error, used to measure the difference between the global triangle number estimation value and a global triangle number true value, is the global triangle number estimation value, is the global triangle number true value;

[0068] The optimization process corresponding to the global triangle number estimation value is:

[0069] ;

[0070] wherein, is a root mean square error, used to measure the closeness between the local triangle number estimated value and the local triangle number true value, is a local triangle number true value, is a local triangle number estimated value. Then, a network information detection result is obtained according to the target global triangle number value and all the target local triangle number values. The present application can quickly identify whether a newly inserted edge forms an abnormal triangle pattern in the detection of false accounts in a real social network, effectively identify a "botnet", and reduce the number of false accounts.

[0071] Further, as Figure 5 shown, based on the network information detection method based on dynamic graph stream triangle counting, the present application also correspondingly provides a network information detection system based on dynamic graph stream triangle counting, wherein the network information detection system based on dynamic graph stream triangle counting comprises:

[0072] an information processing module 51, configured to acquire network information to be detected, perform graph definition according to the network information to be detected, obtain an original dynamic graph stream, and input the original dynamic graph stream to a constructed distributed architecture;

[0073] a node allocation module 52, configured to perform allocation processing on each vertex of the original dynamic graph stream according to a vertex hash allocation strategy through the distributed architecture, obtain a node allocation result, perform triangle sampling according to the node allocation result, and obtain a target sampling result;

[0074] a triangle counting module 53, configured to perform summary processing on all triangle sampling information in the target sampling result through the distributed architecture, obtain a triangle counting result, and obtain a network information detection result according to the triangle counting result.

[0075] Further, as Figure 6 shown, based on the network information detection method based on dynamic graph stream triangle counting, the present application also correspondingly provides a terminal, which comprises a processor 10, a memory 20 and a display 30. Figure 6 Only part of the components of the terminal are shown, but it should be understood that all the shown components are not required, and more or less components can be alternatively implemented.

[0076] The memory 20 can be an internal storage unit of the terminal in some embodiments, such as a hard disk or a memory of the terminal. The memory 20 can also be an external storage device of the terminal in other embodiments, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the terminal. Further, the memory 20 can include both the internal storage unit and the external storage device of the terminal. The memory 20 is used to store application software and various data installed on the terminal, such as program codes of the terminal, etc. The memory 20 can also be used to temporarily store data that has been output or will be output. In an embodiment, the memory 20 stores a network information detection program 40 based on dynamic graph stream triangle counting, which can be executed by the processor 10 to implement the network information detection method based on dynamic graph stream triangle counting in the present application.

[0077] The processor 10 can be a Central Processing Unit (CPU), a microprocessor or other data processing chip in some embodiments, which is used to run program codes or process data stored in the memory 20, such as to execute the network information detection method based on dynamic graph stream triangle counting, etc.

[0078] The display 30 can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, etc. in some embodiments. The display 30 is used to display information of the terminal and to display a visualized user interface.

[0079] In an embodiment, the following steps are implemented when the processor 10 executes the network information detection program 40 based on dynamic graph stream triangle counting in the memory 20:

[0080] Obtaining network information to be detected, defining a graph according to the network information to be detected, obtaining an original dynamic graph stream, and inputting the original dynamic graph stream to a constructed distributed architecture;

[0081] Performing allocation processing on each vertex of the original dynamic graph stream according to a vertex hash allocation strategy by the distributed architecture, obtaining a node allocation result, and performing triangle sampling according to the node allocation result to obtain a target sampling result;

[0082] The distributed architecture is used to aggregate all the triangular sampling information in the target sampling result, to obtain a triangular counting result, and to obtain a network information detection result according to the triangular counting result.

[0083] The distributed architecture includes a master node, a set of worker nodes, and an aggregation node, and the node allocation result includes a first allocation result, a second allocation result, and a third allocation result.

[0084] The distributed architecture is used to allocate each vertex of the original dynamic graph stream according to a vertex hash allocation strategy, to obtain a node allocation result, which specifically includes:

[0085] An initial graph is generated according to the original dynamic graph stream, all triangles of the initial graph are obtained, and a vertex calculation result is obtained by calculating the hash values of the vertices of all the triangles through the master node.

[0086] If there is a first triangle with three vertices having the same hash value in the vertex calculation result, the three vertices of the first triangle are mapped to the set of worker nodes through the master node to obtain a first mapping result, and the first triangle is allocated to nodes according to the first mapping result to obtain a first allocation result.

[0087] If there is a second triangle with two vertices having the same hash value in the vertex calculation result, the two vertices with the same hash value in the second triangle are mapped to the set of worker nodes through the master node to obtain a second mapping result, and the second triangle is allocated to nodes according to the second mapping result to obtain a second allocation result.

[0088] If there is a third triangle with three vertices having different hash values in the vertex calculation result, the third triangle is mapped to the set of worker nodes through the master node to obtain a third mapping result, and the third triangle is allocated to nodes according to the third mapping result to obtain a third allocation result.

[0089] If there is a first triangle with three vertices having the same hash value in the vertex calculation result, the three vertices of the first triangle are mapped to the set of worker nodes through the master node to obtain a first mapping result, and the first triangle is allocated to nodes according to the first mapping result to obtain a first allocation result, which specifically includes:

[0090] If there is a first triangle with three vertices having the same hash value in the vertex calculation result, the three vertices of the first triangle are mapped to the target worker node of the set of worker nodes through the master node to obtain a first mapping result.

[0091] The master node obtains three edges of the first triangle, and distributes all the edges to the target worker node according to the first mapping result to obtain a first distribution result.

[0092] If there is a second triangle with two vertices having the same hash value in the vertex calculation result, the master node maps the two vertices with the same hash value in the second triangle to the worker node set to obtain a second mapping result, and performs node distribution on the second triangle according to the second mapping result to obtain a second distribution result, which specifically includes:

[0093] If there is a second triangle with two vertices having the same hash value in the vertex calculation result, the master node maps the two vertices with the same hash value in the second triangle to the first worker node in the worker node set, and maps the remaining vertices of the second triangle to the second worker node in the worker node set to obtain a second mapping result.

[0094] The master node obtains an edge formed by the two vertices with the same hash value in the second triangle, and distributes the edge to the first worker node according to the second mapping result, and distributes the remaining two edges in the second triangle to the first worker node and the second worker node according to the second mapping result to obtain a second distribution result.

[0095] If there is a third triangle with three vertices having different hash values in the vertex calculation result, the master node maps the third triangle to the worker node set to obtain a third mapping result, and performs node distribution on the third triangle according to the third mapping result to obtain a third distribution result, which specifically includes:

[0096] If there is a third triangle with three vertices having different hash values in the vertex calculation result, the master node maps the three vertices in the third triangle to all the worker nodes in the worker node set to obtain a third mapping result.

[0097] The master node obtains three edges of the third triangle, and distributes all the edges to all the worker nodes in the worker node set according to the third mapping result to obtain a third distribution result.

[0098] According to the node distribution result, triangle sampling is performed to obtain a target sampling result, which specifically includes:

[0099] The target worker node of the first distribution result is obtained, and the target worker node is used to perform triangle sampling on the original dynamic graph stream according to a random pairing sampling method to obtain a first sampling result.

[0100] The first working node and the second working node obtaining the second allocation result perform triangle sampling on the original dynamic graph stream according to a random pairing sampling method to obtain a second sampling result;

[0101] All working nodes obtaining the third allocation result perform triangle sampling on the original dynamic graph stream according to a random pairing sampling method to obtain a third sampling result;

[0102] The initial graph is updated according to the first sampling result, the second sampling result and the third sampling result to obtain a target graph, and a target sampling result is obtained according to the target graph.

[0103] The distributed architecture is used to aggregate and process all triangle sampling information in the target sampling result to obtain a triangle count result, and network information detection result is obtained according to the triangle count result, and the specific steps include:

[0104] The aggregation node is used to aggregate and process all triangle sampling information in the target sampling result to obtain a triangle count result, and numerical calculation is performed on the triangle count result to obtain a global triangle number estimate value and a local triangle number estimate value of each vertex;

[0105] The global triangle number estimate value and all the local triangle number estimate values are optimized to obtain a target global triangle number value and a plurality of target local triangle number values, and network information detection result is obtained according to the target global triangle number value and all the target local triangle number values.

[0106] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a network information detection program based on dynamic graph stream triangle counting, and the network information detection program based on dynamic graph stream triangle counting implements the steps of the network information detection method based on dynamic graph stream triangle counting when executed by a processor.

[0107] In summary, this invention provides a network information detection method, system, terminal, and storage medium based on dynamic graph flow triangle counting. The method includes: acquiring network information to be detected; defining a graph based on the network information to obtain an original dynamic graph flow; inputting the original dynamic graph flow into a constructed distributed architecture; allocating each vertex of the original dynamic graph flow according to a vertex hash allocation strategy through the distributed architecture to obtain a node allocation result; performing triangle sampling based on the node allocation result to obtain a target sampling result; summarizing all triangle sampling information in the target sampling result through the distributed architecture to obtain a triangle counting result; and obtaining a network information detection result based on the triangle counting result. This invention improves the accuracy and efficiency of network information detection results by dynamically allocating vertices to fixed working nodes, ensuring that the three edges of each triangle belong to the same working node, eliminating cross-node communication requirements, increasing the sampling probability of triangles, and ensuring the uniqueness and accuracy of sampling.

[0108] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0109] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0110] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A network information detection method based on dynamic graph stream triangle counting, characterized in that, The network information detection method based on the dynamic graph stream triangle counting comprises the following steps: Obtain the network information to be detected, define a graph according to the network information to be detected, obtain an original dynamic graph stream, and input the original dynamic graph stream into a distributed architecture; Distribute each vertex of the original dynamic graph stream according to a vertex hash distribution strategy through the distributed architecture to obtain a node distribution result, and perform triangle sampling according to the node distribution result to obtain a target sampling result; The distributed architecture comprises a master node, a set of worker nodes, and an aggregation node, and the node distribution result comprises a first distribution result, a second distribution result, and a third distribution result; The distribution of each vertex of the original dynamic graph stream according to the vertex hash distribution strategy through the distributed architecture to obtain the node distribution result comprises the following steps: Generate an initial graph according to the original dynamic graph stream, obtain all triangles of the initial graph, calculate the hash values of the vertices of all the triangles through the master node to obtain a vertex calculation result; If there is a first triangle with three vertices having the same hash value in the vertex calculation result, map the three vertices of the first triangle to the set of worker nodes through the master node to obtain a first mapping result, and perform node distribution on the first triangle according to the first mapping result to obtain a first distribution result; If there is a second triangle with two vertices having the same hash value in the vertex calculation result, map the two vertices with the same hash value in the second triangle to the set of worker nodes through the master node to obtain a second mapping result, and perform node distribution on the second triangle according to the second mapping result to obtain a second distribution result; If there is a third triangle with three vertices having different hash values in the vertex calculation result, map the third triangle to the set of worker nodes through the master node to obtain a third mapping result, and perform node distribution on the third triangle according to the third mapping result to obtain a third distribution result; Summarize all triangle sampling information in the target sampling result through the distributed architecture to obtain a triangle counting result, and obtain a network information detection result according to the triangle counting result.

2. The network information detection method based on dynamic graph stream triangle counting according to claim 1, characterized in that, If there is a first triangle with three vertices having the same hash value in the vertex calculation result, map the three vertices of the first triangle to the target worker nodes of the set of worker nodes through the master node to obtain a first mapping result; Obtain the three edges of the first triangle through the master node, and distribute all the edges to the target worker nodes according to the first mapping result to obtain a first distribution result. ​ 3. The network information detection method based on dynamic graph stream triangle counting according to claim 1, characterized in that, If the vertex calculation result contains a second triangle with two vertices having the same hash value, the master node maps the two vertices in the second triangle having the same hash value to the working node set to obtain a second mapping result, and performs node allocation on the second triangle according to the second mapping result to obtain a second allocation result, specifically including: If the vertex calculation result contains a second triangle with two vertices having the same hash value, the master node maps the two vertices in the second triangle having the same hash value to the first working node in the working node set, and maps the remaining vertices in the second triangle to the second working node in the working node set to obtain a second mapping result. The master node obtains an edge formed by the two vertices in the second triangle having the same hash value, and allocates the edge to the first working node according to the second mapping result, and allocates the remaining two edges in the second triangle to the first working node and the second working node according to the second mapping result to obtain a second allocation result.

4. The network information detection method based on dynamic graph stream triangle counting according to claim 1, characterized in that, If the vertex calculation result contains a third triangle with three vertices having different hash values, the master node maps the third triangle to the working node set to obtain a third mapping result, and performs node allocation on the third triangle according to the third mapping result to obtain a third allocation result, specifically including: If the vertex calculation result contains a third triangle with three vertices having different hash values, the master node maps the three vertices in the third triangle to all working nodes in the working node set to obtain a third mapping result. The master node obtains three edges of the third triangle, and allocates all edges to all working nodes in the working node set according to the third mapping result to obtain a third allocation result.

5. The network information detection method based on dynamic graph stream triangle counting according to claim 1, wherein, According to the node allocation result, triangle sampling is performed to obtain a target sampling result, specifically including: Obtain the target working node of the first allocation result, and perform triangle sampling on the original dynamic graph stream according to the random pairing sampling method through the target working node to obtain a first sampling result; Obtain the first working node and the second working node of the second allocation result, and perform triangle sampling on the original dynamic graph stream according to the random pairing sampling method through the first working node and the second working node to obtain a second sampling result; Obtain all working nodes of the third allocation result, and perform triangle sampling on the original dynamic graph stream according to the random pairing sampling method through all working nodes to obtain a third sampling result; According to the first sampling result, the second sampling result and the third sampling result, the initial graph is updated to obtain a target graph, and a target sampling result is obtained according to the target graph.

6. The network information detection method based on dynamic graph stream triangle counting according to claim 1, wherein, The distributed architecture is used to aggregate and process all triangle sampling information in the target sampling result to obtain a triangle count result, and a network information detection result is obtained according to the triangle count result, specifically including: The polymeric node aggregates all the triangular sampling information in the target sampling result, obtains a triangular counting result, and performs numerical calculation on the triangular counting result to obtain a global triangular number estimate value and a local triangular number estimate value of each vertex; The global triangular number estimate value and all the local triangular number estimate values are optimized to obtain a target global triangular number value and a plurality of target local triangular number values, and a network information detection result is obtained according to the target global triangular number value and all the target local triangular number values.

7. A network information detection system based on dynamic graph stream triangle counting, characterized in that, The network information detection system based on dynamic graph stream triangular counting is used to implement the network information detection method based on dynamic graph stream triangular counting in any one of claims 1-6, and the network information detection system based on dynamic graph stream triangular counting comprises: An information processing module is configured to obtain network information to be detected, define a graph based on the network information to be detected, obtain an original dynamic graph stream, and input the original dynamic graph stream into a constructed distributed architecture; A node allocation module is configured to allocate each vertex of the original dynamic graph stream based on a vertex hash allocation strategy through the distributed architecture to obtain a node allocation result, and perform triangular sampling based on the node allocation result to obtain a target sampling result; A triangular counting module is configured to aggregate all the triangular sampling information in the target sampling result through the distributed architecture to obtain a triangular counting result, and obtain a network information detection result based on the triangular counting result.

8. A terminal, characterized by comprising: The terminal comprises a memory, a processor, and a program stored on the memory and executable on the processor, and the program is executed by the processor to implement the steps of the network information detection method based on dynamic graph stream triangular counting in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium has a computer program stored thereon, and the computer readable storage medium stores a network information detection program based on dynamic graph stream triangular counting, and the network information detection program based on dynamic graph stream triangular counting is executed by the processor to implement the steps of the network information detection method based on dynamic graph stream triangular counting in any one of claims 1-6.

Images