Method and system for data processing

Abstract

The application discloses a data processing method applied to a distributed system formed by a plurality of servers. The method comprises: all servers in the distributed system obtain a split topological structure chart respectively, wherein the topological structure chart is formed by a node set and a link set; all servers process the split topological structure chart and calculate levels of all nodes; and according to the levels of all nodes, main features of the topological structure chart are extracted. According to the method, multiple servers in the distributed system are used for carrying out synchronous and parallel operation, so that the complicated topological structure chart can be processed rapidly. Therefore, the method has advantages of wide application range and high execution efficiency.

Description

technical field [0001] The present application relates to the field of computer technology, in particular to a data processing method and system. Background technique [0002] With the continuous advancement of science and technology, the Internet has become an indispensable part of daily work and life in today's society. The network scale of the Internet is expanding rapidly, and the network topology is also constantly changing. Extracting the main features of the network topology from the ultra-large-scale complex network topology is of great significance for guiding network construction and Internet information utilization. [0003] A network topology can be abstracted as a topology graph composed of node sets and link sets. [0004] The degree value of a node refers to the number of other nodes connected to it, which reflects the number of adjacent nodes of the node, and shows the importance of the node from one aspect. [0005] The K-core of the topology graph refers...

AI Technical Summary

Problems solved by technology

[0008] It is easy to think that as the complexity of the topology graph increases, and the hardware and software resources of the computer are limited, bottlenecks will inevitably appear.

In reality, when the number of nodes and links is on the order of millions, the computer relies on memory and processors to save the topology graph and its subgraphs of K-cores at all levels, and iterative operations to delete nodes whose degree value is less than or equal to a certain value. The nodes and the links connected to the nodes can still try their best to extract the main features of the topological structure diagram

When the order of magnitude of nodes and links is at the billion level, it will cause memory overflow, and the main features of the topology cannot be extracted.

Therefore, the existing data processing methods have a narrow application range and low execution efficiency.

Images