A method, device and storage medium for processing graph data

Abstract

The present application discloses a method, device and storage medium for processing graph data, which can be applied to the field of maps. The initial graph data is divided into a first partition; a partition subgraph corresponding to the first partition is determined; each partition subgraph is segmented to obtain a subgraph segmentation partition corresponding to each partition subgraph; thumbnails are generated using the subgraph segmentation partitions as graph nodes; the thumbnails are segmented into second partitions, and the nodes in the initial graph data are partitioned based on the second partitions to obtain a target node set. Thus, the process of parallel graph data processing is realized. Since multiple partitions are used, different devices can be used for processing, that is, each device performs graph segmentation on its own data, which can be mapped and restored through thumbnails, making full use of the advantages of parallel processing and improving the processing efficiency of graph data.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device and storage medium for processing graph data. Background Art

[0002] With the rapid development of Internet technology, people have higher and higher requirements for social networks. Social networks can be constructed through graph data. The points in the graph data represent entities, and the edges represent the relationship between two entities. Social networks can be associated or simplified by processing graph data.

[0003] Generally, the METIS algorithm can be used to process graph data, that is, by continuously compressing the graph structure, two nodes are merged into a new node. Therefore, a smaller graph is obtained based on the new node.

[0004] However, in the scenario of large-scale data processing, the METIS algorithm can put all data into the memory, which has a high demand on computer memory and has a high computational cost and a long running time, affecting the efficiency of graph data processing. Summary of the invention

[0005] In view of this, the present application provides a method for processing graph data, which can effectively improve the efficiency of graph data processing.

[0006] In a first aspect, the present application provides a method for processing graph data, which can be applied to a system or program including a graph data processing function in a terminal device, and specifically includes:

[0007] Acquire initial graph data, and divide the initial graph data into K first partitions, each of which corresponds to a similar amount of data, and each of which corresponds to a different execution object, K≥2, where K is a positive integer;

[0008] Determine a partition subgraph corresponding to the first partition;

[0009] Performing graph segmentation on each of the partition subgraphs to obtain T subgraph segmentation partitions corresponding to each of the partition subgraphs, where T≥2 and T is a positive integer;

[0010] Using K*T sub-graph partitions as graph nodes to generate thumbnails corresponding to the initial graph data;

[0011] The thumbnail is divided into K second partitions, and the nodes in the initial graph data are partitioned based on the second partitions to obtain a target node set, where the target node set is used for execution of the recommendation task, and each second partition corresponds to a different execution object.

[0012] Optionally, in some possible implementations of the present application, obtaining initial graph data and dividing the initial graph data into K first partitions includes:

[0013] Acquire the initial graph data;

[0014] Determine a neighbor node corresponding to each node in the initial graph data to generate a first two-tuple;

[0015] Generate R random numbers, and combine the random numbers with the first two-tuple to generate a second two-tuple, 0≤R<K;

[0016] Aggregation is performed based on the random number in the second two-tuple to obtain K first partitions.

[0017] Optionally, in some possible implementations of the present application, the method further includes:

[0018] Determine the processing capability information corresponding to the execution object;

[0019] Determine a unit processing parameter corresponding to the execution object based on the number of the neighboring nodes and the processing capability information;

[0020] The value of K is determined according to the ratio of the data volume corresponding to the initial graph data to the unit processing parameter.

[0021] Optionally, in some possible implementations of the present application, dividing the K*T subgraphs into partitions as graph nodes to generate thumbnails corresponding to the initial graph data includes:

[0022] Divide the K*T subgraphs into partitions as graph nodes;

[0023] Configuring weight information based on the number of nodes of the subgraph partition in the graph node;

[0024] Determine, in the graph nodes, connection nodes whose connection edges cross the edges of the subgraph partitions as shortened edges;

[0025] The number of the thumbnail edges is determined based on the weight information to generate the thumbnail image corresponding to the initial graph data.

[0026] Optionally, in some possible implementations of the present application, determining the number of the thumbnail edges based on the weight information to generate the thumbnail corresponding to the initial graph data includes:

[0027] Obtaining the average node degree corresponding to the thumbnail to determine the thumbnail coefficient;

[0028] Performing node sparse processing on each of the graph nodes based on the reduction coefficient, and extracting the edges after the sparse processing to obtain processed edges;

[0029] Sorting based on the weight information of the processed edges to determine the target edge;

[0030] The thumbnail corresponding to the initial graph data is generated according to the target edge.

[0031] Optionally, in some possible implementations of the present application, the method further includes:

[0032] Determine a partition subgraph corresponding to the second partition;

[0033] Performing graph partitioning on each partition subgraph corresponding to the second partition to obtain T subgraph partitions corresponding to the second partition;

[0034] Use K*T sub-graph partitions corresponding to the second partitions as graph nodes to generate thumbnails corresponding to the second partitions;

[0035] The thumbnail corresponding to the second partition is divided into K third partitions, and the nodes in the initial graph data are partitioned based on the third partitions to obtain a circular node set.

[0036] Optionally, in some possible implementations of the present application, the method further includes:

[0037] Obtain the number of split edges corresponding to the cycle node set;

[0038] If the number of segmented edges converges, a recommendation task is performed based on the set of cycle nodes.

[0039] A second aspect of the present application provides a graph data processing device, including:

[0040] An acquisition unit, used for acquiring initial graph data, and dividing the initial graph data into K first partitions, wherein the amount of data corresponding to each first partition is close, K≥2, and K is a positive integer;

[0041] A determining unit, configured to determine a partition subgraph corresponding to the first partition;

[0042] A processing unit, configured to perform graph segmentation on each of the partition subgraphs to obtain T subgraph segmentation partitions corresponding to each of the partition subgraphs, where T≥2 and T is a positive integer;

[0043] The processing unit is further used to divide the K*T subgraphs into partitions as graph nodes to generate thumbnails corresponding to the initial graph data;

[0044] The processing unit is also used to divide the thumbnail into K second partitions, and partition the nodes in the initial graph data based on the second partitions to obtain a target node set, wherein the target node set is used to recommend the execution of the task, and each second partition corresponds to a different execution object.

[0045] Optionally, in some possible implementations of the present application, the acquisition unit is specifically used to acquire the initial graph data;

[0046] The acquisition unit is specifically used to determine the neighbor nodes corresponding to each node in the initial graph data to generate a first two-tuple;

[0047] The acquisition unit is specifically used to generate R random numbers, and combine the random numbers with the first two-tuple to generate a second two-tuple, 0≤R<K;

[0048] The acquisition unit is specifically configured to perform aggregation based on the random number in the second two-tuple to obtain K first partitions.

[0049] Optionally, in some possible implementations of the present application, the acquisition unit is specifically used to determine the processing capability information corresponding to the execution object;

[0050] The acquisition unit is specifically configured to determine the unit processing parameter corresponding to the execution object based on the number of the neighboring nodes and the processing capability information;

[0051] The acquisition unit is specifically used to determine the value of K based on the ratio of the data volume corresponding to the initial graph data to the unit processing parameter.

[0052] Optionally, in some possible implementations of the present application, the processing unit is specifically configured to divide the K*T subgraphs into partitions as graph nodes;

[0053] The processing unit is specifically used to configure weight information based on the number of nodes of the subgraph partition in the graph node;

[0054] The processing unit is specifically configured to determine, in the graph nodes, connection nodes whose connection edges cross the edges of the subgraph partitions, as thumbnail edges;

[0055] The processing unit is specifically configured to determine the number of the thumbnail edges based on the weight information to generate the thumbnail corresponding to the initial graph data.

[0056] Optionally, in some possible implementations of the present application, the processing unit is specifically used to obtain an average node degree corresponding to the thumbnail to determine a thumbnail coefficient;

[0057] The processing unit is specifically used to perform node sparse processing on each of the graph nodes based on the reduction coefficient, and extract the edges after the sparse processing to obtain processed edges;

[0058] The processing unit is specifically configured to sort the processed edges based on the weight information to determine the target edge;

[0059] The processing unit is specifically configured to generate the thumbnail corresponding to the initial graph data according to the target edge.

[0060] Optionally, in some possible implementations of the present application, the processing unit is specifically configured to determine a partition subgraph corresponding to the second partition;

[0061] The processing unit is specifically configured to perform graph segmentation on the partition subgraph corresponding to each second partition to obtain T subgraph segmentation partitions corresponding to the second partition;

[0062] The processing unit is specifically configured to use K*T sub-graph partitions corresponding to the second partitions as graph nodes to generate thumbnails corresponding to the second partitions;

[0063] The processing unit is specifically used to divide the thumbnail corresponding to the second partition into K third partitions, and partition the nodes in the initial graph data based on the third partitions to obtain a circular node set.

[0064] Optionally, in some possible implementations of the present application, the processing unit is specifically used to obtain the number of split edges corresponding to the cycle node set;

[0065] The processing unit is specifically configured to execute a recommendation task based on the cycle node set if the number of segmentation edges converges.

[0066] The third aspect of the present application provides a computer device, including: a memory, a processor and a bus system; the memory is used to store program code; the processor is used to execute the method for processing graph data described in the first aspect or any one of the first aspects according to the instructions in the program code.

[0067] The fourth aspect of the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, it enables the computer to execute the method for processing graph data described in the first aspect or any one of the first aspects.

[0068] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method for processing graph data provided in the first aspect or various optional implementations of the first aspect.

[0069] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0070] By obtaining the initial graph data, and dividing the initial graph data into K first partitions, the data volume corresponding to each first partition is close, and each of the first partitions corresponds to a different execution object; then determining the partition subgraph corresponding to the first partition; and performing graph segmentation on each partition subgraph to obtain T subgraph segmentation partitions corresponding to each partition subgraph; further using the K*T subgraph segmentation partitions as graph nodes to generate thumbnails corresponding to the initial graph data; and then dividing the thumbnails into K second partitions, and partitioning the nodes in the initial graph data based on the second partitions to obtain a target node set, which is used to recommend the execution of the task. Thus, the process of parallel graph data processing is realized. Since multiple first partitions can be processed by different devices, that is, each device performs graph segmentation on its own data, and can be mapped and restored through thumbnails, the advantages of parallel processing are fully utilized to improve the processing efficiency of graph data. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that can be used in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0072] Figure 1 A network architecture diagram for the graph data processing system.

[0073] Figure 2 A flow chart of the processing of graph data provided in an embodiment of the present application;

[0074] Figure 3 A flowchart of a method for processing graph data provided in an embodiment of the present application;

[0075] Figure 4 A schematic diagram of a scenario of a method for processing graph data provided in an embodiment of the present application;

[0076] Figure 5 A schematic diagram of another method for processing graph data provided in an embodiment of the present application;

[0077] Figure 6 A flowchart of another method for processing graph data provided in an embodiment of the present application;

[0078] Figure 7 A schematic diagram of the structure of a graph data processing device provided in an embodiment of the present application;

[0079] Figure 8 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application;

[0080] Fig. 9 A schematic diagram of the structure of a server provided in an embodiment of the present application;

[0081] Fig. 10A A data sharing system provided in an embodiment of the present application;

[0082] Fig. 10B A block composition of a blockchain provided in an embodiment of the present application;

[0083] Fig. 10C Input information of a blockchain node provided in an embodiment of the present application. DETAILED DESCRIPTION

[0084] The embodiment of the present application provides a method for processing graph data and a related device, which can be applied to a system or program in a terminal device that includes a graph data processing function, by obtaining initial graph data and dividing the initial graph data into K first partitions, wherein the amount of data corresponding to each first partition is close, and each of the first partitions corresponds to a different execution object; then determining the partition subgraph corresponding to the first partition; and performing graph segmentation on each partition subgraph to obtain T subgraph segmentation partitions corresponding to each partition subgraph; further using the K*T subgraph segmentation partitions as graph nodes to generate thumbnails corresponding to the initial graph data; and then dividing the thumbnails into K second partitions, and partitioning the nodes in the initial graph data based on the second partitions to obtain a target node set, which is used to recommend the execution of the task. Thus, the process of parallel graph data processing is realized. Since multiple first partitions can be processed by different devices, that is, each device performs graph segmentation on its own data, and can be mapped and restored through thumbnails, the advantages of parallel processing are fully utilized to improve the processing efficiency of graph data.

[0085] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0086] First, some terms that may appear in the embodiments of the present application are explained.

[0087] Graph partitioning: Divide the node set V of the graph G = (V, E) into k subsets of balanced and non-overlapping sizes, denoted as V1, V2, ..., V k V1, V2, ..., V k , where k is a user-defined hyperparameter, V is the set of nodes, and E is the set of edges. Therefore, for any 1≤i<j≤k, we can have and

[0088] It should be understood that the graph data processing method provided in the present application can be applied to a system or program in a terminal device that includes a graph data processing function, such as a social application. Specifically, the graph data processing system can be run on Figure 1 In the network architecture shown in Figure 1 As shown in FIG. 1 , it is a network architecture diagram of the operation of the graph data processing system. As can be seen from the figure, the graph data processing system can provide a graph data processing process with multiple information sources, that is, through the request operation on the terminal side, trigger the server to process the graph data corresponding to the terminal, and then send the corresponding processing results; it can be understood that Figure 1 A variety of terminal devices are shown in FIG. 1 . The terminal devices may be computer devices. In actual scenarios, more or fewer types of terminal devices may participate in the processing of graph data. The specific number and type depend on the actual scenario and are not limited here. In addition, Figure 1 One server is shown in the figure, but in an actual scenario, multiple servers may be involved, and the specific number of servers depends on the actual scenario.

[0089] In this embodiment, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected through wired or wireless communication, and the terminal and the server can be connected to form a blockchain network, which is not limited in this application.

[0090] It can be understood that the above-mentioned graph data processing system can run on a personal mobile terminal, for example, as an application such as a social application, it can also run on a server, and it can also run on a third-party device to provide graph data processing to obtain the processing results of the graph data of the information source; the specific graph data processing system can be run in the above-mentioned device in the form of a program, and it can also be run as a system component in the above-mentioned device, and it can also be run as a cloud service program. The specific operation mode depends on the actual scenario and is not limited here.

[0091] With the rapid development of Internet technology, people have higher and higher requirements for social networks. Social networks can be constructed through graph data. The points in the graph data represent entities, and the edges represent the relationship between two entities. Social networks can be associated or simplified by processing graph data.

[0092] Generally, the METIS algorithm can be used to process graph data, that is, by continuously compressing the graph structure, two nodes are merged into a new node. Therefore, a smaller graph is obtained based on the new node.

[0093] However, in the scenario of large-scale data processing, the METIS algorithm can put all data into the memory, which has a high demand on computer memory and has a high computational cost and a long running time, affecting the efficiency of graph data processing.

[0094] In order to solve the above problems, this application proposes a method for processing graph data, which is applied to Figure 2 In the process framework of graph data processing shown in Figure 2 As shown, it is a flow architecture diagram of graph data processing provided in an embodiment of the present application. The user generates a corresponding social network on the server through interactive operations of the terminal, and processes the graph data of the social network to determine the segmented set, so as to make targeted object recommendations.

[0095] In the process of graph data processing in this embodiment, the divide-and-conquer and iterative strategies are adopted. First, the data is randomly distributed on different machines. Then, each machine performs graph segmentation on its own data. Then, a communication graph is constructed based on the results of all machines. By segmenting the communication graph, all graph data can be segmented again to obtain the results of this round of calculation. Finally, the final result can be obtained through multiple rounds of iterative calculations.

[0096] It can be understood that the method provided in the present application can be written as a program to serve as a processing logic in a hardware system, or as a graph data processing device, which implements the above processing logic in an integrated or external manner. As an implementation method, the graph data processing device obtains the initial graph data and divides the initial graph data into K first partitions, the data amount corresponding to each first partition is close, and each of the first partitions corresponds to a different execution object; then determines the partition subgraph corresponding to the first partition; and performs graph segmentation on each partition subgraph to obtain T subgraph segmentation partitions corresponding to each partition subgraph; further uses K*T subgraph segmentation partitions as graph nodes to generate thumbnails corresponding to the initial graph data; and then divides the thumbnail into K second partitions, and partitions the nodes in the initial graph data based on the second partitions to obtain a target node set, which is used for the execution of the recommended task. Thereby, the process of parallel graph data processing is realized. Since multiple first partitions can be processed by different devices, that is, each device performs graph segmentation on its own data, and can be mapped and restored through thumbnails, the advantages of parallel processing are fully utilized to improve the processing efficiency of graph data.

[0097] Combined with the above process architecture, the following will introduce the processing method of the image data in this application. Please refer to Figure 3 , Figure 3 A flowchart of a method for processing graph data provided in an embodiment of the present application. The management method may be executed by a server or a terminal. The embodiment of the present application includes at least the following steps:

[0098] 301. Obtain initial graph data, and divide the initial graph data into K first partitions.

[0099] In this embodiment, the initial graph data can be the social network corresponding to the user. The complex social network can be segmented (simplified) to obtain targeted object recommendations, such as friend recommendations, master recommendations, etc. That is, this embodiment applies the distributed graph segmentation algorithm to the friend recommendation system of the game as a recall method for friend recommendations. Specifically, in the master-apprentice recommendation of the competitive game, the distributed graph segmentation algorithm can be used to segment the social network of the competitive game. Figure 4 As shown, Figure 4A scenario diagram of a method for processing graph data provided in an embodiment of the present application; the figure shows a scenario in which a user makes a master recommendation, wherein Player 1, Player 2, and Player 3 are obtained by the graph data processing method of this embodiment. Therefore, each player in the competitive game will belong to a partition. For each player, 50 players can be randomly selected from the partition where the player is located, and then ranked and recommended. This method of selecting within a partition can be compared with the method of selecting within the entire user set, and the experimental results show that there is a significant improvement.

[0100] Specifically, the first partition is a process of partitioning, where the amount of data corresponding to each first partition is close, and each first partition corresponds to a different execution object, K≥2, K is a positive integer, so that a distributed processing process can be performed; that is, given a graph data G=(V,E), where V is a node set and E is an edge set. The initial partition (initial graph data) can divide V into k subsets, represented as V1, V2, ..., V k , so that these subsets satisfy these characteristics: (1) (2) For any i and j, (3) For -1<α<1, |V i |=(1+α)|V| / k, that is, the sizes of the subsets are relatively close.

[0101] Optionally, for the partitioning process of the initial graph data, the initial graph data can be obtained first; then the neighbor nodes corresponding to each node in the initial graph data are determined to generate a first tuple; and R random numbers are generated, and the random numbers are combined with the first tuple to generate a second tuple, 0≤R<K; and then the random numbers in the second tuple are aggregated to obtain K first partitions. This is to reduce the computational complexity and avoid the "chicken and egg" problem. The random partitioning method used is to store the graph data G=(V,E) in the form of an adjacency list, that is, each node u stores a neighbor tuple.<u,nbr(u)> (first tuple), where nbr(u) is the set of neighbor nodes of node u on G; in distributed computing, the neighbor tuples of each node u can be processed<u,nbr(u)> When , a random number generator is used to generate a number r that satisfies 0≤r<k. Therefore, the binary tuple can be changed to a new binary tuple <r,<u,nbr(u)> >(second tuple), where the first element is a random number r, and the second element is the neighbor tuple of node u; the new tuple (second tuple) is aggregated according to the first element (using the Group By function in Spark) so that nodes with the same random number and their neighbor tuples are aggregated into the same partition, forming the input partition of the initial distributed graph partitioning algorithm.

[0102] It is understandable that, due to the random allocation, the sizes of the k initial partitions can be relatively close. In addition, the aggregation operation is used so that the data with the same partition number are aggregated on the same machine.

[0103] In addition, regarding the setting of parameter k, the relationship between machine memory size, subgraph segmentation performance and data volume can be considered. Therefore, the processing capacity information corresponding to the execution object can be determined; then the unit processing parameter corresponding to the execution object can be determined based on the number of neighbor nodes and the processing capacity information; and the value of K can be determined according to the ratio of the data volume corresponding to the initial graph data to the unit processing parameter. That is, assuming that the size of the subgraph that can be well processed on a machine is y (unit processing parameter), and the total amount of data is x (the amount of data corresponding to the initial graph data), then k = x / y. Among them, the value of y should consider the relationship between single-machine graph segmentation and machine memory. Let m represent the machine memory size (processing capacity information), then the amount of data that can be processed by single-machine graph segmentation is y = m / d, where d is the average degree of the subgraph (that is, the average number of neighbors of a node). Therefore, the value of k is k = dx / m.

[0104] It can be understood that the initial graph partitioning can use the label propagation method to calculate an initial partition for each node, thereby improving the partitioning efficiency of the initial graph data.

[0105] 302. Determine a partition subgraph corresponding to the first partition.

[0106] In this embodiment, for the same partition V i (first partition) data, that is, new tuples with the same first element <i,<u,nbr(u)> >, construct a subgraph G based on the same partition i =(V i , E i ), specifically, the first element of the neighbor binary can be extracted from the new binary to form a node set V i ; Then from the new two-tuple, for the second element of the neighbor two-tuple, keep it in the set V i The neighbor node v in , that is, v∈nbr(u) and v∈V i . The retained neighbor node v forms an edge (u,v) with node u and puts it into set E i (Partition subgraph).

[0107] 303. Perform graph partitioning on each partition subgraph to obtain T subgraph partitioning partitions corresponding to each partition subgraph.

[0108] In this embodiment, T≥2, T is a positive integer; that is, given a subgraph G i =(V i , Ei ), can be used for G i Perform graph partitioning so that the set V i Divide into t subsets, expressed as Meet these conditions: (1) (2) For any j and j', (3) For -1<α<1, (4) Minimize the number of edges across different subsets. The METIS algorithm can be used to perform subgraph segmentation on a single machine. Therefore, when the k subgraphs have completed their respective graph segmentation, a total of K*T subsets (subgraph segmentation partitions) are generated.

[0109] In addition, regarding the setting of parameter t, we can consider the number of total subsets in the end, that is, the value of K*T cannot be too large. Because the size of the thumbnail is related to K*T, if t is too large, the thumbnail cannot be processed on a single machine. In order to make the thumbnail exist in the memory of a machine, assuming that the memory size is m and the average node degree of the thumbnail is h, then K*Th=m. In other words, we can set t=m / kh.

[0110] 304. Divide the K*T sub-graphs into partitions as graph nodes to generate thumbnails corresponding to the initial graph data.

[0111] In this embodiment, a thumbnail can be constructed based on K*T subsets (sub-graph partitions), so that the overall structure of the graph can be depicted like a sketch.

[0112] Specifically, we can first divide the K*T subgraphs into partitions as graph nodes, that is, for each subgraph G i The jth subset of Node p that forms a thumbnail i，j ; Then configure the weight information based on the number of nodes in the subgraph partition in the graph node, that is, set the node p i，j The weight of the subset The number of nodes, that is And determine the connecting nodes in the graph whose connecting edges cross the edges of the subgraph partition as the abbreviated edges, that is, for any two nodes and If there is an edge (u, v) ∈ E that spans the subset, then the edge (p i，j , p i′，j′ ); and then determine the number of thumbnail edges based on the weight information to generate a thumbnail corresponding to the initial graph data, that is, set the edge of the thumbnail (p i，j , p i′，j′ ) is the number of edges that span different subsets, that is,

[0113] When the quality of graph segmentation is low, the number of edges in the thumbnail may be relatively large, which will cause the size of the thumbnail to exceed the computer memory and cannot be effectively processed. To solve this problem, a graph sparse method can be used to reduce the number of edges in the thumbnail while ensuring the connectivity of the graph. Specifically, the average node degree corresponding to the thumbnail can be obtained first to determine the thumbnail coefficient, that is, for each node p i，j , take the h / 2 neighbor nodes with the largest weight, so each node retains h / 2 edges in this step; perform node sparse processing on each graph node based on the thumbnail coefficient, and extract the edges after sparse processing to obtain processed edges, for example, take the K*Th / 2 edges with the largest overall weight; merge the edges in the above steps to form a new edge set, and obtain the final sparse thumbnail edge set; then sort based on the weight information of the processed edges to determine the target edge; thereby generating a thumbnail corresponding to the initial graph data according to the target edge.

[0114] Furthermore, one method for taking the first K*Th / 2 edges with the largest weight in the above step is to sort all edges in reverse order by weight, and then take the first K*Th / 2 edges after sorting. However, this method has a computational complexity of O(M log M). When the number of edges in the thumbnail is relatively large, the running time is still very long. In order to improve the running efficiency, the sampling method can be used. Specifically, the number of all edges in the thumbnail can be counted first, expressed as M; then the retention ratio is calculated as q=K*Th / 2M; and a fixed number of edges, such as 10,000 edges, are randomly sampled from all edges; then the sampled edges are sorted in reverse order by edge weight; and the minimum weight of the first 10,000q edges with the largest weight among the sampled edges is calculated, that is, the weight of the edge at position 10,000q after reverse sorting, expressed as w'; then the edges with weights greater than or equal to w' in the thumbnail are retained to form the result set of this algorithm. This method only needs to scan all edges once, so it has linear computational complexity and high running efficiency.

[0115] 305. Divide the thumbnail into K second partitions, and partition the nodes in the initial graph data based on the second partitions to obtain a target node set.

[0116] In this embodiment, the target node set is used to execute the recommendation task. The recommendation task is to select objects from the target node set for recommendation. For example, if the recommendation task is the master-apprentice recommendation in a confrontation game, the distributed graph segmentation algorithm of this embodiment can be used to segment the social network of the confrontation game. Since each player in the confrontation game belongs to a partition, for each player, 50 players can be randomly selected from the partition where the player is located, and then the master-apprentice relationship can be ranked and recommended. That is, the recommendation task is to select objects that have social connections with the object that initiated the recommendation task from the 50 randomly selected players for recommendation. Here, this embodiment is not limited to the recommendation of master-apprentice relationships, but is also applicable to object recommendation scenarios associated with social networks, such as friend recommendations, classmate recommendations, etc.

[0117] It is understandable that the recommendation task is not limited to object recommendation based on social networks, but can also be based on transaction relationship networks, interactive relationship networks, etc., that is, the above-mentioned types of relationship networks are equivalent to the process of segmenting social networks, which is not limited here.

[0118] Specifically, the process of partitioning based on the second partition is given a thumbnail G a =(V a , E a ,w,w′), where V a is a set of nodes, E a is the edge set, w is the node weight, and w′ is the edge weight. Different from the graph partitioning on the subgraph, the partitioning of the thumbnail needs to consider the node weight and the edge weight, so that the thumbnail is partitioned into k subsets, denoted as V1, V2, ..., V k , satisfying these conditions: (1) (2) (3) For -1<α<1, (4) Minimize the number of edges across different subsets. In other words, the graph partitioning on the thumbnail should make the sum of the node weights of each partition close, so that the number of nodes in the corresponding original graph will also be close. The METIS algorithm [1] can be used to perform thumbnail segmentation on a single machine.

[0119] Let γ(p) represent the partition ID of the thumbnail node p, then 1≤γ(p)≤k. For the subset P corresponding to the thumbnail node p, the partition IDs of the nodes in the subset P can also be γ(p). So, at this step, the nodes in the original graph can be divided into k new partitions. The number of partition edges and the number of iterations of the new partition can be calculated to determine whether the next round of iterative calculation can be performed. Otherwise, the current partition result will be returned as the final graph segmentation result.

[0120] In each iteration, this embodiment adopts a divide-and-conquer strategy, performs fine-grained segmentation based on the previous segmentation result, and then performs coarse-grained segmentation based on the segmented blocks, and finally obtains the segmentation result of this layer.

[0121] It can be understood that the above process can be iterated, that is, after determining the second partition, determine the partition subgraph corresponding to the second partition; and perform graph partitioning on each partition subgraph corresponding to the second partition to obtain T subgraph partition partitions corresponding to the second partition; then use the K*T subgraph partition partitions corresponding to the second partitions as graph nodes to generate thumbnails corresponding to the second partitions; and then divide the thumbnails corresponding to the second partition into K third partitions, and partition the nodes in the initial graph data based on the third partitions to obtain a circular node set, thereby realizing the iterative process.

[0122] In one possible scenario, each first partition corresponds to a different execution object. When determining the second partition, the second partition also corresponds to a different execution object, so the third partition also corresponds to a different execution object. That is, this implementation adopts a distributed architecture processing method, which can improve computing efficiency based on the parallel processing graph segmentation process of multiple execution objects.

[0123] It is understandable that different execution objects may be different servers, or different partitions obtained by dividing processing resources in the same server, which is not limited here.

[0124] It should be noted that the initiating object of the recommendation task can be the execution object, that is, the process of performing, for example, friend recommendation through the execution object; the initiating object of the recommendation task can be a business server other than the execution object, for example, a social network server responsible for storing social network information initiates the recommendation task, and the execution object in the computing group performs the graph segmentation process of this embodiment. The specific initiating object depends on the actual scenario.

[0125] In addition, the stopping condition of the iteration may be to obtain the number of split edges corresponding to the set of loop nodes; if the number of split edges converges, the recommendation task is performed based on the set of loop nodes. Alternatively, the iteration may be stopped when the preset number of times is reached.

[0126] In one possible scenario, the graph processing process is as follows Figure 5 As shown, Figure 5 A schematic diagram of another method for processing graph data provided in an embodiment of the present application; the diagram shows the following process:

[0127] (1) The initial node is partitioned so that the node is divided into k partitions. The data size of each partition is similar and can be stored in the machine memory for processing;

[0128] (2) Based on the data of each partition, construct a subgraph on this partition;

[0129] (3) partition the subgraph to obtain t subgraph partitions;

[0130] (4) Building a thumbnail based on the subgraph partitions to describe the general structure of the graph;

[0131] (5) Re-segment the thumbnail and then remap it to the original node to obtain the segmentation partition ID of each node;

[0132] (6) Calculate the number of segmentation edges of the new segmentation result;

[0133] If the number of iterations reaches a certain number or the number of segmented edges reaches a relatively convergent state, the process stops; otherwise, it jumps to step (2) to continue a new round of calculation.

[0134] In combination with the above embodiments, it can be known that by obtaining the initial graph data and dividing the initial graph data into K first partitions, the data amount corresponding to each first partition is close, and each of the first partitions corresponds to a different execution object; then determine the partition subgraph corresponding to the first partition; and perform graph segmentation on each partition subgraph to obtain T subgraph segmentation partitions corresponding to each partition subgraph; further use the K*T subgraph segmentation partitions as graph nodes to generate thumbnails corresponding to the initial graph data; and then divide the thumbnails into K second partitions, and partition the nodes in the initial graph data based on the second partitions to obtain a target node set, which is used to recommend the execution of the task. Thus, the process of parallel graph data processing is realized. Since multiple first partitions can be processed by different devices, that is, each device performs graph segmentation on its own data, and can be mapped and restored through thumbnails, the advantages of parallel processing are fully utilized and the processing efficiency of graph data is improved.

[0135] The above embodiment introduces the processing process of graph data. The following describes the processing process of graph data in combination with the social recommendation scenario. Figure 6 , Figure 6 A flowchart of another method for processing graph data provided in an embodiment of the present application, the embodiment of the present application at least includes the following steps:

[0136] 601. Obtain a recommendation request initiated by a target object.

[0137] In this embodiment, the recommendation request is a friend recommendation request, that is, the distributed graph segmentation algorithm is applied to the friend recommendation system of the game as a recall method for friend recommendation. Specifically, in the master-apprentice recommendation of the adversarial game, the social network of the adversarial game can be first segmented using the distributed graph segmentation algorithm. Therefore, each player in the adversarial game will belong to a partition. For each player, 50 players can be randomly selected from the partition where the player is located, and then ranked and recommended. This method of selecting within the partition can be compared with the method of selecting within the entire user set, and the experimental results show that there is a significant improvement.

[0138] 602. Expand the social network of the target object in response to the recommendation request.

[0139] In this embodiment, the social relationship network of the target object is expanded, that is, the edge points in the social network are expanded through the friend relationship of the target object, or the graph data corresponding to a larger-scale social network is called.

[0140] 603. Perform graph segmentation on the expanded social data to obtain a target recommendation set.

[0141] In this embodiment, the process of performing graph segmentation is shown in Figure 3 The process of steps 301-305 is not described in detail here.

[0142] Specifically, in the master-apprentice recommendation scenario of the confrontation game, 10 other players can be recommended to each player A, so that player A can click on these players to establish a master-apprentice relationship. Since the number of other players that can be recommended to player A may be very large, a candidate set can be calculated first, so that the candidate set can be sorted in a refined manner using a sorting model. The refined sorting model uses xgboost, and the candidate set generation method uses the graph segmentation method of the present technical solution. Specifically, the distributed graph segmentation method of this embodiment can be run based on the social network of friendships between players in the confrontation game, so that each player belongs to a certain segmentation partition. Therefore, for each player A, 100 players can be randomly selected from A's segmentation partition as the recommended candidate set for player A. As a comparison method, 100 players can be randomly selected from all players as a candidate set. Both candidate set generation methods use the same refined sorting model. Table 1 compares the application effects of different algorithms, which shows the application effects of the two methods. The distributed graph segmentation method improves the click rate by 11.62% and the same-day pass rate by 3.12% compared with the random method.

[0143] Table 1 Comparison of application effects of different algorithms

[0144] algorithm Click-through rate % Pass rate on the day % Random Candidate Set 31.58 67.26 Distributed graph partition candidate set 35.25 35.25

[0145] This embodiment designs a large-scale distributed graph segmentation computing framework, which is based on the Spark or MapReduce computing framework and can process graph data of tens of billions of scale. This framework gradually improves the segmentation quality through multiple iterative calculations. Each iterative calculation adopts a two-level method, which uses a divide-and-conquer strategy to first coarsely divide the large graph into several blocks, and then each block is finely segmented. Finally, the fine-grained segmented blocks are re-segmented to obtain the segmentation results of this layer. A thumbnail construction method and a graph sparse method are designed to effectively improve the composition efficiency. The distributed graph segmentation algorithm is also used to generate a recommended candidate set, which improves the business application effect.

[0146] In order to better implement the above solution of the embodiment of the present application, the following also provides related devices for implementing the above solution. Figure 7 , Figure 7 A schematic diagram of the structure of a graph data processing device provided in an embodiment of the present application, wherein the graph data processing device 700 includes:

[0147] An acquisition unit 701 is used to acquire initial graph data and divide the initial graph data into K first partitions, where the amount of data corresponding to each first partition is close, K≥2, and K is a positive integer;

[0148] A determining unit 702, configured to determine a partition subgraph corresponding to the first partition;

[0149] The processing unit 703 is configured to perform graph segmentation on each of the partition subgraphs to obtain T subgraph segmentation partitions corresponding to each of the partition subgraphs, where T≥2 and T is a positive integer.

[0150] The processing unit 703 is further configured to divide the K*T subgraphs into partitions as graph nodes to generate thumbnails corresponding to the initial graph data;

[0151] The processing unit 703 is further used to divide the thumbnail into K second partitions, and partition the nodes in the initial graph data based on the second partitions to obtain a target node set, and the target node set is used for executing the recommendation task.

[0152] Optionally, in some possible implementations of the present application, the acquisition unit 701 is specifically used to acquire the initial graph data;

[0153] The acquisition unit 701 is specifically configured to determine a neighbor node corresponding to each node in the initial graph data to generate a first 2-tuple;

[0154] The acquisition unit 701 is specifically configured to generate R random numbers, and combine the random numbers with the first two-tuple to generate a second two-tuple, 0≤R<K;

[0155] The acquisition unit 701 is specifically configured to perform aggregation based on the random numbers in the second two-tuple to obtain K first partitions.

[0156] Optionally, in some possible implementations of the present application, the acquisition unit 701 is specifically used to determine the processing capability information corresponding to the execution object;

[0157] The acquisition unit 701 is specifically configured to determine a unit processing parameter corresponding to the execution object based on the number of the neighboring nodes and the processing capability information;

[0158] The acquisition unit 701 is specifically used to determine the value of K according to the ratio of the data volume corresponding to the initial graph data to the unit processing parameter.

[0159] Optionally, in some possible implementations of the present application, the processing unit 703 is specifically configured to divide the K*T subgraphs into partitions as graph nodes;

[0160] The processing unit 703 is specifically configured to configure weight information based on the number of nodes of the subgraph partition in the graph node;

[0161] The processing unit 703 is specifically configured to determine, in the graph nodes, connection nodes whose connection edges cross the edges of the subgraph partitions, as thumbnail edges;

[0162] The processing unit 703 is specifically configured to determine the number of the thumbnail edges based on the weight information to generate the thumbnail corresponding to the initial graph data.

[0163] Optionally, in some possible implementations of the present application, the processing unit 703 is specifically configured to obtain an average node degree corresponding to the thumbnail to determine a thumbnail coefficient;

[0164] The processing unit 703 is specifically configured to perform node sparse processing on each of the graph nodes based on the reduction coefficient, and extract the edges after the sparse processing to obtain processed edges;

[0165] The processing unit 703 is specifically configured to sort the processed edges based on the weight information of the processed edges to determine the target edge;

[0166] The processing unit 703 is specifically configured to generate the thumbnail corresponding to the initial graph data according to the target edge.

[0167] Optionally, in some possible implementations of the present application, the processing unit 703 is specifically configured to determine a partition subgraph corresponding to the second partition;

[0168] The processing unit 703 is specifically configured to perform graph segmentation on the partition subgraph corresponding to each second partition to obtain T subgraph segmentation partitions corresponding to the second partition;

[0169] The processing unit 703 is specifically configured to use K*T sub-graph partitions corresponding to the second partitions as graph nodes to generate thumbnails corresponding to the second partitions;

[0170] The processing unit 703 is specifically configured to divide the thumbnail corresponding to the second partition into K third partitions, and partition the nodes in the initial graph data based on the third partitions to obtain a circular node set.

[0171] Optionally, in some possible implementations of the present application, the processing unit 703 is specifically configured to obtain the number of split edges corresponding to the set of loop nodes;

[0172] The processing unit 703 is specifically configured to execute a recommendation task based on the cycle node set if the number of segmentation edges converges.

[0173] By obtaining the initial graph data, and dividing the initial graph data into K first partitions, the data volume corresponding to each first partition is close, and each of the first partitions corresponds to a different execution object; then determining the partition subgraph corresponding to the first partition; and performing graph segmentation on each partition subgraph to obtain T subgraph segmentation partitions corresponding to each partition subgraph; further using the K*T subgraph segmentation partitions as graph nodes to generate thumbnails corresponding to the initial graph data; and then dividing the thumbnails into K second partitions, and partitioning the nodes in the initial graph data based on the second partitions to obtain a target node set, which is used to recommend the execution of the task. Thus, the process of parallel graph data processing is realized. Since multiple first partitions can be processed by different devices, that is, each device performs graph segmentation on its own data, and can be mapped and restored through thumbnails, the advantages of parallel processing are fully utilized to improve the processing efficiency of graph data.

[0174] The present application also provides a terminal device, such as Figure 8The figure is a schematic diagram of the structure of another terminal device provided by the embodiment of the present application. For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The terminal can be any terminal device including a mobile phone, a tablet computer, a personal digital assistant (PDA), a point of sales (POS), a car computer, etc., taking the mobile phone as an example:

[0175] Figure 8 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal provided in an embodiment of the present application. Figure 8 The mobile phone includes: a radio frequency (RF) circuit 810, a memory 820, an input unit 830, a display unit 840, a sensor 850, an audio circuit 860, a wireless fidelity (WiFi) module 870, a processor 880, and a power supply 890. Those skilled in the art will understand that Figure 8 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0176] Combine the following Figure 8 A detailed introduction to the various components of the mobile phone:

[0177] The RF circuit 810 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 880 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 810 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 810 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to the global system of mobile communication (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short messaging service (SMS), etc.

[0178] The memory 820 can be used to store software programs and modules. The processor 880 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 820. The memory 820 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 820 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0179] The input unit 830 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 830 may include a touch panel 831 and other input devices 832. The touch panel 831, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 831 or near the touch panel 831 using any suitable object or accessory such as a finger, stylus, etc., and the air touch operation within a certain range on the touch panel 831), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 831 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 880, and can receive and execute commands sent by the processor 880. In addition, the touch panel 831 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 831, the input unit 830 may also include other input devices 832. Specifically, the other input devices 832 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0180] The display unit 840 may be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 840 may include a display panel 841. Optionally, the display panel 841 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. Further, the touch panel 831 may cover the display panel 841. When the touch panel 831 detects a touch operation on or near it, it is transmitted to the processor 880 to determine the type of touch event. Subsequently, the processor 880 provides a corresponding visual output on the display panel 841 according to the type of touch event. Although in Figure 8 In the embodiment, the touch panel 831 and the display panel 841 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 831 and the display panel 841 can be integrated to realize the input and output functions of the mobile phone.

[0181] The mobile phone may also include at least one sensor 850, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 841 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 841 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0182] The audio circuit 860, the speaker 861, and the microphone 862 can provide an audio interface between the user and the mobile phone. The audio circuit 860 can transmit the received audio data to the speaker 861 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 862 converts the collected sound signal into an electrical signal, which is received by the audio circuit 860 and converted into audio data, and then the audio data is output to the processor 880 for processing, and then sent to another mobile phone through the RF circuit 810, or the audio data is output to the memory 820 for further processing.

[0183] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the WiFi module 870. It provides users with wireless broadband Internet access. Figure 8 A WiFi module 870 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted without changing the essence of the invention.

[0184] The processor 880 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. It executes various functions of the mobile phone and processes data by running or executing software programs and / or modules stored in the memory 820, and calling data stored in the memory 820, thereby managing the mobile phone as a whole. Optionally, the processor 880 may include one or more processing units; optionally, the processor 880 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 880.

[0185] The mobile phone also includes a power supply 890 (such as a battery) for supplying power to various components. Optionally, the power supply can be logically connected to the processor 880 through a power management system, thereby managing functions such as charging, discharging, and power consumption management through the power management system.

[0186] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0187] In the embodiment of the present application, the processor 880 included in the terminal also has the function of executing each step of the above-mentioned page processing method.

[0188] The present application also provides a server. Fig. 9 , Fig. 9 : is a structural diagram of a server provided in an embodiment of the present application. The server 900 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 922 (for example, one or more processors) and memory 932, and one or more storage media 930 (for example, one or more mass storage devices) storing application programs 942 or data 944. Among them, the memory 932 and the storage medium 930 can be short-term storage or permanent storage. The program stored in the storage medium 930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 922 can be configured to communicate with the storage medium 930 to execute a series of instruction operations in the storage medium 930 on the server 900.

[0189] The server 900 may also include one or more power supplies 926, one or more wired or wireless network interfaces 950, one or more input and output interfaces 958, and / or one or more operating systems 941, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0190] The steps performed by the management device in the above embodiment can be based on the Fig. 9 The server structure shown.

[0191] The present application also provides a computer-readable storage medium in which a processing instruction for graph data is stored. When the processing instruction is executed on a computer, the computer executes the above-mentioned Figures 3 to 6 The illustrated embodiment describes the steps performed by the apparatus for processing graph data in the method.

[0192] The present application also provides a computer program product including instructions for processing graph data, which, when executed on a computer, enables the computer to execute the above-mentioned Figures 3 to 6 The illustrated embodiment describes the steps performed by the apparatus for processing graph data in the method.

[0193] The present application also provides a system for processing graph data. The system for processing graph data may include: Figure 7 The apparatus for processing graph data in the described embodiment, or Figure 8 The terminal device in the described embodiment, or Fig. 9 The server being described.

[0194] In a possible scenario, the network resource management method in the present application is applied to a blockchain device, that is, the execution object corresponding to each partition is a blockchain device, and the blockchain device is a node in the blockchain, which is described below in conjunction with the accompanying drawings; see Fig. 10A The data sharing system shown, the data sharing system 1000 refers to a system for sharing data between nodes, and the data sharing system may include multiple nodes 1001, and the multiple nodes 1001 may refer to each client in the data sharing system. Each node 1001 can receive input information when performing normal work, and maintain the shared data in the data sharing system based on the received input information. In order to ensure the information intercommunication within the data sharing system, there may be an information connection between each node in the data sharing system, and information can be transmitted between nodes through the above information connection. For example, when any node in the data sharing system receives input information, other nodes in the data sharing system obtain the input information according to the consensus algorithm, and store the input information as data in the shared data, so that the data stored on all nodes in the data sharing system are consistent.

[0195] Each node in the data sharing system has a corresponding node identifier, and each node in the data sharing system can store the node identifiers of other nodes in the data sharing system, so that the generated blocks can be broadcast to other nodes in the data sharing system according to the node identifiers of other nodes. A node identifier list as shown in the following table can be maintained in each node, and the node name and node identifier are stored in the node identifier list accordingly. Among them, the node identifier can be an IP (Internet Protocol, a protocol for interconnecting networks) address and any other information that can be used to identify the node. Table 2 only takes the IP address as an example for explanation.

[0196] Table 2 Correspondence between node names and node identifiers

[0197] Node Name Node ID Node 1 117.114.151.174 Node 2 117.116.189.145 … … Node N 119.123.789.258

[0198] Each node in the data sharing system stores the same blockchain. The blockchain consists of multiple blocks, see Fig. 10B The blockchain consists of multiple blocks. The genesis block includes a block header and a block body. The block header stores the input information characteristic value, version number, timestamp and difficulty value, and the block body stores the input information; the next block of the genesis block takes the genesis block as the parent block, and the next block also includes a block header and a block body. The block header stores the input information characteristic value of the current block, the block header characteristic value, version number, timestamp and difficulty value of the parent block, and so on, so that the block data stored in each block in the blockchain is associated with the block data stored in the parent block, ensuring the security of the input information in the block.

[0199] When generating each block in the blockchain, see Fig. 10C When the node where the blockchain is located receives the input information, it verifies the input information. After the verification is completed, the input information is stored in the memory pool and the hash tree used to record the input information is updated. After that, the update timestamp is updated to the time when the input information is received, and different random numbers are tried to calculate the eigenvalue multiple times so that the calculated eigenvalue can satisfy the following formula:

[0200] SHA256(SHA256(version+prev_hash+merkle_root+ntime+nbits+x))<TARGET

[0201] Among them, SHA256 is the eigenvalue algorithm used to calculate the eigenvalue; version (version number) is the version information of the relevant block protocol in the blockchain; prev_hash is the block header eigenvalue of the parent block of the current block; merkle_root is the eigenvalue of the input information; ntime is the update time of the update timestamp; nbits is the current difficulty, which is a fixed value within a period of time and is determined again after exceeding the fixed time period; x is a random number; TARGET is the eigenvalue threshold, which can be determined based on nbits.

[0202] In this way, when the random number that satisfies the above formula is calculated, the information can be stored accordingly, the block header and block body can be generated, and the current block can be obtained. Subsequently, the node where the blockchain is located sends the newly generated block to other nodes in the data sharing system according to the node identification of other nodes in the data sharing system. Other nodes verify the newly generated block and add the newly generated block to the blockchain stored in them after the verification is completed.

[0203] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0204] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0205] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0206] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0207] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a data processing device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0208] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for processing graph data, It is characterized in that include: Acquire initial graph data, and divide the initial graph data into K first partitions, each of which corresponds to a similar amount of data, and each of which corresponds to a different execution object, K≥2, where K is a positive integer; Determine a partition subgraph corresponding to the first partition; Performing graph segmentation on each of the partition subgraphs to obtain T subgraph segmentation partitions corresponding to each of the partition subgraphs, where T≥2 and T is a positive integer; Using K*T sub-graph partitions as graph nodes to generate thumbnails corresponding to the initial graph data; Divide the thumbnail into K second partitions, and partition the nodes in the initial graph data based on the second partitions to obtain a target node set, wherein the target node set is used for execution of the recommendation task, and each second partition corresponds to a different execution object; The obtaining of initial graph data and dividing the initial graph data into K first partitions includes: Acquire the initial graph data; Determine a neighbor node corresponding to each node in the initial graph data to generate a first two-tuple; Generate R random numbers, and combine the random numbers with the first two-tuple to generate a second two-tuple, 0≤R<K; Aggregating based on the random number in the second two-tuple to obtain K first partitions; The process of determining the value of K includes: Determine the processing capability information corresponding to the execution object; Determine a unit processing parameter corresponding to the execution object based on the number of the neighboring nodes and the processing capability information; The value of K is determined according to the ratio of the data volume corresponding to the initial graph data to the unit processing parameter.

2. The method according to claim 1, It is characterized in that The step of dividing the K*T subgraphs into partitions as graph nodes to generate thumbnails corresponding to the initial graph data includes: Divide the K*T subgraphs into partitions as graph nodes; Configuring weight information based on the number of nodes of the subgraph partition in the graph node; Determine, in the graph nodes, connection nodes whose connection edges cross the edges of the subgraph partitions as shortened edges; The number of the thumbnail edges is determined based on the weight information to generate the thumbnail image corresponding to the initial graph data.

3. The method according to claim 2, It is characterized in that The determining the number of the thumbnail edges based on the weight information to generate the thumbnail corresponding to the initial graph data includes: Obtaining the average node degree corresponding to the thumbnail to determine the thumbnail coefficient; Performing node sparse processing on each of the graph nodes based on the reduction coefficient, and extracting the edges after the sparse processing to obtain processed edges; Sorting based on the weight information of the processed edges to determine the target edge; The thumbnail corresponding to the initial graph data is generated according to the target edge.

4. The method according to any one of claims 1 to 3, It is characterized in that The method further comprises: Determine a partition subgraph corresponding to the second partition; Performing graph partitioning on each partition subgraph corresponding to the second partition to obtain T subgraph partitions corresponding to the second partition; Use K*T sub-graph partitions corresponding to the second partitions as graph nodes to generate thumbnails corresponding to the second partitions; The thumbnail corresponding to the second partition is divided into K third partitions, and the nodes in the initial graph data are partitioned based on the third partitions to obtain a circular node set.

5. The method according to claim 4, It is characterized in that The method further comprises: Obtain the number of split edges corresponding to the cycle node set; If the number of segmented edges converges, a recommendation task is performed based on the set of cycle nodes.

6. A graph data processing device, It is characterized in that include: An acquisition unit, used for acquiring initial graph data, and dividing the initial graph data into K first partitions, each of which corresponds to a similar amount of data, each of which corresponds to a different execution object, K≥2, and K is a positive integer; A determining unit, configured to determine a partition subgraph corresponding to the first partition; A processing unit, configured to perform graph segmentation on each of the partition subgraphs to obtain T subgraph segmentation partitions corresponding to each of the partition subgraphs, where T≥2 and T is a positive integer; The processing unit is further used to divide the K*T subgraphs into partitions as graph nodes to generate thumbnails corresponding to the initial graph data; The processing unit is further used to divide the thumbnail into K second partitions, and partition the nodes in the initial graph data based on the second partitions to obtain a target node set, wherein the target node set is used for the execution of the recommendation task, and each second partition corresponds to a different execution object; The acquisition unit is specifically used to acquire the initial graph data; The acquisition unit is specifically used to determine the neighbor nodes corresponding to each node in the initial graph data to generate a first two-tuple; The acquisition unit is specifically used to generate R random numbers, and combine the random numbers with the first two-tuple to generate a second two-tuple, 0≤R<K; The acquisition unit is specifically configured to perform aggregation based on the random number in the second two-tuple to obtain K first partitions; The device further comprises: the acquisition unit, which is specifically used to determine the processing capability information corresponding to the execution object; The acquisition unit is specifically configured to determine the unit processing parameter corresponding to the execution object based on the number of the neighboring nodes and the processing capability information; The acquisition unit is specifically used to determine the value of K based on the ratio of the data volume corresponding to the initial graph data to the unit processing parameter.

7. The device according to claim 6, It is characterized in that The device further comprises: the processing unit, specifically configured to divide the K*T subgraphs into partitions as graph nodes; The processing unit is specifically used to configure weight information based on the number of nodes of the subgraph partition in the graph node; The processing unit is specifically configured to determine, in the graph nodes, connection nodes whose connection edges cross the edges of the subgraph partitions, as thumbnail edges; The processing unit is specifically configured to determine the number of the thumbnail edges based on the weight information to generate the thumbnail corresponding to the initial graph data.

8. The device according to claim 7, It is characterized in that The device further comprises: the processing unit, which is specifically used to obtain the average node degree corresponding to the thumbnail image to determine the thumbnail coefficient; The processing unit is specifically used to perform node sparse processing on each of the graph nodes based on the reduction coefficient, and extract the edges after the sparse processing to obtain processed edges; The processing unit is specifically configured to sort the processed edges based on the weight information to determine the target edge; The processing unit is specifically configured to generate the thumbnail corresponding to the initial graph data according to the target edge.

9. The device according to any one of claims 6 to 8, It is characterized in that The apparatus further includes: the processing unit, specifically configured to determine a partition subgraph corresponding to the second partition; The processing unit is specifically configured to perform graph segmentation on the partition subgraph corresponding to each second partition to obtain T subgraph segmentation partitions corresponding to the second partition; The processing unit is specifically configured to use K*T sub-graph partitions corresponding to the second partitions as graph nodes to generate thumbnails corresponding to the second partitions; The processing unit is specifically used to divide the thumbnail corresponding to the second partition into K third partitions, and partition the nodes in the initial graph data based on the third partitions to obtain a circular node set.

10. The device according to claim 9, It is characterized in that The device further comprises: the processing unit, specifically configured to obtain the number of split edges corresponding to the set of cycle nodes; The processing unit is specifically configured to execute a recommendation task based on the cycle node set if the number of segmentation edges converges.

11. A computer device, It is characterized in that The computer device comprises a processor and a memory: The memory is used to store program code; the processor is used to execute the method for processing graph data described in any one of claims 1 to 5 according to instructions in the program code.

12. A computer program product comprising a computer program / instructions, It is characterized in that When the computer program / instructions are executed by a processor, the steps of the method for processing graph data described in any one of claims 1 to 5 are implemented.

Images