Data processing method and device

Abstract

The embodiments of the present specification provide a data processing method and device, the method comprising: determining an initial active graph node in an initial graph network, marking the initial active graph node as target data of a current network layer of a screening network; in the initial graph network, determining a candidate active graph node of a next network layer of the current network layer according to the initial active graph node and the neighbor graph nodes of the initial active graph node, marking the candidate active graph node as target data of the next network layer of the current network layer; taking the next network layer of the current network layer as the current network layer, taking the candidate active graph node as the initial active graph node of the current network layer, continuing to determine the candidate active graph node of the next network layer of the current network layer according to the initial active graph node and the neighbor graph nodes of the initial active graph node until the current network layer is the last layer of the screening network; deleting isolated data in the target data of each network layer of the screening network to obtain a target data set.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of data processing technology, and in particular, to a data processing method. Background Art

[0002] For the training of graph learning of very large graphs, due to limitations of memory and communication, mini-batch is generally used, that is, training subgraphs are generated first, and then the training subgraphs are used for graph learning. This also increases randomness (sampling, small batches, dropout (refers to the process of training deep learning networks, for neural network units, they are temporarily discarded from the network according to a certain probability)), etc.), striving to achieve the effect from local optimality to global optimality. However, the current method of constructing training subgraphs still has the problem of using a large amount of memory. Summary of the invention

[0003] In view of this, an embodiment of this specification provides a data processing method. One or more embodiments of this specification also relate to a data processing device, a computing device, a computer-readable storage medium and a computer program to solve the technical defects existing in the prior art.

[0004] According to a first aspect of an embodiment of this specification, a data processing method is provided, including:

[0005] Determine an initial active graph node in the initial graph network, and mark the initial active graph node as target data of a current network layer of the screening network, wherein the current network layer is an initial network layer of the screening network;

[0006] In the initial graph network, according to the initial active graph node and the neighbor graph nodes of the initial active graph node, determine the candidate active graph node of the next network layer of the current network layer, and mark the candidate active graph node as the target data of the next network layer of the current network layer;

[0007] Taking the next network layer of the current network layer as the current network layer, taking the candidate active graph node as the initial active graph node of the current network layer, continuing to determine the candidate active graph node of the next network layer of the current network layer according to the initial active graph node and the neighbor graph nodes of the initial active graph node, marking the candidate active graph node as the target data of the next network layer of the current network layer, until the current network layer is the last layer of the screening network;

[0008] Determine and delete isolated data in the target data of each network layer of the screening network to obtain a target data set.

[0009] According to a second aspect of an embodiment of this specification, there is provided a data processing device, including:

[0010] A first node determination module is configured to determine an initial active graph node in an initial graph network, and mark the initial active graph node as target data of a current network layer of a screening network;

[0011] A second node determination module is configured to determine, in the initial graph network, a candidate active graph node of a next network layer of the current network layer according to the initial active graph node and the neighbor graph nodes of the initial active graph node, and mark the candidate active graph node as target data of the next network layer of the current network layer;

[0012] A loop module is configured to use the next network layer of the current network layer as the current network layer, use the candidate active graph node as the initial active graph node of the current network layer, continue to determine the candidate active graph node of the next network layer of the current network layer according to the initial active graph node and the neighbor graph node of the initial active graph node, and mark the candidate active graph node as the target data of the next network layer of the current network layer, until the current network layer is the last layer of the screening network;

[0013] The determination module is configured to determine and delete isolated data in the target data of each network layer of the screening network to obtain a target data set.

[0014] According to a third aspect of an embodiment of this specification, a computing device is provided, including:

[0015] Memory and processor;

[0016] The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above data processing method are implemented.

[0017] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned data processing method are implemented.

[0018] According to a fifth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned data processing method.

[0019] The embodiments of the present specification provide a data processing method and device, the method comprising: determining an initial active graph node in an initial graph network, marking the initial active graph node as target data of the current network layer of the screening network; in the initial graph network, determining a candidate active graph node of the next network layer of the current network layer according to the initial active graph node and the neighbor graph node of the initial active graph node, marking the candidate active graph node as target data of the next network layer of the current network layer; taking the next network layer of the current network layer as the current network layer, taking the candidate active graph node as the initial active graph node of the current network layer, and continuing to determine the candidate active graph node of the next network layer of the current network layer according to the initial active graph node and the neighbor graph node of the initial active graph node, marking the candidate active graph node as target data of the next network layer of the current network layer, until the current network layer is the last layer of the screening network; determining and deleting isolated data in the target data of each network layer of the screening network, and obtaining a target data set. By determining the initial active graph node, determining the candidate active graph node in each network layer of the screening network, and determining and deleting the isolated graph node in the candidate active graph node, memory usage can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of a data processing method provided by an embodiment of this specification;

[0021] Figure 2 is a processing flow chart of a data processing method provided by an embodiment of this specification;

[0022] Figure 3 is a structural schematic diagram of a data processing device provided by an embodiment of this specification;

[0023] Figure 4 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0024] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0025] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0026] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0027] First, the terms involved in one or more embodiments of this specification are explained.

[0028] A graph network (GN) is a collection of functions organized in a graph structure in a topological space for relational reasoning. A graph network consists of interconnected graph network blocks (GN blocks), also called "nodes" in neural network implementations. The connections between nodes are called "edges", which represent the dependencies between nodes.

[0029] Graph Neural Network (GNN): refers to the general term for algorithms that use neural networks to learn graph structured data, extract and discover features and patterns in graph structured data, and meet the needs of graph learning tasks such as clustering, classification, prediction, segmentation, and generation.

[0030] Machine learning: It is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance.

[0031] Epoch: All data is sent into the network, completing a forward calculation and back propagation process.

[0032] Dropout: refers to the process of temporarily discarding neural network units from the network with a certain probability during the training process of the deep learning network.

[0033] Batch: A batch of samples in network training.

[0034] In this specification, a data processing method is provided. This specification also relates to a data processing apparatus, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0035] See also Figure 1 , Figure 1 A flow chart of a data processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0036] Step 102: Determine an initial active graph node in the initial graph network, and mark the initial active graph node as target data of a current network layer of a screening network, wherein the current network layer is an initial network layer of the screening network.

[0037] Among them, the initial graph network can be the original graph network containing all nodes; the active graph nodes can be nodes with identifiable attributes, that is, active nodes, for example, nodes with social attributes in the graph network; the screening network can be a machine learning network including multiple layers of convolutional layers; the network layer can be a convolutional layer, an input layer, an output layer, etc.; the initial network layer can be the first layer of the screening network, also called the innermost layer.

[0038] In practical applications, for the training of graph learning of very large graphs (with billions of nodes), due to limitations of memory and communication, mini-batch is generally used, which also increases randomness (sampling, small batches, dropout, etc.) and strives to achieve the effect from local optimality to global optimality. Therefore, it is necessary to first screen the nodes in the graph network.

[0039] In one achievable manner, determining the initial active graph nodes in the initial graph network includes:

[0040] Determining an initial graph node carrying a target attribute identifier in the initial graph network;

[0041] A preset number of sample initial graph nodes are selected from the initial graph nodes as initial active graph nodes.

[0042] The target attribute identifier can be understood as a true value attribute, that is, an identifiable attribute; the preset sample number is the set number of samples for training;

[0043] In practical applications, for true value samples with community attributes (or other identifiable attributes), when each epoch starts training, a preset number of nodes are first selected from these true value samples as true value points.

[0044] For example, there are 100 nodes in the graph network, of which 50 have community attributes. These 50 nodes are used as initial active graph nodes, and 10 nodes are selected from these 50 nodes and input into the innermost layer of the convolutional layer as the initial active graph nodes of the innermost layer.

[0045] The embodiments of the present specification improve the screening efficiency by first determining a portion of the initial active graph nodes and then performing subsequent screening, and the number of initial active graph nodes can be customized, thereby improving the personalization capability.

[0046] Step 104: In the initial graph network, determine candidate active graph nodes of the next network layer of the current network layer based on the initial active graph nodes and the neighboring graph nodes of the initial active graph nodes, and mark the candidate active graph nodes as target data of the next network layer of the current network layer.

[0047] Among them, the neighbor graph node can be a node that has a connection relationship with the initial active graph node. For example, there are node A and node B in the graph network, and there is an edge between node A and node B, then node A and node B have a connection relationship, node A can be called the neighbor graph node of node B, and node B can be called the neighbor graph node of node A; the candidate active graph node can be understood as a node determined to be an active node.

[0048] In practical applications, for each convolutional layer, a graph expansion is performed to determine the active graph nodes of the next network layer.

[0049] For example, there is a node A in the graph network. Node A is input into the innermost layer of the convolution layer as the initial active graph node of the innermost layer. There is an edge between node A and node B. It can be determined that node B is a neighbor graph node of node A. Then, the candidate active graph nodes of the next layer of the innermost layer are determined based on node A and node B, and the candidate active graph nodes are marked as the target data of the next network layer of the innermost layer.

[0050] In an achievable manner, determining, according to the initial active graph node and the neighbor graph nodes of the initial active graph node, a candidate active graph node of the next network layer of the current network layer includes:

[0051] Determine a graph node connected to the initial active graph node as a neighboring graph node of the initial active graph node;

[0052] According to the network layer corresponding to the neighbor graph node, a candidate active graph node of the next network layer of the current network layer is determined.

[0053] For example, there is a node A in the graph network. Node A is input into the innermost layer of the convolution layer as the initial active graph node of the innermost layer. There is an edge between node A and node B. Node B can be determined as the neighbor graph node of node A. The network layer where node B is located is determined. According to the network layer where node B is located, the candidate active graph node of the next layer of the current network layer is determined.

[0054] Specifically, determining the candidate active graph node of the next network layer of the current network layer according to the network layer corresponding to the neighbor graph node includes:

[0055] Determining whether the neighbor graph node belongs to the current network layer;

[0056] If not, the initial active graph node and the neighbor graph node are used as candidate active graph nodes of the next network layer of the current network layer.

[0057] For example, there is node A in the graph network. Node A is input into the innermost layer of the convolution layer as the initial active graph node of the innermost layer. There is an edge between node A and node B. Node B can be determined to be the neighbor graph node of node A. The network layer where node B is located is determined. If the network layer where node B is located is the innermost layer, node B is determined to be the initial active graph node of the innermost layer. It is determined that the candidate active graph nodes of the current network layer include node A and node B.

[0058] For another example, there is a node A in the graph network. Node A is input into the innermost layer of the convolution layer as the initial active graph node of the innermost layer (layer 0). There is an edge between node A and node B. Node B can be determined to be a neighbor graph node of node A. The network layer where node B is located is determined. If the network layer where node B is located is not the innermost layer, it is determined that node B is not the initial active graph node of the innermost layer, and node B is used as a candidate active graph node for the next layer (the first layer).

[0059] In an achievable manner, in the initial graph network, after determining the candidate active graph nodes of the next network layer of the current network layer according to the initial active graph nodes and the neighbor graph nodes of the initial active graph nodes, the method further includes:

[0060] Determine the relevant device nodes of the candidate active graph node, and send a notification message to the relevant device node, wherein the notification message includes the marking status of the candidate active graph node at each network layer.

[0061] Among them, the related device nodes can be device nodes in a distributed system, which refers to device nodes that are different from the device nodes where the candidate active graph node is located and contain the candidate active graph node. For example, node A is in mirror node 1, and the related device nodes of node A include mirror node 2 and mirror node 3; the marking status can be the status of the node in which network layers it is an active graph node.

[0062] In practical applications, if node u is determined to be an active graph node of a certain network layer, point u will be marked as an active graph node of the corresponding network layer on all machines.

[0063] For example, if node A is determined to be the innermost active graph node, then the related device nodes of node A are notified: mirror node 2 and mirror node 3.

[0064] Specifically, notifying the relevant device node of the marking state of the candidate active graph node includes:

[0065] Determine the node type of the device node where the candidate active graph node is located, and determine the notification method for the related device node according to the node type;

[0066] Send a notification message to the relevant device node according to the notification method.

[0067] In actual applications, if the device node where node u is located is a mirror node, it is necessary to notify the master node corresponding to the device node where u is located, and the master node sends a notification message to all mirror nodes where u exists, marking point u as an active node of the corresponding network layer on all machines. If the device node where node u is located is a master node, the master node directly sends a notification message to all mirror nodes where u exists, marking point u as an active node of the corresponding network layer on all machines.

[0068] In one achievable manner, if the device node where the node is located is a mirror node, the specific implementation manner is as follows.

[0069] The determining, according to the node type, a notification method for the related device node includes:

[0070] In the case where the node type is a mirror node, the notification message is sent to the corresponding master node, so that the master node sends the notification message to the relevant mirror node.

[0071] For example, node A is determined to be the innermost active graph node, and the type of the device node where node A is located is determined. If the type of the device node where node A is located is a mirror node, the master node of node A is notified, and the related device nodes of node A include mirror node 2 and mirror node 3. A notification message is sent to mirror node 2 and mirror node 3 through the master node of node A, and the notification message includes that node A is the innermost active graph node, so that mirror node 2 and mirror node 3 update the marking status of node A.

[0072] In another achievable manner, if the device node where the node is located is a master node, the specific implementation manner is as follows.

[0073] The determining, according to the node type, a notification method for the related device node includes:

[0074] In the case where the node type is a primary node, a notification message is sent to the relevant mirror nodes.

[0075] For example, node A is determined to be the innermost active graph node, and the type of the device node where node A is located is determined. The type of the device node where node A is located is the main node, and the related device nodes of node A include mirror node 2 and mirror node 3. A notification message is directly sent to mirror node 2 and mirror node 3, and the notification message includes that node A is the innermost active graph node, so that mirror node 2 and mirror node 3 update the marking status of node A.

[0076] The embodiments of this specification determine the active nodes in each network layer, thereby facilitating subsequent deletion of isolated active nodes and reducing the use of memory resources.

[0077] Step 106: Use the next network layer of the current network layer as the current network layer, use the candidate active graph node as the initial active graph node of the current network layer, continue to determine the candidate active graph node of the next network layer of the current network layer according to the initial active graph node and the neighbor graph nodes of the initial active graph node, mark the candidate active graph node as the target data of the next network layer of the current network layer, until the current network layer is the last layer of the screening network.

[0078] In practical applications, for each convolutional layer, a graph expansion is performed (for example, from layer k to layer K+1, with the innermost layer being layer 0). That is, for each node in layer k, the set N(v) of all its neighboring nodes is found. If there is a node u in N(v) that does not belong to the active point of layer k (active[k], the set of active points of layer k), then point u needs to be added to the active points of layer k+1.

[0079] For example, there is a node A in the graph network. Node A is input into the innermost layer of the convolution layer as the initial active graph node of the innermost layer (layer 0). There is an edge between node A and node B. It can be determined that node B is the neighbor graph node of node A. The network layer where node B is located is determined. If the network layer where node B is located is not the innermost layer, it is determined that node B is not the initial active graph node of the innermost layer. Node B is used as a candidate active graph node of the next layer (the first layer). The neighbor graph node of node B is searched in the next layer. Node C is determined to be the neighbor graph node of node B. The network layer where node C is located is determined. If node C is in the first layer, it is determined that both node B and node C are candidate active graph nodes of the first layer. If node C is no longer in the first layer, node C is used as the initial active graph node of the next layer (the second layer). In this way, the candidate active graph nodes of the entire network layer are determined.

[0080] The embodiments of this specification determine the active nodes in each network layer, thereby facilitating subsequent deletion of isolated active nodes and reducing the use of memory resources.

[0081] Step 108: Determine and delete isolated data in the target data of each network layer of the screening network to obtain a target data set.

[0082] The isolated data may be understood as an isolated graph node. An isolated graph node may be a node that has no connection relationship with any other node, and is also called an isolated node or an isolated point.

[0083] In actual applications, there may be a node in a mirror node that is an isolated graph node on its machine (not connected to any active node). In this case, it needs to be removed from the active node.

[0084] Specifically, the determining and deleting isolated data in the target data of each network layer of the screening network to obtain a target data set includes:

[0085] Determine a network level of a screening network corresponding to first data of the target data, and a network level of a screening network corresponding to second data having no connection relationship with the first data, and determine isolated data from the second data, wherein both the first data and the second data are included in the target data;

[0086] The mark of the target data carried by the isolated data is deleted to obtain a target data set.

[0087] For example, node A is a candidate active graph node, and node A is in the outermost layer. Then the nodes that are connected to node A are determined, including node B and node C. According to the network layer where node B is located and the network layer where node C is located, it is determined whether node A is an isolated point. If node A is determined to be an isolated point, the mark of the target data of the isolated point is deleted.

[0088] In an achievable manner, determining a network level of a screening network corresponding to first data of the target data, and a network level of a screening network corresponding to second data having no connection relationship with the first data, and determining isolated data from the second data includes:

[0089] When the network level of the screening network corresponding to the first data is the last layer of the screening network, and the network level of the screening network corresponding to the second data is not the last layer of the screening network, it is determined that the second data is isolated data.

[0090] In practical applications, for a node in the N-1 layer (in active[N-1] but not in active[N-2]), whether it is an isolated point is determined by whether there is an edge between it and the nodes in the active[N-2] layer.

[0091] For example, node A is a candidate active graph node, and node A is in the outermost layer. Then the nodes that are connected to node A include node B and node C. The network layer where node B is located is the outermost layer, and the network layer where node C is located is the second to last layer. Then it is determined that node A is not an isolated point.

[0092] For another example, node A is a candidate active graph node, and node A is in the outermost layer. Then the nodes that are connected to node A are determined to include node B and node C. The network layer where node B is located is the outermost layer, and the network layer where node C is located is the outermost layer, then node A is determined to be an isolated point.

[0093] In another achievable manner, determining a network level of a screening network corresponding to first data of the target data, and a network level of a screening network corresponding to second data having no connection relationship with the first data, and determining isolated data from the second data includes:

[0094] When the network level of the screening network corresponding to the first data is not the last layer of the screening network, and the network level of the screening network corresponding to the second data is the last layer of the screening network, it is determined that the second data is isolated data.

[0095] In practical applications, for nodes in other layers (belonging to active[N-2]), that is, active nodes in the network layer other than the outermost layer, whether it is an isolated node is determined by whether it has an edge with any point in active[N-1] (outermost layer).

[0096] For example, node A is a candidate active graph node. Node A is in the second layer, and there are three layers of network in total. The nodes connected to node A are determined, including node B and node C. The network layer where node B is located is the outermost layer, and the network layer where node C is located is the second to last layer. It is determined that node A is not an isolated point.

[0097] For another example, node A is a candidate active graph node. Node A is in the second layer, and there are three layers of network in total. The nodes connected to node A are determined to include node B and node C. The network layer where node B is located is the innermost layer, and the network layer where node C is located is the second layer. Therefore, node A is determined to be an isolated point.

[0098] The embodiments of this specification reduce the points involved in calculation and the use of memory resources by cutting out isolated points in active nodes.

[0099] The method of the embodiment of this specification may also include: constructing a target graph network according to the association relationship between the graph nodes corresponding to the target data set.

[0100] Among them, the association relationship between graph nodes can be understood as the edges between graph nodes.

[0101] In practical applications, for the active[k] nodes u, v of each layer in the screening network, all their edges are considered to be the active edges of this layer. Finally, the pruned runtime graph layers are formed through the active edges.

[0102] For example, the target active nodes in the innermost layer include node A and node B, and the edge between node A and node B is taken as the innermost active edge.

[0103] The embodiments of the present specification provide a data processing method and device, the method comprising: determining an initial active graph node in an initial graph network, marking the initial active graph node as target data of the current network layer of the screening network; in the initial graph network, determining a candidate active graph node of the next network layer of the current network layer according to the initial active graph node and the neighbor graph node of the initial active graph node, marking the candidate active graph node as target data of the next network layer of the current network layer; taking the next network layer of the current network layer as the current network layer, taking the candidate active graph node as the initial active graph node of the current network layer, and continuing to determine the candidate active graph node of the next network layer of the current network layer according to the initial active graph node and the neighbor graph node of the initial active graph node, marking the candidate active graph node as target data of the next network layer of the current network layer, until the current network layer is the last layer of the screening network; determining and deleting isolated data in the target data of each network layer of the screening network, and obtaining a target data set. By determining the initial active graph node, determining the candidate active graph node in each network layer of the screening network, and determining and deleting the isolated graph node in the candidate active graph node, memory usage can be reduced.

[0104] The following combination Figure 2 , taking the application of the data processing method provided in this specification in the first node of a distributed system as an example, the data processing method is further described. Figure 2 A processing flow chart of a data processing method provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0105] Step 202: Determine an initial graph node carrying a target attribute identifier in the initial graph network.

[0106] Step 204: Select a preset sample number of initial graph nodes from the initial graph nodes as initial active graph nodes.

[0107] Step 206: Mark the initial active graph node as target data of the current network layer of the screening network.

[0108] Step 208: Determine the graph nodes connected to the initial active graph node as neighbor graph nodes of the initial active graph node.

[0109] Step 210: Determine whether the neighbor graph node belongs to the current network layer. If yes, execute step 212. If no, execute step 214.

[0110] Step 212: Use the initial active graph node and the neighbor graph node as candidate active graph nodes of the current network layer. Execute step 216.

[0111] Step 214: The next network layer of the current network layer is used as the current network layer, and the candidate active graph node is used as the initial active graph node of the current network layer, and the candidate active graph node of the next network layer of the current network layer is determined according to the initial active graph node and the neighbor graph node of the initial active graph node, and the candidate active graph node is marked as the target data of the next network layer of the current network layer, until the current network layer is the last layer of the screening network. Continue from step 208.

[0112] Step 216: Determine the node type of the device node where the candidate active graph node is located, and determine the notification method for the related device node according to the node type.

[0113] Step 218: Send a notification message to the relevant device node according to the notification method.

[0114] Step 220: Determine isolated data based on the network layer where the graph node corresponding to the target data is located and the network layers corresponding to other graph nodes that have no connection relationship with the graph node corresponding to the target data.

[0115] Step 222: Delete the isolated data from all network layers to obtain a target data set.

[0116] By determining initial active graph nodes, determining candidate active graph nodes in each network layer of the screening network, and determining and deleting isolated graph nodes among the candidate active graph nodes to reduce memory usage.

[0117] Corresponding to the above method embodiment, this specification also provides a data processing device embodiment, Figure 3 FIG. 1 is a schematic diagram showing the structure of a data processing device provided by an embodiment of the present specification. Figure 3 As shown, the device comprises:

[0118] A first node determination module 302 is configured to determine an initial active graph node in the initial graph network, and mark the initial active graph node as target data of a current network layer of the screening network;

[0119] The second node determination module 304 is configured to determine, in the initial graph network, a candidate active graph node of a next network layer of the current network layer according to the initial active graph node and the neighbor graph node of the initial active graph node, and mark the candidate active graph node as target data of the next network layer of the current network layer;

[0120] The loop module 306 is configured to use the next network layer of the current network layer as the current network layer, use the candidate active graph node as the initial active graph node of the current network layer, continue to determine the candidate active graph node of the next network layer of the current network layer according to the initial active graph node and the neighbor graph node of the initial active graph node, and mark the candidate active graph node as the target data of the next network layer of the current network layer, until the current network layer is the last layer of the screening network;

[0121] The determination module 308 is configured to determine and delete isolated data in the target data of each network layer of the screening network to obtain a target data set.

[0122] Further, the first node determination module 302 is further configured to:

[0123] Determining an initial graph node carrying a target attribute identifier in the initial graph network;

[0124] A preset number of sample initial graph nodes are selected from the initial graph nodes as initial active graph nodes.

[0125] Further, the second node determination module 304 is further configured to:

[0126] Determine a graph node connected to the initial active graph node as a neighboring graph node of the initial active graph node;

[0127] According to the network layer corresponding to the neighbor graph node, a candidate active graph node of the next network layer of the current network layer is determined.

[0128] Further, the second node determination module 304 is further configured to:

[0129] Determining whether the neighbor graph node belongs to the current network layer;

[0130] Determining whether the neighbor graph node belongs to the current network layer;

[0131] If not, the initial active graph node and the neighbor graph node are used as candidate active graph nodes of the next network layer of the current network layer.

[0132] Furthermore, the determination module 308 is further configured to:

[0133] Determine the isolated data according to the network layer where the graph node corresponding to the target data is located and the network layers corresponding to other graph nodes that have no connection relationship with the graph node corresponding to the target data;

[0134] The isolated data is deleted from all network layers to obtain a target data set.

[0135] Furthermore, the determination module 308 is further configured to:

[0136] When the network layer where the graph node corresponding to the target data is located is the last layer of the screening network, and other graph nodes in other network layers have no connection relationship with the graph node, it is determined that the graph node is isolated data.

[0137] Furthermore, the determination module 308 is further configured to:

[0138] When the network layer where the graph node corresponding to the target data is located is not the last layer of the screening network, and other graph nodes in the last layer of the screening network have no connection relationship with the graph node, it is determined that the graph node is isolated data.

[0139] Further, the second node determination module 304 is further configured to:

[0140] Determine the relevant device nodes of the candidate active graph node, and send a notification message to the relevant device node, wherein the notification message includes the marking status of the candidate active graph node at each network layer.

[0141] Further, the second node determination module 304 is further configured to:

[0142] Determine the node type of the device node where the candidate active graph node is located, and determine the notification method for the related device node according to the node type;

[0143] Send a notification message to the relevant device node according to the notification method.

[0144] Further, the second node determination module 304 is further configured to:

[0145] In the case where the node type is a mirror node, the notification message is sent to the corresponding master node, so that the master node sends the notification message to the relevant mirror node.

[0146] Further, the second node determination module 304 is further configured to:

[0147] In the case where the node type is a primary node, a notification message is sent to the relevant mirror nodes.

[0148] The embodiment of the present specification provides a data processing method and device, wherein the device determines an initial active graph node in an initial graph network, marks the initial active graph node as target data of the current network layer of the screening network; in the initial graph network, according to the initial active graph node and the neighbor graph node of the initial active graph node, determines the candidate active graph node of the next network layer of the current network layer, marks the candidate active graph node as target data of the next network layer of the current network layer; takes the next network layer of the current network layer as the current network layer, takes the candidate active graph node as the initial active graph node of the current network layer, continues to determine the candidate active graph node of the next network layer of the current network layer according to the initial active graph node and the neighbor graph node of the initial active graph node, marks the candidate active graph node as target data of the next network layer of the current network layer, until the current network layer is the last layer of the screening network; determines and deletes isolated data in the target data of each network layer of the screening network, and obtains a target data set. By determining the initial active graph node, determining the candidate active graph node in each network layer of the screening network, and determining and deleting the isolated graph node in the candidate active graph node, memory usage can be reduced.

[0149] The above is a schematic scheme of a data processing device of this embodiment. It should be noted that the technical scheme of the data processing device and the technical scheme of the above data processing method belong to the same concept, and the details of the technical scheme of the data processing device that are not described in detail can be referred to the description of the technical scheme of the above data processing method.

[0150] Figure 4 The block diagram of a computing device 400 according to one embodiment of the present specification is shown. The components of the computing device 400 include but are not limited to a memory 410 and a processor 420. The processor 420 is connected to the memory 410 via a bus 430, and the database 450 is used to store data.

[0151] The computing device 400 also includes an access device 440 that enables the computing device 400 to communicate via one or more networks 460. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 440 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0152] In one embodiment of the present specification, the above components of the computing device 400 and Figure 4 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 4 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0153] Computing device 400 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. Computing device 400 may also be a mobile or stationary server.

[0154] The processor 420 is used to execute the following computer executable instructions, which implement the steps of the above data processing method when executed by the processor.

[0155] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above data processing method.

[0156] An embodiment of the present specification further provides a computer-readable storage medium storing computer-executable instructions, which can implement the steps of the above-mentioned data processing method when executed by a processor.

[0157] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above data processing method.

[0158] An embodiment of the present specification further provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned data processing method.

[0159] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the above data processing method belong to the same concept, and the details not described in detail in the technical scheme of the computer program can be referred to the description of the technical scheme of the above data processing method.

[0160] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0161] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0162] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0163] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0164] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A data processing method, comprising: Determine an initial active graph node in the initial graph network, and mark the initial active graph node as target data of a current network layer of the screening network, wherein the current network layer is an initial network layer of the screening network; In the initial graph network, according to the initial active graph node and the neighbor graph nodes of the initial active graph node, the candidate active graph node of the next network layer of the current network layer is determined, and the candidate active graph node is marked as the target data of the next network layer of the current network layer, wherein, in the initial graph network, after determining the candidate active graph node of the next network layer of the current network layer according to the initial active graph node and the neighbor graph nodes of the initial active graph node, it also includes: determining the relevant device nodes of the candidate active graph node, sending a notification message to the relevant device nodes, the notification message including the marking status of the candidate active graph node in each network layer, the relevant device node is a device node in a distributed system, which refers to a device node that is different from the device node where the candidate active graph node is located and contains the candidate active graph node; Taking the next network layer of the current network layer as the current network layer, taking the candidate active graph node as the initial active graph node of the current network layer, continuing to determine the candidate active graph node of the next network layer of the current network layer according to the initial active graph node and the neighbor graph nodes of the initial active graph node, marking the candidate active graph node as the target data of the next network layer of the current network layer, until the current network layer is the last layer of the screening network; Determine and delete isolated data in the target data of each network layer of the screening network to obtain a target data set, wherein the isolated data is an isolated graph node, and the isolated graph node is a node that has no connection relationship with any node.

2. According to the method of claim 1, the determining of the initial active graph nodes in the initial graph network comprises: Determining an initial graph node carrying a target attribute identifier in the initial graph network; A preset number of sample initial graph nodes are selected from the initial graph nodes as initial active graph nodes.

3. The method according to claim 1, wherein determining the candidate active graph node of the next network layer of the current network layer according to the initial active graph node and the neighbor graph node of the initial active graph node comprises: Determine a graph node connected to the initial active graph node as a neighboring graph node of the initial active graph node; According to the network layer corresponding to the neighbor graph node, a candidate active graph node of the next network layer of the current network layer is determined.

4. The method according to claim 3, wherein determining the candidate active graph node of the next network layer of the current network layer according to the network layer corresponding to the neighbor graph node comprises: Determining whether the neighbor graph node belongs to the current network layer; If not, the initial active graph node and the neighbor graph node are used as candidate active graph nodes of the next network layer of the current network layer.

5. The method according to claim 1, wherein determining and deleting isolated data in the target data of each network layer of the screening network to obtain the target data set comprises: Determine a network level of a screening network corresponding to first data of the target data, and a network level of a screening network corresponding to second data having no connection relationship with the first data, and determine isolated data from the second data, wherein both the first data and the second data are included in the target data; The mark of the target data carried by the isolated data is deleted to obtain a target data set.

6. The method according to claim 5, wherein determining the network level of the screening network corresponding to the first data of the target data and the network level of the screening network corresponding to the second data having no connection relationship with the first data, and determining the isolated data from the second data comprises: When the network level of the screening network corresponding to the first data is the last layer of the screening network, and the network level of the screening network corresponding to the second data is not the last layer of the screening network, it is determined that the second data is isolated data.

7. The method according to claim 5, wherein determining the network level of the screening network corresponding to the first data of the target data and the network level of the screening network corresponding to the second data having no connection relationship with the first data, and determining the isolated data from the second data comprises: When the network level of the screening network corresponding to the first data is not the last layer of the screening network, and the network level of the screening network corresponding to the second data is the last layer of the screening network, it is determined that the second data is isolated data.

8. The method according to claim 1, wherein notifying the relevant device node of the marking status of the candidate active graph node comprises: Determine the node type of the device node where the candidate active graph node is located, and determine the notification method for the related device node according to the node type; Send a notification message to the relevant device node according to the notification method.

9. The method according to claim 8, wherein determining the notification method for the related device node according to the node type comprises: In the case where the node type is a mirror node, sending the notification message to the corresponding master node, so that the master node sends the notification message to the relevant mirror node; In the case where the node type is a primary node, a notification message is sent to the relevant mirror nodes.

10. The method according to claim 8, wherein the screening network is a machine learning network; Accordingly, the method comprises: Determine an initial active graph node in the initial graph network, and mark the initial active graph node as target data of a current network layer of the machine learning network, wherein the current network layer is an initial network layer of the machine learning network; In the initial graph network, according to the initial active graph node and the neighbor graph nodes of the initial active graph node, determine the candidate active graph node of the next network layer of the current network layer, and mark the candidate active graph node as the target data of the next network layer of the current network layer; Taking the next network layer of the current network layer as the current network layer, taking the candidate active graph node as the initial active graph node of the current network layer, continuing to determine the candidate active graph node of the next network layer of the current network layer according to the initial active graph node and the neighbor graph nodes of the initial active graph node, marking the candidate active graph node as the target data of the next network layer of the current network layer, until the current network layer is the last layer of the machine learning network; Determine and delete isolated data in the target data of each network layer of the machine learning network to obtain a target data set.

11. A data processing device, comprising: A first node determination module is configured to determine an initial active graph node in an initial graph network, and mark the initial active graph node as target data of a current network layer of a screening network, wherein the current network layer is an initial network layer of the screening network; A second node determination module is configured to determine, in the initial graph network, a candidate active graph node of the next network layer of the current network layer according to the initial active graph node and the neighbor graph nodes of the initial active graph node, and mark the candidate active graph node as the target data of the next network layer of the current network layer, wherein the second node determination module is further configured to: determine a related device node of the candidate active graph node, and send a notification message to the related device node, wherein the notification message includes a marking status of the candidate active graph node in each network layer, and the related device node is a device node in a distributed system, which refers to a device node that is different from the device node where the candidate active graph node is located and contains the candidate active graph node; A loop module is configured to use the next network layer of the current network layer as the current network layer, use the candidate active graph node as the initial active graph node of the current network layer, continue to determine the candidate active graph node of the next network layer of the current network layer according to the initial active graph node and the neighbor graph node of the initial active graph node, and mark the candidate active graph node as the target data of the next network layer of the current network layer, until the current network layer is the last layer of the screening network; The determination module is configured to determine and delete isolated data in the target data of each network layer of the screening network to obtain a target data set, wherein the isolated data is an isolated graph node, and the isolated graph node is a node that has no connection relationship with any node.

12. A computing device comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the data processing method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the data processing method according to any one of claims 1 to 10.

Images