Atlas processing method, atlas processing device, electronic equipment and storage medium
By merging nodes in super-large maps, the target map is formed, and the problems of low graph computing efficiency and high storage resources are solved, and effective compression and efficiency improvement of the map are achieved.
Patent Information
- Application Number
- CN202411081827.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-11-08
AI Technical Summary
In the prior art, ultra-large maps are difficult to effectively process and apply due to large data volume and sparseness, resulting in low computing efficiency and high storage resources.
By obtaining the node neighbor relationship in the to-process map, determining the node feature information, forming a set of nodes to be merged, and merging some nodes to generate a target map, thereby achieving the compression and efficiency improvement of the map.
The calculation complexity and storage resource requirements of the map are reduced, the adjacent relationships in the map are retained, the effectiveness and accuracy of the target map are ensured, and the efficiency of map processing is improved.
Smart Images

Figure CN118916495A_ABST
Abstract
Description
Background Art
[0002] At present, a large amount of data is generated in various application fields. In order to mine data information and data relationships, people often build graphs based on data. However, graphs built based on actual data often have the problem that the information in them is difficult to use due to the large amount of data. For example, in a graph built based on e-commerce user and product data, there may be millions or even tens of millions of user nodes alone. Considering the associations between user nodes, such as friend relationships, and the associations between users and products, a graph containing massive data will be obtained. On the one hand, such a super-large graph consumes a lot of computing and storage resources, and it takes more time to process the tasks of adding, deleting, modifying and querying; on the other hand, the data of such a super-large graph is too sparse. For example, a user node will not have any connection with most user nodes, and the corresponding adjacency matrix is a sparse matrix, which leads to the inability to obtain good application effects when applied to other scenarios. For example, when used to train a graph embedding model, the model cannot converge, resulting in the inability to convert the graph data of the graph into vector data, which is difficult to use for further analysis.
[0003] Therefore, how to effectively process graphs and improve the efficiency of graph-based computing while reducing the storage pressure of graph resources is an urgent problem to be solved in existing technologies.
[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention
[0005] The present disclosure provides a graph processing method, a graph processing device, an electronic device and a computer-readable storage medium, thereby overcoming, at least to a certain extent, the problems in the prior art of low computing efficiency and high storage resources caused by dense graphs.
[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.
[0007] According to one aspect of the present disclosure, there is provided a graph processing method, comprising: obtaining a graph to be processed; the graph to be processed comprises a plurality of nodes; determining characteristic information of the nodes according to the node adjacency relationship; forming the nodes into one or more sets of nodes to be merged according to the characteristic information of the nodes; merging at least some of the nodes in the sets of nodes to be merged, and determining a target graph according to the merged nodes.
[0008] According to one aspect of the present disclosure, there is provided a graph processing device, comprising: a graph acquisition module for acquiring a graph to be processed; the graph to be processed comprises a plurality of nodes; a feature information determination module for determining feature information of the nodes according to the node adjacency relationship; a node set formation module for forming one or more to-be-merged node sets according to the feature information of the nodes; and a target graph determination module for merging at least some of the nodes in the to-be-merged node sets, and determining a target graph according to the merged nodes.
[0009] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above-mentioned methods by executing the executable instructions.
[0010] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any one of the above methods is implemented.
[0011] The exemplary embodiments of the present disclosure have the following beneficial effects:
[0012] Obtain a graph to be processed; the graph to be processed includes multiple nodes; determine the characteristic information of the nodes according to the node adjacency relationship; form one or more node sets to be merged according to the characteristic information of the nodes; merge at least some of the nodes in the node set to be merged, and determine the target graph according to the merged nodes. On the one hand, this exemplary embodiment proposes a new graph processing method, which can divide the nodes in the graph to be processed into sets, and reduce the nodes in the graph to be processed by merging the nodes in the node set to be merged, so as to achieve the purpose of compressing the graph, and reduce the problem of high computational complexity and low processing efficiency of graph application due to the over-density of the graph; on the other hand, this exemplary embodiment can form a node set to be merged according to the characteristic information of the nodes determined according to the adjacency relationship of the nodes, and divide the node set to be merged based on the adjacency relationship of the nodes in the graph, so that when the nodes in the node set to be merged are merged, the adjacency relationship of the nodes in the graph can be retained as much as possible, and the influence of node merging on the graph relationship can be reduced, and the validity and accuracy of the target graph can be guaranteed while compressing the graph.
[0013] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.
[0015] Figure 1 A flow chart schematically illustrates a method for processing a graph in this exemplary embodiment;
[0016] Figure 2 A schematic diagram schematically illustrates a graph in this exemplary embodiment;
[0017] Figure 3 A sub-flow chart schematically illustrates a method for processing a graph in this exemplary embodiment;
[0018] Figure 4 A schematic diagram schematically illustrates another type of graph in this exemplary embodiment;
[0019] Figure 5 Another sub-flow chart of a graph processing method in this exemplary embodiment is schematically shown;
[0020] Figure 6 Schematically showing another sub-flow chart of a graph processing method in this exemplary embodiment;
[0021] Figure 7 A schematic diagram schematically illustrates another type of spectrum in this exemplary embodiment;
[0022] Figure 8 A structural block diagram of a graph processing device in this exemplary embodiment is schematically shown;
[0023] Fig. 9 An electronic device for implementing the above method in this exemplary embodiment is schematically shown. DETAILED DESCRIPTION
[0024] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0025] An exemplary embodiment of the present disclosure first provides a graph processing method.
[0026] The following is combined with Figure 1This exemplary embodiment is further described as follows. Figure 1 As shown, the graph processing method may include the following steps S110 to S140:
[0027] Step S110, obtaining a graph to be processed; the graph to be processed includes a plurality of nodes.
[0028] Among them, the graph to be processed refers to any graph including multiple nodes that needs to be compressed. The graph can be a knowledge graph, or other network relationship topology graph including nodes and edges.
[0029] In this exemplary embodiment, the graph to be processed may include multiple nodes and edges between nodes. The types of nodes may be the same or different. For example, 1 、a 2 、a 3 The node can represent the student object information node, b 1 , b 2 , b 3 The node can represent the node of teacher object information, etc., then a 1 、a 2 、a 3 For nodes of the same type, b 1 , b 2 , b 3 are nodes of the same type, and a 1 , b 1 For nodes of different types, etc. Nodes can be associated with each other and connected by edges, or they can be unassociated and become independent nodes.
[0030] like Figure 2 As shown, the graph contains type A nodes, such as A 1 , A 2 Node, B-type node, such as B 1 , B 2 and B 3 Node, C type node, such as C 1 The edge types between any two types of nodes are different. For example, the relationship type between type A nodes and type A nodes is recorded as R AA ,For example The relationship type between the edge and the B node is denoted by R BB ,For example Wait a minute.
[0031] To facilitate subsequent calculations, this exemplary embodiment can initialize the feature vector of each node according to the node attribute information. Specifically, the node can be converted into text, and the node attributes and attribute values can be concatenated after the node name. Different attribute values are distinguished by semicolons, and features with missing values are processed with a special identifier null. For example, a user node can be converted into the text "User: ID 999; Nickname: Zhang San; Date of birth: null; Registration date: January 1, 2024". Then, the text is input into the text embedding model to obtain an initialization vector. For example, a 512-dimensional vector can be obtained by inputting the text into the "bge-small-zh-v1.5" text embedding model. This exemplary embodiment initializes the node vector based on the node's text data, which can retain the node's semantic information so that the vectors corresponding to nodes with similar information are also similar.
[0032] Step S120, determining feature information of the node according to the node neighbor relationship.
[0033] The node neighbor relationship can be the graph structure relationship of the node in the graph, such as which nodes are connected to the node by edges, which first-level neighbor nodes or second-level neighbor nodes are included, the attribute information of specific neighbor nodes, etc. Feature information refers to feature data determined based on the node neighbor relationship, which can be information directly determined by the neighbor relationship, such as the node feature information can be the node's neighbor node set, which includes the neighbor nodes directly connected to the node, feature information can also be feature vectors, such as feature vectors determined based on the node's own attribute information and the node neighbor relationship, etc., which can be set according to actual needs.
[0034] Step S130: According to the characteristic information of the nodes, the nodes are formed into one or more sets of nodes to be merged.
[0035] The set of nodes to be merged refers to a set used for merging nodes. This exemplary embodiment can divide multiple nodes in the graph into different sets of nodes to be merged, and then achieve compression of the graph and reduction of nodes by merging the nodes in the set of nodes to be merged. The formation of a specific set of nodes to be merged can be determined based on the characteristic information of different nodes. When the characteristic information of the node is the adjacent nodes of the node, it can be divided according to the similarity of the adjacent nodes. For example, nodes with the same adjacent nodes are divided into a set of nodes to be merged, or nodes with similar adjacent nodes are divided into a set of nodes to be merged, etc. When dividing the set of nodes to be merged, the types of nodes can also be combined to make the node types in the set of nodes to be merged consistent. For example, the nodes can be first classified by type, and then the set of nodes to be merged can be determined under each type, or the nodes can be first divided into different sets of nodes to be merged, and then the nodes in each set of nodes to be merged are classified by type to obtain a set of nodes to be merged with consistent node types within the set, etc. This order is not specifically limited in the present disclosure. When the characteristic information of a node is a characteristic vector of the node, for example, a characteristic vector determined according to the attributes of the node itself and / or the attributes of the node's adjacent nodes, one or more sets of nodes to be merged can be formed by calculating the similarity between the nodes. For example, nodes of the same type can be randomly divided into different sets, and then the similarity of the nodes in the different sets can be calculated. The nodes in the sets can be continuously updated to improve the similarity of the nodes in each set. When the iteration is completed, the multiple sets determined are the sets of nodes to be merged, etc.
[0036] Step S140, merging at least part of the nodes in the node set to be merged, and determining the target graph according to the merged nodes.
[0037] Finally, at least part of the nodes in each node set to be merged can be merged, for example, the nodes in the node set to be merged can be merged into one node, such as selecting a representative node from multiple nodes, or calculating a central node based on multiple nodes; the nodes in the node set to be merged can also be merged into multiple nodes, and the number of merged nodes is less than the original number of nodes in the node set to be merged, and then, the target graph is determined based on the merged nodes. Compared with the graph to be processed, the nodes in the target graph are merged and reduced, thereby achieving graph compression.
[0038] When merging at least part of the nodes in the node set to be merged, the node set to be merged can also be divided into multiple subsets, and the nodes in the node set to be merged can be merged by merging the nodes in the subsets. For example, the nodes in each subset are merged, and one subset corresponds to one merged node. If the node set to be merged includes several subsets, several merged nodes will be obtained, and so on.
[0039] In an exemplary embodiment, the above step S140 may include:
[0040] Determine a central node according to the nodes included in the set of nodes to be merged;
[0041] Keep the central node and delete other nodes in the set of nodes to be merged;
[0042] According to the central node, the target map is determined.
[0043] The central node refers to a point that can represent a node in the set of nodes to be merged. It can be one of the nodes already included in the set of nodes to be merged, or a new node calculated based on the nodes in the set of nodes to be merged. For example, the central node can be obtained by calculating the average value of the feature vectors of the nodes included in the set of nodes to be merged. Then, the target graph can be determined based on the central node by retaining the central node and deleting other nodes in the set of nodes to be merged. For example, the target graph can be determined based on the merged central node and other nodes that have not been deleted, such as nodes that are not classified into the set of nodes to be merged.
[0044] Based on the above description, in this exemplary embodiment, a graph to be processed is obtained; the graph to be processed includes multiple nodes; the characteristic information of the nodes is determined according to the node adjacent relationship; the nodes are formed into one or more node sets to be merged according to the characteristic information of the nodes; at least part of the nodes in the node set to be merged are merged, and the target graph is determined according to the merged nodes. On the one hand, this exemplary embodiment proposes a new graph processing method, which can divide the nodes in the graph to be processed into sets, and by merging the nodes in the node set to be merged, the nodes in the graph to be processed are reduced to achieve the purpose of compressing the graph, and the problem of high computational complexity and low processing efficiency of the graph application caused by the graph being too dense is reduced; on the other hand, this exemplary embodiment can form a node set to be merged according to the characteristic information of the nodes determined according to the adjacent relationship of the nodes, and divide the node set to be merged based on the adjacent relationship of the nodes in the graph, so that when the nodes in the node set to be merged are merged, the adjacent relationship of the nodes in the graph can be retained as much as possible, and the influence of node merging on the graph relationship can be reduced, and the validity and accuracy of the target graph can be guaranteed while compressing the graph.
[0045] In an exemplary embodiment, the above-mentioned graph processing method further includes:
[0046] If there are isolated nodes in the graph to be processed, the isolated nodes are clustered, and based on the clustering results of the isolated nodes, a set of nodes to be merged including the isolated nodes is formed; an isolated node is a node without adjacent nodes.
[0047] In the graph, there may be some nodes that have no association with other nodes, that is, there are nodes without adjacent nodes. For example, in the graph, there are some nodes that have no association with any other nodes, such as some user nodes that have never purchased any products and have never established friendships with any user nodes. These nodes can be determined as isolated nodes. For isolated nodes, this exemplary embodiment can perform clustering on the isolated nodes to form a set of nodes to be merged including the isolated nodes based on the clustering results of the isolated nodes. Specifically, clustering can be performed based on the similarity between the isolated nodes to obtain clustering results, or clustering can be performed based on the type of the isolated nodes to obtain clustering results, etc. In this exemplary embodiment, the clustering results can be directly used as the set of nodes to be merged, or the clustering results can be further divided into subsets, and the subsets can be used as the set of nodes to be merged, etc. This disclosure does not make specific limitations on this. Then, the isolated nodes in the node set to be merged can be merged to obtain one or more merged nodes. For example, the isolated nodes in the node set to be merged can be merged into one node, or the isolated nodes in the node set to be merged can be merged into multiple nodes which are less than the initial number of nodes in the node set to be merged. Finally, the target graph is generated according to the merged nodes to reduce the number and complexity of nodes in the graph to be processed and realize compression of the graph to be processed.
[0048] In an exemplary embodiment, if there are isolated nodes in the graph to be processed, clustering the isolated nodes, and forming a set of nodes to be merged including the isolated nodes according to the clustering results of the isolated nodes, may include the following steps:
[0049] Cluster the isolated nodes in the graph to be processed according to the node type, and obtain clustering results corresponding to multiple node types;
[0050] In each clustering result, determine the number of node sets to be merged and the maximum number of isolated nodes in the node sets to be merged;
[0051] According to the number of node sets to be merged and the maximum capacity, multiple node sets to be merged are formed in each clustering result.
[0052] This exemplary embodiment can adopt a clustering method according to the type of isolated nodes to obtain clustering results of multiple isolated nodes under different node categories. Each clustering result can include multiple isolated nodes. Clustering can adopt a variety of algorithms. For example, K-means algorithm can be used to cluster isolated nodes to obtain clustering results.
[0053] In this exemplary embodiment, in order to ensure the effectiveness of node merging and avoid excessive merging affecting the relationship of nodes in the graph, after clustering the isolated nodes to obtain the clustering results, this exemplary embodiment can also determine the number of node sets to be merged and the maximum number of isolated nodes in the node sets to be merged in each clustering result, and divide multiple subsets according to the number of node sets to be merged and the maximum number of isolated nodes to be merged, and use the subsets as node sets to be merged. By merging the isolated nodes in each subset, the merged nodes corresponding to different subsets are determined, and the target graph is determined according to the merged nodes corresponding to each subset. For example, the clustering results of isolated nodes of type A can be divided into 5 subsets, that is, the clustering results of isolated nodes of type A can correspond to 5 node sets to be merged. By merging the isolated nodes in the subsets, each subset corresponds to a merged node, and then 5 merged isolated nodes of type A can be obtained, or each subset corresponds to two merged nodes, then 10 merged isolated nodes of type A can be obtained, and the specific number and standard of merging can be set according to actual needs, and the present disclosure does not make specific limitations on this.
[0054] This exemplary embodiment can determine the central node based on the isolated nodes of the node set to be merged, and use the central node as the node that constitutes the target graph. The central node refers to a node in the node set to be merged that can represent some or all of the isolated nodes therein. It can be a node selected from multiple isolated nodes, such as randomly selecting an isolated node as the central node, or it can be a node calculated based on the isolated nodes in the node set to be merged, such as taking the average value of all isolated nodes in the node set to be merged as the central node. After determining the central node, the merging of the node set to be merged can be achieved by retaining the central node and deleting other isolated nodes. Finally, the target graph can be determined based on the central node, for example, the target graph can be determined based on the central isolated node and the nodes that have not been merged in other to-be-processed graphs.
[0055] Specifically, in this exemplary embodiment, the division of the to-be-merged node set of the isolated node and the merging of the nodes may further include the following steps:
[0056] Step S310: cluster all isolated nodes according to node type to obtain multiple clustering results; the isolated nodes in each clustering result are of the same node type; for example, clustering result 1 includes all isolated nodes of type A, and clustering result 2 includes all isolated nodes of type B;
[0057] Step S320, under each clustering result, the subsets are divided again to determine the number K of isolated nodes that can be accommodated in each subset, for example, K=10, and based on the number of isolated nodes that currently need to be allocated to subsets, the number M of subsets that need to be divided is determined;
[0058] Step S330, randomly selecting M nodes as the initial central nodes of each subset;
[0059] Step S340, classifying the K nodes closest to the central node into a subset; for example, the node closest to the central node may be determined by calculating the Euclidean distance between the feature vector of the central node and the feature vectors of other nodes;
[0060] Step S350, updating the feature vector of the central node in the subset to the average value of the feature vectors of all nodes in the subset;
[0061] Step S360, repeatedly iterating the above steps S320-S340 of dividing the node subsets under each node set to be merged until the subsets converge, and taking the subsets as the node set to be merged;
[0062] Step S370: the central node of each subset is recorded as the representative node of this subset, retained in the graph, and all other isolated nodes in the subset are deleted.
[0063] After the above merging process, each set of nodes to be merged can retain a central node. Each isolated node category includes M sets of nodes to be merged, and M central nodes can be retained in each isolated node category. These M central nodes can be retained in the graph as merged nodes.
[0064] In an exemplary embodiment, the characteristic information includes neighboring nodes of the node; according to the characteristic information of the node, the node is formed into one or more sets of nodes to be merged, including:
[0065] According to the similarity of adjacent nodes, multiple nodes are divided into sets to obtain one or more sets of nodes to be merged.
[0066] Among them, the adjacent nodes are the nodes that have an edge connection relationship with the node. In the graph, there may be some nodes whose adjacent nodes are highly similar. In this case, you can consider merging these nodes to achieve compression processing of the graph. Specifically, you can divide multiple nodes into sets according to the similarity of adjacent nodes, for example, divide nodes with completely identical adjacent nodes into the same set, divide nodes with partially identical adjacent nodes into the same set, divide nodes with different adjacent nodes into the same set, and so on, and then merge the nodes to be merged consisting of nodes with completely identical adjacent nodes or nodes to be merged consisting of nodes with partially identical adjacent nodes.
[0067] In an exemplary embodiment, the above-mentioned dividing the multiple nodes into sets according to the similarity of adjacent nodes to obtain one or more sets of nodes to be merged may include:
[0068] Nodes with the same adjacent nodes are divided into the same set to obtain one or more sets of nodes to be merged.
[0069] In the graph, there are some nodes that have exactly the same relationship with other nodes, such as Figure 2 As shown, Node B 1 and Node B 3 Both are only with Node B 2 Associated nodes, that is, for two nodes, the adjacent nodes are the same, and their graph structures are considered to be consistent, so they can be divided into the same set of nodes to be merged and the node merging process can be performed. Since the nodes have the same graph structure, after merging the nodes, the association relationship between the original node and other nodes can be directly used as the association relationship of the new node. For example Figure 4 As shown, Figure 4 (a) is a schematic diagram of the graph before node merging. Figure 4 (b) is a schematic diagram of the graph after node merging, A 1 and A 2 The adjacent nodes of 1 and B 2 , A 1 and A 2 Have the same graph structure, so A can be merged 1 and A 2 Get an A 12 , A 12 With B 1 , B 2 The association relationship of the new node after merging is basically consistent with the association relationship of the nodes in the original node set to be merged.
[0070] Specifically, it can be achieved through the following steps: number each node, and each node number is unique; for each node, obtain its adjacent nodes, and put the numbers of the adjacent nodes into its adjacent node set; divide the nodes with exactly the same adjacent node set into the same set. At this time, nodes with the same graph structure are included in the same set, and nodes with different structures are in different sets.
[0071] In this exemplary embodiment, one or more sets of nodes to be merged can be obtained by dividing nodes with the same adjacent nodes into the same set. For example, the set obtained by dividing nodes with the same adjacent node set can be directly used as the set of nodes to be merged. Alternatively, steps similar to the above steps S320-S370 can be adopted, where the set of nodes with the same adjacent node set is first used as a cluster, and then multiple subsets are divided therein. The subsets are used as the set of nodes to be merged, and the nodes in the set of nodes to be merged are merged. Each set of nodes to be merged can correspond to one merged node, and multiple merged nodes can correspond to the node cluster with the same graph structure. After the merged nodes in each set of nodes to be merged are obtained by merging the nodes in each set of nodes to be merged, the target graph can be determined based on the merged nodes.
[0072] In this exemplary embodiment, since the nodes to be merged have the same graph structure, the set division of the surrounding nodes will not be changed after the merger, and only the adjacent node sets of the surrounding nodes need to be updated. Figure 4 As shown, A 1 and A 2 Merge into A 12 After that, just update B 1 and B 2 The adjacent node set of B does not need to be changed 1 and B 2 A new association relationship is established between the newly generated representative node and the surrounding nodes, and the adjacent nodes of the representative node are consistent with the adjacent nodes of the original node set to be merged.
[0073] In an exemplary embodiment, if Figure 5 As shown, the above-mentioned characteristic information includes a characteristic vector of a node; and the above-mentioned determination of the characteristic information of a node according to the node neighbor relationship may include the following steps:
[0074] Step S510, determining the feature vector of the node according to the neighbor relationship of the node and the node's own attributes;
[0075] According to the feature vectors of the nodes, the nodes are formed into one or more sets of nodes to be merged, including:
[0076] Step S520, forming the nodes into an initial set of one or more nodes to be merged;
[0077] Step S530, determining a similarity score of the node set to be merged according to the similarity of the feature vectors of the nodes in the node set to be merged;
[0078] Step S540: updating the set of nodes to be merged so that the similarity score of the updated set of nodes to be merged increases.
[0079] This exemplary embodiment can determine the feature vector of a node based on the node's adjacent relationship and the node's own attributes. For example, the feature vector of the node can be generated directly based on the node's text information and graph structure information, or the initial feature vector of the node can be first generated based on the node's attribute information, such as user name, identification information, date of birth, registration date and other text information, and then the initial feature vector can be updated based on the node's adjacent relationship, such as the attribute information or structure information of adjacent nodes, to obtain the node's feature vector.
[0080] In this exemplary embodiment, the attribute information of the node can be first input into the text embedding model to obtain an initialization vector. For example, the text is input into the "bge-small-zh-v1.5" text embedding model to obtain a 512-dimensional vector, i.e., the initial feature vector. The vector is then updated using the graph structure information. Specifically, for any node u in the graph, its adjacent node set is denoted as R(u). The vector v of node u can be obtained by the following formula: u Update to v′ u :
[0081]
[0082] Among them, |R(u)| represents the cardinality of the set R(u), that is, the number of adjacent nodes of node u, v i represents the vector of the neighboring nodes of node u. K is a predefined weight used to control the importance of the graph structure: the larger the value of K, the greater the influence of the surrounding nodes on the vector of u. In particular, when K takes |R(u)|, the above v′ u The calculation of degenerates to:
[0083]
[0084] That is, we directly take the mean of the vectors of u and u’s adjacent nodes. Intuitively, v′ u The calculation formula is to first find the average of the vectors of the adjacent nodes of u, and then calculate the vector v of node u. u and∑ i∈R(u) v i Perform weighted summation between them to obtain the new vector v′ of node u u Updating the node vector based on the above method can make the node vector reflect the information of the surrounding nodes to a certain extent.
[0085] Then, the nodes can be formed into an initial one or more sets of nodes to be merged, for example, nodes of the same type in the knowledge graph can be randomly divided into multiple non-overlapping sets of nodes to be merged, and further, the similarity score of the set of nodes to be merged can be determined based on the similarity of the feature vectors of the nodes in the set of nodes to be merged; finally, the set of nodes to be merged can be updated so that the similarity score of the updated set of nodes to be merged is increased to determine a stable set of nodes to be merged.
[0086] In an exemplary embodiment, the method of updating the set of nodes to be merged includes:
[0087] Move one or more nodes in one set of nodes to be merged to another set of nodes to be merged.
[0088] This exemplary embodiment can update the node set to be merged by moving one or more nodes in one node set to be merged to another node set to be merged. For example, the first node set to be merged m includes node a 1 、a 2 、a 3 , the second node set n to be merged includes node a 4 、a 5 、a 6 , the first node set to be merged m and the second node set to be merged n are the initial node sets to be merged, and the similarity of each node in the first merged node set m is calculated to obtain the similarity score s 1 , and calculate the similarity of each node in the second merged node set n to obtain the similarity score s 2 Then we can update the set of nodes to be merged. Here we can use a 1 The node moves to the second set of nodes to be merged. The first set of nodes to be merged after the move includes node a. 2 、a 3 , the second node set n to be merged includes node a 1 、a 4 、a 5 、a 6 At this time, the similarity of each node in the first merged node set m is calculated to obtain the similarity score s 1 ', and calculate the similarity of each node in the second merged node set n to obtain the similarity score s 2 ', if the similarity score s 1 'Greater than the similarity score s 1 , and the similarity score s 2 'Greater than the similarity score s 2 , then node a 1The movement is the correct movement direction. When the nodes in each node set to be merged are stable, the final node set to be merged can be determined. When moving nodes, this exemplary embodiment can use any node set to be merged as a reference, move one or more nodes therein to other node sets to be merged, and determine the optimal node set to be merged by traversing the nodes in the node set to be merged of the moving reference. When the nodes in one node set to be merged are traversed, the reference set can be updated, and the nodes in other sets can be traversed, etc. This disclosure does not make specific limitations on this.
[0089] The following is a specific example. First, nodes of the same type in the knowledge graph can be randomly divided into several non-overlapping clusters, that is, the nodes form an initial set of one or more nodes to be merged. In order to measure the similarity of the set of nodes to be merged, the calculation formula of "set similarity" can be defined as follows: C Measure the node similarity within the set of nodes to be merged:
[0090]
[0091] Where C represents the set of nodes to be merged, |C| represents the cardinality of the set, and S C Represents the similarity of the node set to be merged, and · represents the inner product of two vectors. The formula for calculating the similarity of the node set to be merged can calculate the vector inner product between any two nodes in the node set to be merged, and take the average of all inner products. The higher the similarity of the node set to be merged, the more similar the points in the node set to be merged are, and the less information is lost after the merge. For any node u in the node set to be merged, if moving node u to another node set to be merged can make the current node set to be merged and the target node set to be merged more similar, then this movement can be considered beneficial. Based on this, this exemplary embodiment can gradually optimize the similarity of the node set to be merged through the following greedy algorithm, so that the nodes in the same node set to be merged are more similar:
[0092] like Figure 6 As shown, for each node in each set of nodes to be merged, the following steps can be performed:
[0093] Step S610, traversing other to-be-merged node sets one by one, and calculating the similarity of the to-be-merged node sets when the current node is moved to other to-be-merged node sets;
[0094] Step S620 , when moving a node to a certain node set to be merged can make both the original node set to be merged and the new node set to be merged obtain a higher node set similarity to be merged, the node is moved to the new node set to be merged.
[0095] The above steps are repeatedly performed until the specified number of cycles is reached or the nodes in the node set to be merged no longer change, and an updated and stable node set to be merged can be obtained.
[0096] Finally, considering that the nodes in the same node set to be merged are relatively similar, the nodes in the node set to be merged can be merged. Specifically, the method for merging the nodes in the node set to be merged in this exemplary embodiment is as follows: First, the node set to be merged can be recorded as N, and the node vectors in the set can be recorded as: v′ 1 , v′ 2 , ..., v′ n , the merged node vector is recorded as v new ,but:
[0097]
[0098] The relationship between the new node and other nodes outside the set is the union of the relationship between the original node and other external nodes. Figure 7 As shown, Figure 7 (a) shows a schematic diagram of the graph before node merging. Figure 7 (b) shows a schematic diagram of the graph after node merging, node A 1 The adjacent node is B 1 and B 2 , node A 3 The adjacent node is B 2 and B 3 If you find A 1 and A 3 If they are similar, we can consider merging A 1 and A 3 To reduce the size of the knowledge graph and avoid losing too much information. After the nodes are merged, node A 1 and A 2 Merge to get node A 12 , due to A 1 With external Node B 1 and B 2 There is a relationship, A 2 With external Node B 2 and B 3 There is a relationship, so the merged A 12 At the same time with B 1 , B 2 and B 3 Establishing association relationships.
[0099] This exemplary embodiment can accurately and effectively compress isolated nodes, nodes with the same graph structure, and nodes with similar graph structures in the processed graph through the above method. By merging nodes with high similarity in the knowledge graph, it can retain as much effective information as possible in the knowledge graph and reduce the amount of data in the knowledge graph. By building a smaller and denser knowledge graph, it can reduce the use of computing and storage resources, speed up the processing of adding, deleting, modifying, and querying knowledge graphs, and promote the use of knowledge graph data.
[0100] The exemplary embodiments of the present disclosure also provide a graph processing device. Figure 8 The device 800 may include a to-be-processed graph acquisition module 810, for acquiring a to-be-processed graph; the to-be-processed graph includes a plurality of nodes; a characteristic information determination module 820, for determining characteristic information of a node according to a node neighbor relationship; a node set formation module 830, for forming one or more to-be-merged node sets according to the characteristic information of the node; and a target graph determination module 840, for merging at least some of the nodes in the to-be-merged node set, and determining a target graph according to the merged nodes.
[0101] In an exemplary embodiment, the graph processing device may also include: an isolated node clustering module, which is used to cluster the isolated nodes if there are isolated nodes in the graph to be processed, and form a set of nodes to be merged including the isolated nodes based on the clustering results of the isolated nodes; an isolated node is a node without adjacent nodes.
[0102] In an exemplary embodiment, the isolated node clustering module includes: a clustering result obtaining unit, which is used to cluster the isolated nodes in the graph to be processed according to the node type to obtain clustering results corresponding to multiple node types; a quantity determination unit, which is used to determine the number of node sets to be merged and the maximum number of isolated nodes in the node sets to be merged in each clustering result; and a node set formation unit, which is used to form multiple node sets to be merged in each clustering result according to the number of node sets to be merged and the maximum number of isolated nodes.
[0103] In an exemplary embodiment, the feature information includes adjacent nodes of the node; the node set forming module includes: a similarity judgment unit for dividing multiple nodes into sets according to the similarity of adjacent nodes to obtain one or more sets of nodes to be merged.
[0104] In an exemplary embodiment, the similarity determination unit is used to group nodes with the same adjacent nodes into the same set to obtain one or more sets of nodes to be merged.
[0105] In an exemplary embodiment, the feature information includes a feature vector of a node; the feature information determination module includes: a vector determination unit, used to determine the feature vector of a node based on the adjacent relationship of the node and the node's own attributes; a node set formation module, including: an initial set formation unit, used to form the nodes into an initial one or more sets of nodes to be merged; a similarity score determination unit, used to determine the similarity score of the set of nodes to be merged based on the similarity of the feature vectors of the nodes in the set of nodes to be merged; and a set update unit, used to update the set of nodes to be merged so that the similarity score of the updated set of nodes to be merged increases.
[0106] In an exemplary embodiment, the method of updating the to-be-merged node set is configured as: moving one or more nodes in one to-be-merged node set to another to-be-merged node set.
[0107] In an exemplary embodiment, the target graph determination module includes: a central node determination unit, used to determine the central node based on the nodes included in the set of nodes to be merged; a node deletion unit, used to retain the central node and delete other nodes in the set of nodes to be merged; and a graph determination unit, used to determine the target graph based on the central node.
[0108] The specific details of each module / unit in the above-mentioned device have been described in detail in the embodiments of the method part. The undisclosed details can be found in the embodiments of the method part, so they will not be repeated here.
[0109] Exemplary embodiments of the present disclosure also provide an electronic device capable of implementing the above method.
[0110] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".
[0111] Refer to the following Fig. 9 hereinafter, an electronic device 900 according to such an exemplary embodiment of the present disclosure is described. Fig. 9 The electronic device 900 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.
[0112] like Fig. 9As shown, the electronic device 900 is in the form of a general computing device. The components of the electronic device 900 may include, but are not limited to: the at least one processing unit 910, the at least one storage unit 920, a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910), and a display unit 940.
[0113] The storage unit stores program codes, which can be executed by the processing unit 910, so that the processing unit 910 executes the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 910 can execute Figure 1 The steps described above, etc.
[0114] The storage unit 920 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 921 and / or a cache memory unit 922 , and may further include a read-only memory unit (ROM) 923 .
[0115] The storage unit 920 may also include a program / utility 924 having a set (at least one) of program modules 925, such program modules 925 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0116] Bus 930 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0117] The electronic device 900 may also communicate with one or more external devices 1000 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 900, and / or communicate with any device that enables the electronic device 900 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 950. Furthermore, the electronic device 900 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 960. As shown, the network adapter 960 communicates with other modules of the electronic device 900 via a bus 930. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.
[0118] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the exemplary embodiment of the present disclosure.
[0119] The exemplary embodiments of the present disclosure also provide a computer-readable storage medium on which a program product capable of implementing the above-mentioned method of the present specification is stored. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above-mentioned "Exemplary Method" section of the present specification.
[0120] The exemplary embodiments of the present disclosure also provide a program product for implementing the above method, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.
[0121] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0122] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0123] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.
[0124] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).
[0125] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.
[0126] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.
[0127] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.
[0128] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A graph processing method, comprising: Obtaining a graph to be processed; the graph to be processed includes a plurality of nodes; Determining characteristic information of the node according to the node adjacency relationship; According to the characteristic information of the nodes, the nodes are formed into one or more sets of nodes to be merged; At least part of the nodes in the set of nodes to be merged are merged, and a target graph is determined according to the merged nodes.
2. The method according to claim 1, characterized in that The method further comprises: If there are isolated nodes in the graph to be processed, the isolated nodes are clustered, and a set of nodes to be merged including the isolated nodes is formed according to the clustering results of the isolated nodes; the isolated nodes are nodes without adjacent nodes.
3. The method according to claim 2, characterized in that If there are isolated nodes in the to-be-processed graph, clustering the isolated nodes, and forming a to-be-merged node set including the isolated nodes according to the clustering results of the isolated nodes, including: Clustering the isolated nodes in the to-be-processed graph according to the node type to obtain clustering results corresponding to multiple node types; In each of the clustering results, determining the number of node sets to be merged and the maximum number of isolated nodes in the node sets to be merged; According to the number of the to-be-merged node sets and the maximum accommodating number, a plurality of to-be-merged node sets are formed in each of the clustering results.
4. The method according to claim 1, characterized in that: The characteristic information includes adjacent nodes of the node; and forming the nodes into one or more to-be-merged node sets according to the characteristic information of the nodes includes: According to the similarity of the adjacent nodes, the multiple nodes are divided into sets to obtain one or more sets of nodes to be merged.
5. The method according to claim 4, characterized in that The step of dividing the plurality of nodes into sets according to the similarity of the adjacent nodes to obtain one or more sets of nodes to be merged includes: The nodes with the same adjacent nodes are divided into the same set to obtain one or more sets of nodes to be merged.
6. The method according to claim 1, characterized in that The characteristic information includes a characteristic vector of the node; and determining the characteristic information of the node according to the node neighbor relationship includes: Determining a feature vector of the node according to the neighbor relationship of the node and the node's own attributes; The forming of the nodes into one or more to-be-merged node sets according to the feature vectors of the nodes comprises: Forming the nodes into an initial set of one or more nodes to be merged; Determining a similarity score of the set of nodes to be merged according to the similarities of the feature vectors of the nodes in the set of nodes to be merged; The set of nodes to be merged is updated so that the similarity score of the updated set of nodes to be merged increases.
7. The method according to claim 1, characterized in that The merging process of at least part of the nodes in the set of nodes to be merged, and determining the target graph according to the merged nodes, comprises: Determine a central node according to the nodes included in the set of nodes to be merged; Keep the central node and delete other nodes in the set of nodes to be merged; According to the central node, the target graph is determined.
8. A graph processing device, characterized in that: include: A graph acquisition module to be processed, used to acquire a graph to be processed; the graph to be processed includes a plurality of nodes; A feature information determination module, used to determine the feature information of the node according to the node adjacency relationship; A node set forming module, used for forming the nodes into one or more node sets to be merged according to the characteristic information of the nodes; The target graph determination module is used to merge at least part of the nodes in the set of nodes to be merged, and determine the target graph according to the merged nodes.
9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to perform the method of any one of claims 1 to 7 by executing the executable instructions.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
