31 results about "Graph partition" patented technology

The invention discloses an archive content semantic clustering method based on a graph neural network, and the method comprises the following steps: S1, carrying out the word segmentation, denoising and vector expression of an archive text, and generating a text feature vector set; s2, constructing a semantic graph structure model according to the semantic similarity and the reference relationship between the texts; s3, generating a division result with a balanced structure on the semantic graph by adopting a graph division algorithm; s4, performing double-layer node merging on each graph division cluster, and generating a graph structure coarsening result and a mapping relation; s5, inputting the original graph and the coarsened graph into the graph neural network model, and calculating and fusing each layer of semantic representation; s6, cross-layer consistency constraint optimization node semantic representation is introduced, and a unified embedded vector set is generated; and S7, inputting the embedded vector into the clustering model, and outputting a semantic clustering category of the archive text. According to the invention, semantic recognition and automatic grouping of archive contents are realized.

This invention relates to a method for large-scale graph partitioning using streaming clustering in a GPU environment. [Solution] The present invention relates to a technology that takes into account the structural environment of the GPU, gradually compresses graph data using the CPU, then transmits it to GPU memory to perform graph partitioning, thereby making the most efficient use of GPU memory and enabling rapid graph processing. As a result, the connectivity information for all clusters can be processed effectively in parallel using the high computational power of the GPU, and calculations requiring a large amount of memory space can be processed by the CPU, thereby overcoming the limitations of GPU memory.

A computer-implemented method includes accessing, by one of more devices of a first region, an input graph comprising a plurality of nodes and a plurality of edges, each edge connecting two nodes from the plurality of nodes, where each node represents one or more users from the first region. For each node and using a secret sharing protocol: 1) one or more modularity gains for moving the node from an original community into one or more respective candidate communities is calculated and 2) an identified direction for moving the node based on the one or more modularity gains is calculated. The input graph is partitioned into a plurality of communities based on moving each node in the respective identified direction. If a determination is made that a threshold condition has been satisfied, an output graph is generated for the plurality of communities.

The invention relates to the field of numerical simulation, and provides a dynamic load balancing method and device for parallel numerical simulation and a storage medium. The method comprises the following steps: constructing a load graph based on discrete grids of a computational domain; extracting a multi-dimensional feature vector for each vertex; processing the multi-dimensional feature vector by using a pre-trained graph neural network model to predict the load weight of each vertex in parallel calculation; according to the load weight and the grid connection relation of each vertex in parallel computing, generating a dynamic partitioning scheme through a graph partitioning tool, and mapping partitions to a plurality of processors; monitoring the load unbalance degree of each processor in real time in the numerical simulation process, and triggering a re-partitioning operation when the unbalance degree exceeds a preset threshold value; and repeating the steps of load prediction, partitioning and monitoring until the simulation is finished. According to the technical scheme provided by the invention, the accuracy and adaptive capacity of load balancing in parallel numerical simulation are improved, so that the calculation efficiency is improved.

In some examples, a method of determining a shortest path between two locations includes: a first device receiving a first graph representing a graph network including a plurality of regions, each region including a plurality of vertices and a plurality of edges; identifying each of the plurality of regions and sending the first map to a mobile device; the first device partitions the first map based on the identified plurality of regions, thereby generating a set of partition maps, each region of the identified plurality of regions corresponding to a respective partition map of the set of partition maps; generating, by the first device, a second graph defining information including the set of partitioned graphs and defining connectivity between the set of partitioned graphs, and transmitting the second graph to a mobile device; traversing, by the mobile device, the second graph to determine an advanced path including a plurality of shortcuts representing a side of the plurality of sides of the first graph that connects a first vertex and a second vertex; the mobile device determines a shortest path between the first vertex and the second vertex, the shortest path being determined based on the advanced path, the shortest path comprising an edge connecting the first vertex and the second vertex in the plurality of edges of the first graph; and the mobile device outputs the shortest path.

A computer-implemented method includes: receiving data encoding an input graph comprising a group of nodes and a group of edges each connecting two nodes from the group of nodes, wherein each node represents a user on a platform with a group of users clustered into an initial set of communities; partitioning the input graph into a set of subgraphs each comprising more than one partitioned community of users; processing each subgraph using a Union-Find algorithm to generate a processed subgraph showing a revised community with a intra-community connectivity metric higher than the more than one partitioned community of users; processing each processed subgraph to split the revised community into a set of fine-grained communities; and merging the set of fine-grained communities to construct an output graph showing the plurality of users clustered into a new set of communities.

The application belongs to the technical field of artificial intelligence, graph data processing and high-performance computing, and discloses a communication-efficient memory fusion method and system for super-memory graph neural network training and a medium. A task-specific memory management scheme, zero-copy CPU-GPU transmission, a layout-aware GPU intercommunication pipeline and an out-neighbor gradient aggregation mechanism are used to significantly reduce the overall data communication overhead while ensuring training accuracy. In the fusion memory-aware graph partitioning, a same subgraph is maintained by the storage layout of multiple GPUs. For P task partitioning characteristics and T task partitioning vertex dimensions, the number of graph partitions required by the system is reduced, thereby reducing neighbor replication, and the overall CPU-GPU communication overhead of the system is successfully reduced by 50% to 64%. The back propagation of the out-neighbor gradient aggregation of the application changes the back propagation into an out-neighbor aggregation process in the preprocessing stage.

A computer-implemented method includes: accessing, by one of more devices of a first region, an input graph comprising a plurality of nodes and a plurality of edges, each edge connecting two nodes from the plurality of nodes, wherein each node represents one or more users from the first region; calculating, for each node and using a secret sharing protocol, one or more modularity gains for moving the node from an original community into one or more respective candidate communities; generating, for each node and using the secret sharing protocol, an identified direction for moving the node based on the one or more modularity gains; partitioning the input graph into a plurality of communities based on moving each node in the respective identified direction; determining a threshold condition has been satisfied; and in response to determining the threshold condition has been satisfied, generating an output graph for the plurality of communities.

The invention discloses a power flow calculation node spatial feature extraction method based on a large electric power model, and the method generates a multi-scene operation model based on historical data, meteorological conditions and load prediction, carries out the dimension reduction processing of the multi-scene operation model, retains key features, and guarantees the simplification and accuracy of the model. And secondly, converting the data after dimension reduction into a graph model, dividing the power system into a plurality of independent sub-regions by using a graph partitioning algorithm, and simplifying the Jacobian matrix and remarkably reducing the calculation complexity by eliminating interconnected nodes of which the power injection is zero. Meanwhile, the node parallel computing technology is adopted, load flow calculation of all the sub-regions is achieved, cross-region communication is reduced, and efficient power system load flow analysis is achieved.

The application discloses a kind of power grid control method and system based on graph partitioning, applied to power grid control technical field, method includes: the electrical influence degree between each node in the topology graph of target power grid obtained by analysis is acquired;Topology graph is divided based on each electrical influence degree, and first sub power grid set is obtained;The running correlation between each node is obtained by analyzing the historical operation data of target power grid obtained;First sub power grid set is adjusted based on running correlation, and second sub power grid set is obtained;Analysis path historical operation data, obtain sub power grid connection relationship;Real-time operation data of target power grid obtained is input into power grid partition model and is handled, and control strategy is generated based on processing result;Control target power grid executes control strategy.The power grid control method and system based on graph partitioning provided by the application can obtain accurate and effective control strategy, improve power grid operation stability.

The invention relates to the technical field of expressway electronic toll collection, and discloses an expressway toll automatic deduction method and system based on credit pre-authorization, and the method comprises the following steps: carrying out the sub-region division of an expressway network topological graph based on a graph partitioning algorithm, carrying out equivalence class aggregation on reachable paths starting from each entrance portal according to sub-regions through a sequence, calculating an expected cost value and a cost variance for each path cluster, and generating a compressed path cluster graph; and performing type affiliation analysis on the candidate formation cluster, and when no single type proportion in the cluster reaches an effective formation confirmation threshold and a mixed formation condition is satisfied, confirming the candidate formation cluster as a mixed formation, and generating a mixed formation record. According to the invention, independent grouping pre-authorization freezing, independent portal adjustment and independent exit settlement processes of each type enable each type to enjoy formation-level optimization and maintain independent expense accounting and timely fund release at the same time.

The invention provides a graph partitioning method and system, and the method comprises the steps: obtaining an original graph to be partitioned, enabling the original graph to comprise nodes and edges, enabling the nodes to represent an entity, enabling the edge between any two nodes to represent the correlation between any two nodes, determining a plurality of communities corresponding to the original graph, enabling one community to correspond to one community identifier, and enabling the community identifier to correspond to one community identifier; one community comprises a plurality of nodes, each community identifier is numbered, each node module is mapped to a corresponding worker according to the numbered identifier, the numbering processing is used for enabling the N workers to be balanced in the number of allocated communities, and the nodes in one community are allocated to the same worker. According to the method and the device, the structural information among the nodes in the original graph is effectively utilized, and a large amount of cross-worker communication is avoided, so that the cross-worker data exchange and communication overhead can be reduced, and the performance of a distributed system can be remarkably improved.

The embodiment of the application provides a blockchain sharding method fusing community discovery and graph division, comprising: S1. At the end of each round epoch, obtaining a transaction ledger of each shard of the blockchain in the round, and processing the transaction ledger to obtain graph structure data of the transaction; S2. Dividing the graph data structure of the transaction to obtain a community state graph; S3. Dividing the community state graph of the transaction to obtain a division result of the transaction data in the current period; S4. Evaluating and calculating the division result of the node until the optimal shard division of the blockchain is obtained; and S5. Dividing the node into different shards according to the optimal shard division, and configuring a state ledger for the divided account. The shard blockchain is optimized through the aggregation of the fusion community based on the community discovery algorithm and the node division based on the graph division, and the transaction and storage efficiency in the Internet of Things is improved.

Methods, apparatus, and software for on-the-fly graph partitioning resource utilization. The graph includes a plurality of subgraphs comprising hierarchies of subsets of ternary keys having one or more wildcards. A move operation to be executed is identified under which ternary keys and associated structures for a subset in a source subgraph are to be moved to a destination subgraph. Prior to executing the move operation, a projection is made to whether there are sufficient memory and hardware resources to execute the move operation without hitting resource capacity limits. The move operation is executed when it is projected resource capacity limits will not be hit. Under one approach, an emulation of the move operation considering resource utilization required to execute the move is performed. Under another approach, current resource utilization for the graph across memory resources and hardware resources are compiled and peak resource utilization for the move operation is projected.

The invention discloses a distributed graph neural network training method based on virtual node aggregation, and belongs to the technical field of computers, and the method comprises the following steps: dividing original graph data through distributed graph partitions, screening HALO nodes, dividing subsets according to source partitions, calculating the local connectivity of each HALO node, and establishing a mapping table; creating a virtual node for each subset, constructing a connection relationship, calculating an embedded vector in a weighted aggregation manner, and transmitting the embedded vector of the virtual node to realize O (1)-level cross-partition communication; message passing and parameter updating are completed based on the virtual nodes, and an iterative model is trained to converge; according to the method, the virtual nodes replace the HALO nodes for direct transmission, so that the communication complexity is remarkably reduced, and meanwhile, the training precision is guaranteed; the method is adaptive to various GNN models and training targets, adopts an optimized storage and calculation strategy, is high in resource utilization rate, and is suitable for a distributed training scene of large-scale graph data.

The invention discloses an intelligent network space topology construction method fusing multivariate data and an alarm tracing method in the technical field of network management and network security. The method comprises the following steps: according to asset data and physical connection relation data, sequentially carrying out asset grouping, topology framework firewall layout, topology framework switch layout, topology framework asset offset, layout of all other assets and topology layout optimization based on graph segmentation through a physical mapping service to obtain JSON data capable of completely describing a topology structure; according to the JSON data, a front-end rendering engine based on WebGL is utilized to render a visual topological graph in a browser, events are added in the visual topological graph through event driving so as to support a user to drag and edit topology, and an interactive visual topological graph is obtained. According to the method, remarkable technical progress is generated in the authenticity, the automation degree, the field applicability and the safety analysis value of the topology.

A method and system for accelerating deep neural network hybrid parallel inference in heterogeneous trusted execution environments are presented. This method first acquires the hardware parameters of the heterogeneous cluster and establishes a segmented model of a secure memory paging penalty. Second, it performs active parallel partitioning based on a directed acyclic graph, identifying Fork and Join nodes and transforming them into independent scheduling boundaries to generate macro-level graph partitions. Next, for overloaded operators, a cost-aware operator-level parallel algorithm is employed to construct a three-dimensional cost function integrating computation, paging, and communication, determining the optimal parallelism and dividing the data into multiple micro-operator slices. Finally, a two-level hybrid parallel cooperative scheduling is executed, constructing a unified heterogeneous task dependency graph through virtual synchronization nodes and utilizing priority assignment of the global critical path and cooperative group constraints to complete physical node mapping. This invention avoids loss of topological concurrency, overcomes the bottleneck of single-point physical memory limits, eliminates deadlock in heterogeneous scheduling, and significantly reduces end-to-end inference latency.

The invention relates to a large-scale GNN distributed training optimization method based on AVX. The method comprises the following steps: acquiring a graph data set comprising nodes and edges; the graph data set is a training data set used for training a graph neural network GNN; constructing a target adjacent matrix according to the graph data set; on the basis of the target adjacency matrix, using an AVX instruction set to load neighbor node information of the multiple nodes in parallel so as to carry out graph partitioning in parallel to obtain multiple sub-graph data; carrying out sampling processing on the sub-graph data in parallel by using an AVX instruction set to obtain sampling data; and in each training node of distributed training, training the GNN based on the sampling data. Through the method, the calculation efficiency of GNN distributed training can be improved.

The application discloses an event template induction method and system based on a large-scale language model. The method mainly comprises three modules: context-based text conceptualization, confidence-based event template structuring and graph-based event template integration. Specifically, the context-based text conceptualization fully utilizes the generation ability and analogy ability of a large-scale generative pre-training language model through example learning, and converts diversified event natural language expressions into unified conceptualized event template language; the confidence-based event template structuring filters the conceptualized event categories and event argument roles through saliency, reliability and consistency, and thus structures the event template language; and the graph-based event template integration integrates the scattered event templates of the same event through a graph partition clustering algorithm. The application can effectively discover high-quality and high-coverage event templates in an open scene.

The invention relates to an electromagnetic transient modeling method, device and equipment for a power system based on typical component multiplexing. The method comprises the following steps: acquiring a data file, a component model library and computing resource data; analyzing the data file to obtain analyzed data; obtaining a long-distance tie line from the network topology; taking the long-distance tie lines as candidate decoupling points, and carrying out network division on the network topology by adopting an objective function of a graph segmentation algorithm to obtain a plurality of sub-networks with low coupling degrees; each called basic typical element model is screened out from the element model library and modified according to corresponding element parameters, and electromagnetic transient simulation elements are obtained; and performing model construction according to the load data, the generator data, all the sub-networks and the electromagnetic transient simulation elements of the power devices to obtain an electromagnetic transient model of the power system. According to the method, the electromagnetic transient simulation elements are constructed through the element model library, and the network topology of the power system is subjected to sub-network division by adopting the graph segmentation algorithm, so that the modeling efficiency is improved.

The invention discloses a large-scale social network graph division method based on a DDQN algorithm. The method comprises the following steps: randomly dividing a social network graph to obtain initial partitions; constructing a graph feature matrix according to the social network graph attributes and the partition information; inputting the graph feature matrix into a Q network for prediction to obtain a division action and an expected Q value; executing a prediction action to obtain a new partition and an actual reward; calculating mean square error loss according to the actual reward and the evaluated Q value, and updating network parameters by using gradient descent; after the division action is executed, the division convergence degree is judged; and if the division converges, stopping division and outputting the partition, otherwise, using the new partition to replace the initial partition to continue. Attribute information and structural features of the social network are fully utilized, and the minimum number of cutting edges is achieved while social network graph partition load balancing is achieved.

The application discloses a kind of based on graph node feature pre-computation's super large scale e-commerce network partition optimization method and device, in pre-division stage, using the graph information of initial time scale relatively smaller extraction homogeneous enumeration feature and homogeneous scalar feature of this e-commerce network.In each distributed server partition, different number of central nodes are maintained, the similarity of two types of homogeneous features of the newly added node and the central nodes in each partition of distributed server, node degree similarity and the load of each partition of distributed server are calculated to obtain the partition quality score, and the node is divided into the distributed server partition with the highest score, so as to improve the quality of each partition of distributed server.In dynamic updating stage, the application uses the designed graph partition quality scoring function to divide the input stream information and update the central nodes of each partition of distributed server in real time, so as to complete the maintenance of super large scale dynamic e-commerce network, and improve the computing efficiency of distributed graph database storage engine.

The invention discloses a big data access control method based on a graph neural network, and the method comprises the following steps: S1, obtaining information elements in a big data environment, and carrying out the preprocessing; s2, constructing a heterogeneous graph structure, defining nodes and edges, and endowing the edges with additional attribute tags; s3, performing graph division on the heterogeneous graph structure, constructing a relation triple, and forming a multi-hop access path in combination with an attribute tag; s4, performing graph feature aggregation on the multi-hop access path, and fusing relation triple adjacent node features and attribute tags; s5, performing multi-layer graph convolution and normalization operation on a fusion result, extracting cross-sub-graph global features, and generating a confidence score table; s6, executing real-time matching and authorization decision on the access request, and outputting an access control result; and S7, optimizing an edge weight based on the access control result and the access behavior feedback. According to the invention, accurate authorization judgment and dynamic permission adjustment of the access request in the big data environment are realized.

The application discloses a positive sample selection enhancement method based on structural fingerprints and dynamic time warping, first constructs a multi-hop degree sequence as a structural fingerprint at a node level, calculates the structural distance of any two nodes by using hierarchical DTW, and aggregates the structural distance into a structural similarity matrix; then performs row Top-k sparsification on the similarity matrix to determine a node-level positive sample set with consistent roles; maps the node-level positive sample to a cluster level based on a hard assignment matrix obtained by graph partitioning, and performs contrastive learning on cluster features without message passing. Compared with the existing positive sample construction method which depends on random augmentation, the application can significantly reduce "false positive / false negative" noise, and still maintain stable and consistent semantic alignment on graphs with low homomorphism and many noise edges; meanwhile, with the help of a cluster-level training path, the training complexity is reduced from the edge scale to the node and cluster scale, realizing an efficient, reproducible and deterministic graph contrastive learning process.

The application discloses a neural network topology structure mapping method for a many-core architecture, uses a four-step algorithm framework based on scale reduction, preliminary segmentation, scale expansion and mapping scheme construction, saves the topology structure to a file system, and applies a graph partition algorithm and a force guiding algorithm, so that memory occupation during compilation of the topology structure of a large-scale neural network is greatly reduced, and the range of the neural network that can be deployed to a neural computing chip is expanded. Meanwhile, by using a heuristic algorithm specific to a mapping problem, the number of iterations and running time are greatly reduced, the compilation efficiency is improved, and the quality of the compilation result is ensured.

The application discloses a graph structure perception random walk cache management method, a controller and a system, and belongs to the field of cache optimization, and comprises the following steps: obtaining that the average vertex degree of current to-be-processed graph data is divided into a plurality of subintervals, maintaining a plurality of corresponding barrels, and dividing each vertex into the corresponding barrel; initializing a vertex graph reordering processing queue, always selecting an unprocessed vertex from the barrel with the largest average vertex degree to add the queue as a starting vertex for current processing and traversing all adjacent vertices of the starting vertex, and adding unprocessed adjacent vertices to a candidate processing set; at the same time, low-degree vertices are uniformly post-processed to the end of the ordering sequence to form a cold data partition; after graph reordering, hot and cold data are divided, pre-sampling is performed on the hot data partition, the pre-sampling result is cached, and direct sampling is performed on the cold data partition. The application can reduce the total number of I / O operations of loading graph partitions in large-scale graph data random walk under the condition that the memory capacity is limited, and effectively reduce the I / O overhead.

The invention discloses a distributed co-purchase recommendation method based on graph pattern matching, and aims to realize rapid combination recommendation. According to the scheme, a distributed co-purchase recommendation system based on graph pattern matching is constructed on a distributed cluster with NUMA, and each NUMA node comprises a graph partition module, a task generation module, a task buffer module, a task scheduling module, a task execution module, a vertex request module, a vertex response module and a vertex buffer module; the method comprises the following steps: constructing a co-purchase relationship data graph and a co-purchase relationship mode graph, enabling all modules of a co-purchase recommendation system to work in parallel, determining a vertex matching sequence according to a connection relationship of the co-purchase relationship mode graph, matching vertexes in the co-purchase relationship mode graph to vertexes of the co-purchase relationship data graph, and obtaining sub-graphs isomorphic with the co-purchase relationship mode graph; and obtaining an isomorphic sub-graph with the maximum frequency according to the common purchase frequency model, wherein a set of products represented by vertexes of each isomorphic sub-graph is a combined recommendation result. The recommendation efficiency and speed can be improved.

The application provides an edge partition method for graph partition, which comprises the following steps: firstly, loading a graph to be partitioned and calculating a partition boundary value; then, unfolding the graph to obtain a conversion graph, creating and searching a set and a priority queue for the conversion graph; then, folding edges according to a greedy strategy, and partitioning subgraphs meeting the partition boundary value; finally, when the priority queue is empty, combining the remaining subgraphs to form a scale and partitioning. The application can achieve the advantages of simple partition and load balancing by actually using point partition and essentially using the edge partition method to partition the graph. In addition, the application folds edges under the greedy strategy, so that the edges adjacent in space tend to be folded in the same subgraph, thereby reducing the number of node replication times as much as possible and reducing communication overhead.

A computer-implemented method includes: receiving data encoding an input graph comprising a group of nodes and a group of edges each connecting two nodes from the group of nodes, wherein each node represents a user on a platform with a group of users clustered into an initial set of communities; partitioning the input graph into a set of subgraphs each comprising more than one partitioned community of users; processing each subgraph using a Union-Find algorithm to generate a processed subgraph showing a revised community with a intra-community connectivity metric higher than the more than one partitioned community of users; processing each processed subgraph to split the revised community into a set of fine-grained communities; and merging the set of fine-grained communities to construct an output graph showing the plurality of users clustered into a new set of communities.