Hypergraph segmentation method based on self-supervised learning

Abstract

The invention belongs to the technical field of hypergraph segmentation, and particularly relates to a hypergraph segmentation method based on self-supervised learning, so as to provide a solution of a hypergraph segmentation problem with fewer cut edges and better balance in industrial application scenes such as integrated circuit design. According to the method, only the hypergraph structure and the number of the sub-blocks needing to be segmented are needed to serve as input, the hypergraph segmentation problem is trained and automatically solved through the end-to-end self-supervised neural network model, and due to the fact that operation included in the adopted loss function is matrix operation such as PyTorch supported by a current deep learning framework, the method has high operation efficiency, and the method is easy to implement. A traditional heuristic graph segmentation algorithm represented by multi-layer hypergraph segmentation needs to use heuristic information, so that local optimum is easy to fall into, and compared with the traditional heuristic graph segmentation algorithm, a result closer to a globally optimal solution can be obtained under some conditions; compared with a traditional method, the method has a certain advantage in the aspect of reducing the number of cut edges while ensuring the balance of each block obtained by hypergraph segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of hypergraph segmentation, and in particular to a hypergraph segmentation method based on self-supervised learning. Background Art

[0002] The hypergraph partition problem is one of the basic problems of combinatorial optimization and is currently widely used in integrated circuit design, image segmentation and other fields. The optimization goal of the hypergraph partition problem is to Node set Split into disjoint subsets , the sizes of these subsets are as similar as possible, and the number of cut edges between these subsets is as small as possible. At present, the efficient algorithm for the hypergraph segmentation problem is mainly the multi-layer segmentation algorithm, which is a traditional heuristic algorithm. This type of algorithm is a three-stage algorithm. The first stage is the coarsening clustering stage, which reduces the complexity of the hypergraph segmentation problem by merging nodes with similar distances in the hypergraph according to certain rules; the second stage is the initial segmentation stage, the purpose of which is to find the initial solution of the hypergraph segmentation on the coarse-grained hypergraph after node aggregation; the third stage is the refinement stage, the purpose of which is to improve the initial hypergraph segmentation results from the coarse-grained to the fine-grained level, so that the number of cut edges corresponding to the final solution is minimized. In this patent, the solution to the hypergraph segmentation problem adopts a neural network architecture based on self-supervised learning, which is a completely different technical route from the traditional heuristic algorithm represented by the multi-layer segmentation algorithm. The implementation scheme similar to the present invention is mainly aimed at the segmentation problem of ordinary graphs, and there are certain differences in the problem definition between ordinary graph segmentation and hypergraph segmentation. At present, no similar method for hypergraph segmentation has been found.

[0003] In the hypergraph segmentation method represented by the multi-level segmentation algorithm, since its basic idea is to reduce the complexity of the hypergraph segmentation problem through coarsening in the first stage, a suboptimal solution is often obtained, and one of the disadvantages of the traditional heuristic algorithm is that it is easy to fall into the local optimum in the process of using heuristic information. Using an end-to-end deep learning model in ordinary graph segmentation problems can surpass traditional heuristic algorithms in some cases. However, relevant research has not yet gone deep into the hypergraph segmentation problem. Based on such a problem. The present invention proposes to design a self-supervised learning model using a hypergraph neural network, and proposes a multi-objective loss function for the hypergraph segmentation problem that can be implemented based on the PyTorch interface and can be quickly trained, which supports the training of the self-supervised learning model. The present invention can only input the topological structure information of the hypergraph and automatically solve the hypergraph segmentation problem through an end-to-end neural network model. Summary of the invention

[0004] 1. Technical issues to be resolved In view of the deficiencies of the prior art, the present invention provides a hypergraph segmentation method based on self-supervised learning, which solves one of the drawbacks of traditional heuristic algorithms, that is, the problem of being easily trapped in local optimality in the process of using heuristic information.

[0005] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions: A hypergraph segmentation method based on self-supervised learning to provide a solution to the hypergraph segmentation problem with fewer cut edges and better balance in industrial application scenarios such as integrated circuit design; There are three known conditions for the implementation of this method: hypergraph structure , a predefined set of nodes Split into disjoint subsets Number of , Hypergraph Neural Network HGNN; The hypergraph segmentation method is divided into two modules: hypergraph embedding module and hypergraph segmentation module; In the hypergraph embedding module, the hypergraph neural network is trained to obtain the The probability distribution matrix ;In the hypergraph segmentation module, enter the probability , through the hypergraph segmentation model loss function training, let the probability distribution matrix Converges and approaches the optimal solution to the hypergraph partitioning problem.

[0006] Furthermore, the known conditions (i.e. input variables) are: (1) A hypergraph :Specifically, a hypergraph is defined as a series of nodes and a series of hyperedges , where each hyperedge yes A subset of; A hypergraph can be expressed as an index matrix , among which hour, ,otherwise .

[0007] (2) Predefined node sets Split into disjoint subsets Number of .

[0008] (3) Hypergraph neural network: The hypergraph embedding module uses the graph structure to learn the node embedding matrix and generates the node belonging to each node through softmax. The probability distribution matrix , in the model No. A row can represent each node Belongs to block The probability of Specifically, the hypergraph embedding module uses a hypergraph neural network structure to learn the embedding matrix of the hypergraph; Given a hypergraph and the incidence matrix , randomly generate node feature matrix And input into the hypergraph neural network to obtain the node embedding matrix; Hypergraph Neural Network The construction of the convolutional layer is as follows:

[0009] Here, By The embedding matrix generated by the convolutional layer, Is a nonlinear activation function, such as ReLU;

[0010] For a vertex , whose degree is defined as . Similarly, the edge The degree is defined as . and Diagonal matrices used to represent vertex degrees and edge degrees respectively; can be considered as the weight of a hyperedge; parameter is learned during training.

[0011] Furthermore, the variables to be determined are: the probability matrix obtained after the model is trained and converged ; After the model is trained and converged It means Each node belongs to The probability of , which is the solution to the hypergraph partitioning problem; The definition of the hypergraph segmentation problem is: given a hypergraph , the hypergraph partitioning problem aims to separate the node set Split into disjoint subsets , the number of nodes in these subsets is as similar as possible, while minimizing the number of cut edges; The concept of cut edge refers to a hyperedge that spans multiple partitioned subgraphs; Step 1: Given a hypergraph and the incidence matrix , first randomly generate the node feature matrix And input to the hypergraph neural network to output the node embedding matrix ,Right now Each node belongs to The probability distribution matrix of Will After being input into the above hypergraph neural network, the output matrix is ​​transformed into Transform into a probability distribution matrix and generate a probability matrix ; Step 2: Input the hypergraph segmentation module, train it through the loss function defined by the hypergraph segmentation module and use convergence; because , so in fact The elements in the array have values ​​between 0 and 1, that is, ; Define the following loss function:

[0012] in,

[0013]

[0014]

[0015]

[0016] in and is a hyperparameter, is the number of predefined sub-blocks into which the hypergraph is partitioned, is the incidence matrix of the hypergraph, is the coefficient vector used for normalization; The matrix after convergence Each row of is approximately a one-hot code, indicating which sub-block of the hypergraph partition the corresponding node belongs to.

[0017] (III) Beneficial effects Compared with the prior art, the present invention provides a hypergraph segmentation method based on self-supervised learning, which has the following beneficial effects: The present invention only requires the hypergraph structure and the number of sub-blocks to be divided as input, and automatically solves the hypergraph segmentation problem through end-to-end self-supervised neural network model training. Since the operations included in the adopted loss function are all matrix operations supported by current deep learning frameworks such as PyTorch, it has higher computational efficiency. In addition, traditional heuristic graph segmentation algorithms represented by multi-layer hypergraph segmentation need to use heuristic information, so they are prone to fall into local optimal solutions. In contrast, this method may obtain results that are closer to the global optimal solution in some cases. Compared with traditional methods, it can ensure the balance of each block obtained by hypergraph segmentation while reducing the number of cut edges. It has certain advantages. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a diagram of the neural network model architecture of the present invention; Figure 2 This is an example diagram of formulas (2)-(6) of the present invention. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] Example

[0021] like Figure 1 and Figure 2 As shown, an embodiment of the present invention proposes: a hypergraph segmentation method based on self-supervised learning, so as to provide a solution to the hypergraph segmentation problem with fewer cut edges and better balance in industrial application scenarios such as integrated circuit design.

[0022] There are three known conditions for the implementation of this method: hypergraph structure , a predefined set of nodes Split into disjoint subsets Number of , Hypergraph Neural Network HGNN.

[0023] The hypergraph segmentation method in the present invention is divided into two modules: a hypergraph embedding module and a hypergraph segmentation module. In the hypergraph embedding module, the hypergraph neural network is trained to obtain the representation of each node. The probability distribution matrix ;In the hypergraph segmentation module, enter the probability , through the hypergraph segmentation model loss function training proposed in this patent, let the probability distribution matrix Converges and approaches the optimal solution to the hypergraph partitioning problem.

[0024] The known conditions (i.e. input variables) are: (1) A hypergraph :Specifically, a hypergraph is defined as a series of nodes and a series of hyperedges , where each hyperedge yes A subset of; A hypergraph can be expressed as an index matrix , among which hour, ,otherwise .

[0025] (2) Predefined node sets Split into disjoint subsets Number of .

[0026] (3) Hypergraph neural network: The hypergraph embedding module uses the graph structure to learn the node embedding matrix and generates the node belonging to each node through softmax. The probability distribution matrix , in the model No. A row can represent each node Belongs to block The probability of Specifically, the hypergraph embedding module uses the hypergraph neural network structure to learn the embedding matrix of the hypergraph. and the incidence matrix , randomly generate node feature matrix And input into the hypergraph neural network to obtain the node embedding matrix; Hypergraph Neural Network The construction of the convolutional layer is as follows:

[0027] Here, By The embedding matrix generated by the convolutional layer, Is a nonlinear activation function, such as ReLU; For a vertex , whose degree is defined as ; Similarly, the edge The degree is defined as . and Diagonal matrices used to represent vertex degrees and edge degrees respectively; can be considered as the weight of a hyperedge; parameter is learned during training.

[0028] For example, to connect the hypergraph embedding module and the hypergraph segmentation module to form a complete learning model, a two-layer hypergraph neural network can use the learnable parameter matrix ,in , output an embedding matrix ; Then, apply the softmax function to generate the probability matrix ; Finally, the hypergraph segmentation module receives , and trained in an unsupervised manner to minimize the loss function.

[0029] The variables to be sought are: After the model is trained and converged, the probability matrix is ​​obtained After the model is trained and converged It means Each node belongs to The probability of , which is the solution to the hypergraph partitioning problem; The definition of the hypergraph segmentation problem is: given a hypergraph , the hypergraph partitioning problem aims to separate the node set Split into disjoint subsets , the number of nodes in these subsets is as similar as possible, while minimizing the number of cut edges. The concept of cut edges refers to hyperedges that span multiple partitioned subgraphs; Step 1: Given a hypergraph and the incidence matrix , first randomly generate the node feature matrix And input to the hypergraph neural network to output the node embedding matrix ,Right now Each node belongs to The probability distribution matrix of Will After being input into the above hypergraph neural network, the output matrix is ​​transformed into Transform into a probability distribution matrix and generate a probability matrix ; Step 2: Input the hypergraph segmentation module, train it through the loss function defined by the hypergraph segmentation module and use convergence; because , so in fact The elements in the array have values ​​between 0 and 1, that is, ; Define the following loss function:

[0030] in,

[0031]

[0032]

[0033]

[0034] in and is a hyperparameter, is the number of predefined sub-blocks into which the hypergraph is partitioned, is the incidence matrix of the hypergraph, is the coefficient vector used for normalization; The matrix after convergence Each row of is approximately a one-hot code, indicating which sub-block of the hypergraph partition the corresponding node belongs to.

[0035] The following is an implementation of this method: The effectiveness of this method compared with the traditional multi-layer hypergraph segmentation method is verified on the common hypergraph dataset Cora; Cora is a hypergraph dataset constructed through a co-citation network, which contains 1330 nodes and 1413 hyperedges. This method is implemented using the PyTorch 1.13 interface. The hardware equipment mainly includes two NVIDIA Geforce RTX 3090 graphics cards with 24G video memory. The experimental results are compared with the currently popular Mt-KaHyPar multi-layer hypergraph segmentation framework. The HGNN+ model is used to construct the hypergraph embedding module in the experiment. The verification indicators are the number of cut edges and the Balance indicator. For a set of partitions of the hypergraph ..., , let the set , then the definition of the Balance index is as follows:

[0036] in It represents a collection The mean of the elements in the ,balance index can measure whether the number of nodes in each subgraph is close,under the premise that the edges between the subgraphs divided by the hypergraph are as few as possible.,The Balance index actually uses the mean square error; The experimental results are shown in the following table: Table 1 Number of sub-blocks 2 3 4 5 6 Kj Number of cutting edges 123 172 201 237 270 Kj Balance 0 1.70 0 2.60 2.21 Methods Number of cutting edges 122 169 203 237 261 Methods Balance 0 0 0 10.53 0 From the experimental results in Table 1, it can be seen that this method can achieve better hypergraph partitioning results in some cases, while ensuring balance and obtaining fewer cut edges.

[0037] The following is an example of the steps of this method, as follows: Step 1: To connect the hypergraph embedding module and the hypergraph segmentation module to form a complete machine learning model, a two-layer hypergraph neural network can use a learnable parameter matrix ,in , output an embedding matrix ; Then, the softmax function is applied to generate the probability matrix ; Finally, the hypergraph segmentation module receives , and trained in an unsupervised manner to minimize the loss function. The machine learning model learns , its A row can represent each node Belongs to the partition probability.

[0038] Step 2: Formulas (2)-(6) can be implemented using PyTorch's matrix operation interface, such as Figure 2 As shown in the example given, for a hypergraph , , ,and , ; Probability distribution matrix and the incidence matrix ,like Figure 2 As shown, use express ,but express and Probability in different partitions; Afterwards, and Multiply them together to get the matrix ,in express pass The point in is the probability of a cut edge; For example, in the figure yes pass , , Make the hyperedge The sum of the probabilities of being a cut edge, however, in reality Can't let Becomes a cutting edge, because Does not belong to ; Therefore, we use the matrix and matrix The Hadamard product of Each node in Become a probability distribution of a cut edge. Next, use To express ; Coefficient Matrix Used to normalize the sum of the probabilities that each hyperedge is a cut edge; Then through the Subtract all 1s from the left to get the matrix ,in express make is the probability of a non-cut edge; Finally, by Perform a column-wise Hadamard product and weight the matrix elements to get the final value of the loss function.

[0039] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A hypergraph segmentation method based on self-supervised learning, characterized in that: The steps include: Step 1: Given a hypergraph and the incidence matrix , first randomly generate the node feature matrix And input to the hypergraph neural network to output the node embedding matrix ,Right now Each node belongs to The probability distribution matrix of Step 2: Input the hypergraph segmentation module, train it through the loss function defined by the hypergraph segmentation module and use convergence.

2. The hypergraph segmentation method based on self-supervised learning according to claim 1, characterized in that: The hypergraph Defined as a series of nodes and a series of hyperedges , where each hyperedge yes A subset of; A hypergraph can be expressed as an index matrix , among which hour, ,otherwise ; Among them, the predefined node set Split into disjoint subsets Number of ; The hypergraph embedding module uses the graph structure to learn the node embedding matrix and generates the node belonging to each node through softmax. The probability distribution matrix , in the model No. A row can represent each node Belongs to block probability.

3. The hypergraph segmentation method based on self-supervised learning according to claim 2, characterized in that: The hypergraph embedding module adopts a hypergraph neural network structure to learn the embedding matrix of the hypergraph; Given a hypergraph and the incidence matrix , randomly generate node feature matrix And input into the hypergraph neural network to obtain the node embedding matrix; Hypergraph Neural Network The construction of the convolutional layer is as follows: Here, By The embedding matrix generated by the convolutional layer, Is a nonlinear activation function, such as ReLU; For a vertex , whose degree is defined as ; Similarly, the edge The degree is defined as ; and Diagonal matrices used to represent vertex degrees and edge degrees respectively; can be considered as the weight of a hyperedge; parameter is learned during training.

4. The hypergraph segmentation method based on self-supervised learning according to claim 3, characterized in that: After the model is trained and converged, the probability matrix ; After the model is trained and converged It means Each node belongs to The probability of , which is the solution to the hypergraph partitioning problem; The definition of the hypergraph segmentation problem is: given a hypergraph , the hypergraph partitioning problem aims to separate the node set Split into disjoint subsets , the number of nodes in these subsets is as similar as possible, while minimizing the number of cut edges; The concept of cut edge refers to a hyperedge that spans multiple partitioned subgraphs.

5. The hypergraph segmentation method based on self-supervised learning according to claim 3, characterized in that: Will After being input into the above hypergraph neural network, the output matrix is ​​transformed into Transform into a probability distribution matrix and generate a probability matrix .

6. The hypergraph segmentation method based on self-supervised learning according to claim 1, characterized in that: In step 2, due to , so in fact The elements in the array have values ​​between 0 and 1, that is, ; Define the following loss function: in, in, and is a hyperparameter, is the number of predefined sub-blocks into which the hypergraph is partitioned, is the incidence matrix of the hypergraph, is the coefficient vector used for normalization; The matrix after convergence Each row of is approximately a one-hot code, indicating which sub-block of the hypergraph partition the corresponding node belongs to.

Images