Graph neural network anti-overfitting method and device
By determining the actual probability of node discarding and the sub-graph to be discarded in the graph neural network, the problem of overfitting the graph neural network is solved, and the anti-overfit ability of the model is improved.
Patent Information
- Application Number
- CN202411994127.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-31
- Publication Date
- 2025-05-09
AI Technical Summary
Graph neural networks often overfit during training, resulting in the model performing well on the training set but poorly on the test set.
By obtaining the target graph, determining the node characteristics and attention graph, calculating the average node drop probability, and determining the node's actual drop probability based on the attention graph and drop probability, thereby determining the subgraph to be discarded, avoiding overfitting of the graph neural network.
This method can automatically focus on the key learning areas of graph neural networks, avoid overfitting, and improve the anti-overfit ability of graph convolutional networks.
Smart Images
Figure CN119962569A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for resisting overfitting of a graph neural network. Background Art
[0002] In recent years, graph neural networks have been widely used. However, graph neural networks often suffer from overfitting during training, which hinders further improvement of their performance. In the field of artificial intelligence, overfitting refers to the large gap between training error and test error, that is, the model performs well on the training set but performs poorly on the test set.
[0003] In the prior art, some nodes in the graph or some features of the nodes can be discarded to prevent the graph neural network from over-learning and over-fitting the features of the training set.
[0004] However, due to the correlation between the activation units in the graph problem, even if a node is discarded, the graph neural network can still obtain the information contained in it from the neighboring nodes of the node. Therefore, the ordinary random discard method is not ideal in resisting overfitting in the graph neural network. Summary of the invention
[0005] The present invention provides a graph neural network anti-overfitting discarding method and device, which are used to solve the problem of over-learning and over-fitting of training set features by graph neural networks in the prior art.
[0006] The present invention provides a graph neural network anti-overfitting method, comprising: obtaining a target graph; determining node features according to the target graph, and determining an attention map according to the node features; the attention map includes the attention degree of each node; determining an average node discard probability according to the target graph, and determining an actual node discard probability according to the attention map and the average node discard probability; determining a subgraph to be discarded from the target graph according to the actual node discard probability; wherein the subgraph to be discarded includes a core node, and a plurality of adjacent nodes connected to the core node.
[0007] According to a graph neural network anti-overfitting method provided by the present invention, determining the attention map based on the node features includes: taking the absolute value of the node feature of each node and then determining the corresponding average value to obtain the attention map.
[0008] According to a graph neural network anti-overfitting method provided by the present invention, the average node discard probability is determined according to the target graph, including: calculating the average number of edges of the nodes in the target graph; calculating the average number of adjacent nodes at different orders according to the average number of edges; determining the sum of the number of nodes of the subgraph according to the average number of adjacent nodes; and determining the average node discard probability according to a preset discard probability and the sum of the number of nodes.
[0009] According to a graph neural network anti-overfitting method provided by the present invention, the average number of adjacent nodes at different orders is calculated according to the average number of edges, including: according to the formula calculate The average number of adjacent nodes of a node ;in, represents the average number of edges.
[0010] The present invention also provides a graph neural network anti-overfitting device, comprising the following modules: an acquisition module and a processing module; the acquisition module is used to acquire a target graph; the processing module is used to determine node features based on the target graph, and determine an attention map based on the node features; the attention map includes the attention degree of each node; the average node discard probability is determined based on the target graph, and the actual node discard probability is determined based on the attention map and the average node discard probability; the subgraph to be discarded is determined from the target graph according to the actual node discard probability; wherein the subgraph to be discarded includes a core node, and multiple adjacent nodes connected to the core node.
[0011] According to a graph neural network anti-overfitting device provided by the present invention, the processing module is used to take the absolute value of the node feature of each node and then determine the corresponding average value to obtain the attention map.
[0012] According to a graph neural network anti-overfitting device provided by the present invention, the processing module is used to calculate the average number of edges of nodes in the target graph; calculate the average number of adjacent nodes at different orders based on the average number of edges; determine the sum of the number of nodes of the subgraph based on the average number of adjacent nodes; and determine the average discard probability of the node based on a preset discard probability and the sum of the number of nodes.
[0013] According to a graph neural network anti-overfitting device provided by the present invention, the processing module is used to calculate The average number of adjacent nodes of a node ;in, represents the average number of edges.
[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements any of the graph neural network anti-overfitting methods described above.
[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the graph neural network anti-overfitting methods described above.
[0016] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the graph neural network anti-overfitting methods described above.
[0017] The graph neural network anti-overfitting method and device provided by the present invention, on the one hand, because the actual node discarding probability includes the addition of the attention map, it can automatically focus on the key learning area of the graph neural network, thereby discarding the key learning part of the graph neural network to avoid overfitting; on the other hand, because the subgraph includes a core node and multiple adjacent nodes connected to the core node, it can avoid the graph neural network from recovering the semantic and structural feature information contained in the neighboring nodes of the discarded nodes, thereby improving the anti-overfitting ability of the graph convolutional network. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0019] Figure 1 It is a schematic diagram of the flow of the graph neural network anti-overfitting method provided by the present invention; Figure 2 It is a structural schematic diagram of a graph neural network anti-overfitting device provided by the present invention; Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0020] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0021] It should be noted that, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.
[0022] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0023] In order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and order of execution.
[0024] The embodiments of the present application describe some exemplary embodiments for the purpose of explanation. It should be understood that the present application can be implemented in other ways that are not specifically shown in the drawings.
[0025] Graph neural network is a convolutional neural network that directly acts on graphs and uses their structural information. Its basic principle is to use the information of other graph nodes to deduce the information of the graph node. It can automatically learn the node features in the graph and the connections between nodes. Many problems in real life can be naturally represented in the form of graphs.
[0026] The basic form of graph neural network can be expressed as follows: ; in, is initialized to , , Usually set to 0.001 to avoid the denominator being zero. It is a division of the neighborhood of graph nodes.
[0027] Graph neural networks often experience overfitting during training. The reason for this is that the model overfits the features of the training set, fails to understand the rules behind the data, and has poor generalization ability.
[0028] In order to solve the above problems, Figure 1 As shown, the embodiment of the present application provides a graph neural network anti-overfitting method, which can be applied to a graph neural network anti-overfitting device. The graph neural network anti-overfitting method may include S101-S104: S101. A graph neural network anti-overfitting device obtains a target graph.
[0029] The target graph G can be expressed as , where V represents the node set and E represents the edge set. If there is an edge connecting node i and node j in node set V, then .
[0030] The adjacency matrix of the target graph G can be represented as A if ,but ,on the contrary, .
[0031] The degree matrix of the target graph G can be expressed as D, where D is a diagonal matrix. Represents the number of edges connected to node i.
[0032] S102. The graph neural network anti-overfitting device determines node features according to the target graph, and determines an attention graph according to the node features.
[0033] The attention map includes the degree of attention received by each node.
[0034] Optionally, the graph neural network anti-overfitting device determines an attention map based on the node features, including: taking the absolute value of the node features of each node and determining the corresponding average value to obtain the attention map.
[0035] Specifically, the graph neural network anti-overfitting device can obtain the node features of the target graph G through the image encoder. Then, the node features of each node are taken at an absolute value and the corresponding average value is determined to obtain the attention graph α. According to the attention graph α, the node The popularity of .
[0036] S103. The graph neural network anti-overfitting device determines the average node discard probability according to the target graph, and determines the actual node discard probability according to the attention graph and the average node discard probability.
[0037] Optionally, the graph neural network anti-overfitting device determines the attention graph and the average node discard probability based on the node features, including: calculating the average number of edges of the nodes in the target graph; calculating the average number of adjacent nodes at different orders based on the average number of edges; determining the sum of the number of nodes in the subgraph based on the average number of adjacent nodes; and determining the average node discard probability based on a preset discard probability and the sum of the number of nodes.
[0038] Optionally, the graph neural network anti-overfitting device calculates the average number of adjacent nodes at different orders according to the average number of edges, including: According to the formula calculate The average number of adjacent nodes of a node ; in, represents the average number of edges.
[0039] Specifically, when the overall node discard probability is set to γ and the subgraph range is Q (that is, after a node is selected, its Q-order adjacent nodes will be discarded), the following steps can be followed to calculate the core node: Subgraph of Probability of being discarded: If the target graph G has N nodes and M edges, then the average number of edges of the nodes in the target graph, that is, the average number of edges connected to each node, is: ; For a randomly sampled node, the average number of adjacent nodes at different orders, that is, the average number of neighbors of the i-th order, is: ; The sum of the number of nodes in the subgraph, that is, the sum of the number of nodes in all discarded subgraphs is: ; When the preset drop probability is γ, the average drop probability of the node is: ; After multiplying the average node drop probability by the attention map α, we can get: ; In order to keep the probability sum equal to 1, the coefficients can be After normalization, we get: ; in, is the number of elements in α, is the sum of all elements in the attention map α.
[0040] S104. The graph neural network anti-overfitting device determines a subgraph to be discarded from the target graph according to the actual discard probability of the node.
[0041] The subgraph to be discarded includes a core node and a plurality of adjacent nodes connected to the core node.
[0042] It should be noted that the plurality of adjacent nodes connected to the core node include the core node Step node.
[0043] It should be noted that the graph neural network anti-overfitting device can determine the subgraphs to be discarded from the target graph in order from high to low according to the preset discard ratio and the actual discard probability of the nodes, and discard the determined subgraphs to be discarded.
[0044] In the embodiments of the present application, on the one hand, since the actual discard probability of the node includes the addition of the attention map, it is possible to automatically focus on the key learning area of the graph neural network, thereby discarding the key learning part of the graph neural network to avoid overfitting; on the other hand, since the subgraph includes a core node and multiple adjacent nodes connected to the core node, it is possible to avoid the graph neural network from recovering the semantic and structural feature information contained in the neighboring nodes of the discarded node, thereby improving the graph convolutional network's ability to resist overfitting.
[0045] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiment disclosed herein, the embodiment of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.
[0046] It should be noted that the device in the embodiment of the present application includes a virtual device and a physical device. The virtual device can be a graph neural network anti-overfitting device, and the physical device can include an electronic device, a computer storage medium, and a computer program product.
[0047] The graph neural network anti-overfitting method provided in the embodiment of the present application can be executed by a graph neural network anti-overfitting device, or a control module for graph neural network anti-overfitting in the graph neural network anti-overfitting device. In the embodiment of the present application, the graph neural network anti-overfitting method executed by a graph neural network anti-overfitting device is taken as an example to illustrate the graph neural network anti-overfitting device provided in the embodiment of the present application.
[0048] It should be noted that the embodiment of the present application can divide the functional modules of the graph neural network anti-overfitting device according to the above method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. Optionally, the division of modules in the embodiment of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.
[0049] like Figure 2 As shown, an embodiment of the present application provides a graph neural network anti-overfitting device 200. The graph neural network anti-overfitting device 200 includes: an acquisition module 201 and a processing module 202. The acquisition module 201 can be used to acquire a target graph; the processing module 202 is used to determine node features according to the target graph, and determine an attention map according to the node features; the attention map includes the attention of each node; the average node discard probability is determined according to the target graph, and the actual node discard probability is determined according to the attention map and the average node discard probability; the subgraph to be discarded is determined from the target graph according to the actual node discard probability; wherein the subgraph to be discarded includes a core node, and a plurality of adjacent nodes connected to the core node.
[0050] Optionally, the processing module 202 is used to take the absolute value of the node feature of each node and determine the corresponding average value to obtain the attention map.
[0051] Optionally, the processing module 202 is used to calculate the average number of edges of the nodes in the target graph; calculate the average number of adjacent nodes at different orders based on the average number of edges; determine the sum of the number of nodes in the subgraph based on the average number of adjacent nodes; and determine the average discard probability of the node based on a preset discard probability and the sum of the number of nodes.
[0052] Optionally, the processing module 202 is used to calculate the calculate The average number of adjacent nodes of a node ;in, represents the average number of edges.
[0053] In the embodiments of the present application, on the one hand, since the actual discard probability of the node includes the addition of the attention map, it is possible to automatically focus on the key learning area of the graph neural network, thereby discarding the key learning part of the graph neural network to avoid overfitting; on the other hand, since the subgraph includes a core node and multiple adjacent nodes connected to the core node, it is possible to avoid the graph neural network from recovering the semantic and structural feature information contained in the neighboring nodes of the discarded node, thereby improving the graph convolutional network's ability to resist overfitting.
[0054] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the graph neural network anti-overfitting method, the method comprising: obtaining a target graph; determining node features according to the target graph, and determining an attention graph according to the node features; the attention graph includes the attention of each node; determining the average node discard probability according to the target graph, and determining the actual node discard probability according to the attention graph and the average node discard probability; determining the subgraph to be discarded from the target graph according to the actual node discard probability; wherein the subgraph to be discarded includes a core node and a plurality of adjacent nodes connected to the core node.
[0055] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0056] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the graph neural network anti-overfitting method provided by the above methods, and the method includes: obtaining a target graph; determining node features based on the target graph, and determining an attention map based on the node features; the attention map includes the attention of each node; determining the average node discard probability based on the target graph, and determining the actual node discard probability based on the attention map and the average node discard probability; determining a subgraph to be discarded from the target graph according to the actual node discard probability; wherein the subgraph to be discarded includes a core node, and multiple adjacent nodes connected to the core node.
[0057] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the graph neural network anti-overfitting method provided by the above-mentioned methods, the method comprising: obtaining a target graph; determining node features based on the target graph, and determining an attention map based on the node features; the attention map includes the attention degree of each node; determining an average node discard probability based on the target graph, and determining an actual node discard probability based on the attention map and the average node discard probability; determining a subgraph to be discarded from the target graph according to the actual node discard probability; wherein the subgraph to be discarded includes a core node, and a plurality of adjacent nodes connected to the core node.
[0058] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0059] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0060] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A graph neural network anti-overfitting method, characterized in that: include: Get the target image; Determining node features according to the target graph, and determining an attention graph according to the node features; The attention map includes the attention degree of each node; Determine an average node drop probability according to the target graph, and determine an actual node drop probability according to the attention graph and the average node drop probability; Determine a subgraph to be discarded from the target graph according to the actual discard probability of the node; The subgraph to be discarded includes a core node and a plurality of adjacent nodes connected to the core node.
2. The graph neural network anti-overfitting method according to claim 1 is characterized in that: The determining of the attention graph according to the node features comprises: The node feature of each node is taken at an absolute value and the corresponding average value is determined to obtain the attention map.
3. The graph neural network anti-overfitting method according to claim 1, characterized in that: Determining the average node drop probability according to the target graph includes: Calculate the average number of edges of nodes in the target graph; Calculate the average number of adjacent nodes at different orders according to the average number of edges; Determine the sum of the number of nodes of the subgraph according to the average number of adjacent nodes; The average node discard probability is determined according to the preset discard probability and the number of nodes.
4. The graph neural network anti-overfitting method according to claim 3 is characterized in that: The calculating the average number of adjacent nodes at different orders according to the average number of edges includes: According to the formula calculate The average number of adjacent nodes of a node ; in, represents the average number of edges.
5. A graph neural network anti-overfitting device, characterized in that: include: Acquisition module and processing module; The acquisition module is used to acquire the target image; The processing module is used to determine node features according to the target graph, and determine an attention graph according to the node features; the attention graph includes the attention of each node; determine an average node discard probability according to the target graph, and determine an actual node discard probability according to the attention graph and the average node discard probability; determine a subgraph to be discarded from the target graph according to the actual node discard probability; The subgraph to be discarded includes a core node and a plurality of adjacent nodes connected to the core node.
6. The graph neural network anti-overfitting device according to claim 5, characterized in that: The processing module is used to take the absolute value of the node feature of each node and determine the corresponding average value to obtain the attention map.
7. The graph neural network anti-overfitting device according to claim 5, characterized in that: The processing module is used to calculate the average number of edges of nodes in the target graph; calculate the average number of adjacent nodes at different orders based on the average number of edges; determine the sum of the number of nodes in the subgraph based on the average number of adjacent nodes; and determine the average discard probability of the node based on a preset discard probability and the sum of the number of nodes.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the graph neural network anti-overfitting method as described in any one of claims 1 to 4 is implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the graph neural network anti-overfitting method as described in any one of claims 1 to 4 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the graph neural network anti-overfitting method as described in any one of claims 1 to 4 is implemented.