A graph neural network adaptive discarding method and system based on information entropy

Abstract

The present invention discloses a graph neural network adaptive discarding method and system based on information entropy, which relates to the technical field of random discarding strategies of graph neural networks, including: receiving a feature vector of a node, performing Softmax normalization processing on the feature vector of the node, and calculating the entropy value corresponding to the node based on information entropy, normalizing the entropy value corresponding to the node, and multiplying it with a preset global maximum discarding rate to obtain a personalized discarding probability of the node; mapping the personalized discarding probability of the node through the edge index mechanism in the PyTorch Geometric framework to obtain a mapping edge, sampling based on the mapping edge to obtain a Bernoulli mask; scaling based on the Bernoulli mask and the personalized discarding probability of the node to obtain a perturbed message matrix, and inputting the perturbed message matrix into a pre-established graph neural network model, thereby realizing adaptive discarding of the graph neural network.

Description

Technical Field

[0001] The present invention relates to the technical field of random discarding strategies for graph neural networks, and specifically to a graph neural network adaptive discarding method and system based on information entropy. Background Art

[0002] Graph Neural Networks (GNN), as a deep learning model specialized for processing non-Euclidean spatial data, have demonstrated powerful modeling capabilities in graph representation learning tasks.

[0003] Despite significant progress in theoretical research and practical applications, graph neural networks (GNNs) continue to face key technical challenges in practical deployment, including overfitting, oversmoothing, and high noise sensitivity, driven by factors such as excessive model complexity, feature convergence caused by deep information aggregation, and high sensitivity to graph topology. Overfitting is primarily due to GNNs' high reliance on node features and graph structure. In training sets with simple graph structures or limited information, the model may overly rely on the local features of specific nodes, resulting in poor generalization to test data. Furthermore, uneven label distribution can cause GNNs to overfocus on classes with large sample sizes and neglect minority classes, further exacerbating overfitting. Deep GNNs aggregate information across multiple layers of neighborhoods, leading to increasingly similar node representations. This suppresses the model's ability to discern local patterns and further exacerbates overfitting. Oversmoothing stems from multi-layer information aggregation. As the number of layers increases, node features become more similar, reducing feature differentiation. This is particularly pronounced in graphs with simple structures or high node similarity. The lack of effective regularization mechanisms exacerbates this problem. High noise sensitivity is due to the high dependence of graph neural networks on graph topology. Any inaccurate or missing edge information may interfere with the propagation of node features, resulting in feature deviations learned by the model, which in turn affects the performance and generalization ability of the model under poor data quality or noisy data. Summary of the Invention

[0004] In order to solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide a graph neural network adaptive discarding method and system based on information entropy.

[0005] In a first aspect, the purpose of the present invention can be achieved by the following technical solution: a graph neural network adaptive discarding method based on information entropy, the method comprising the following steps:

[0006] Receive the node's feature vector, perform Softmax normalization on the node's feature vector, calculate the node's corresponding entropy value based on information entropy, normalize the node's corresponding entropy value, and multiply it by the preset global maximum drop rate to obtain the node's personalized drop probability;

[0007] The personalized drop probability of the node is mapped through the edge index mechanism in the PyTorch Geometric framework to obtain the mapped edge, and sampling is performed based on the mapped edge to obtain the Bernoulli mask;

[0008] Based on the Bernoulli mask and the personalized drop probability of the node, scaling is performed to obtain the perturbed message matrix, which is then input into the pre-established graph neural network model to achieve adaptive drop of the graph neural network.

[0009] In combination with the first aspect, in some implementations of the first aspect, the method further includes: performing Softmax normalization on the feature vector of the node:

[0010] Assume that the node feature matrix is , among which OK Representation node The eigenvector of , uses the Softmax function to normalize the eigenvector into a probability distribution:

[0011]

[0012] Where, Representative Node The probability of belonging to a particular category, represents the original input logits, The exponential operation is part of the Softmax function. is the step of normalizing all categories, An index representing different categories or features; Indicates the Node No. Features, Indicates the Node No. Features.

[0013] In conjunction with the first aspect, in certain implementations of the first aspect, the method further includes: a calculation formula for calculating the entropy value corresponding to the node based on the information entropy is as follows:

[0014]

[0015] Where, is a node The information entropy of is a node After the Softmax function normalization, The probability components of the dimensions, is an index variable representing each dimension in the node feature vector; is the total number of dimensions of node features, Is to prevent constant.

[0016] In conjunction with the first aspect, in certain implementations of the first aspect, the method further includes: normalizing the entropy values corresponding to the nodes, including:

[0017]

[0018] Where, is the normalized information entropy, indicating that the node The information entropy of is the normalized value. It is the maximum value of the information entropy of all nodes.

[0019] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the personalized drop probability of the node is defined by the following formula: the adaptive drop rate of a single node for:

[0020]

[0021] in is the global maximum drop rate hyperparameter, which indicates the highest drop probability that the node with maximum entropy can reach.

[0022] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: a process of performing scaling based on the Bernoulli mask and the personalized drop probability of the node to obtain a perturbed message matrix:

[0023] For each element in the message matrix , according to the source node Adaptive drop rate , performs Bernoulli sampling as follows, where Represents a slave node To Node Message:

[0024]

[0025] Where, is a random variable, is a Bernoulli distribution, is the probability of success from the Bernoulli distribution, Is with the node Related constants.

[0026] Then generate the perturbed message matrix , where the element calculation formula is as follows:

[0027]

[0028] Where, This is the disturbed news. is a node To Node The original message.

[0029] In combination with the first aspect, in some implementations of the first aspect, the method further includes: when When , the message is retained and scaled; when When , the message is completely discarded; the scaling factor Ensure that the perturbed message is consistent with the original value in expectation.

[0030] In conjunction with the first aspect, in certain implementations of the first aspect, the method further includes: the pre-established graph neural network model is as follows:

[0031] In each layer of the graph neural network, each node receives feature information from neighboring nodes, integrates neighbor information through aggregation operations, and performs nonlinear transformations based on its own features, enabling the graph neural network to achieve node classification, graph classification, and link prediction.

[0032] Suppose the undirected graph is denoted by ,in Represents a node collection represents the edge set; the node feature matrix is ,in Representation node The eigenvector of It is the characteristic dimension of the node. The edge describes the relationship between nodes using the adjacency matrix Indicates that The adjacency matrix represents OK, Representation node and If the relationship between , then it means the node and There is an edge connection, and the node degree vector is recorded as ,in Representation and Node The number of connected edges, i.e. , Compute and Node The sum of the weights of all connected edges, the total degree of the graph is given by Calculation; When applying the message passing graph neural network, the message matrix can be expressed as ,in It is the message transmitted between nodes. is the total number of messages passed in the graph, It is the dimension of the message;

[0033] The specific steps of the message passing graph neural network are as follows:

[0034] (1) Message generation of neighbor nodes:

[0035] The characteristics of each node are generated by the message generation function The message generation formula is:

[0036]

[0037] Where, It is in Slave nodes in the layer To Node the message delivered; and Represents nodes respectively and nodes In the The feature vector of the layer, Indicates in Slave nodes in the layer To Node The edge, It is The message generation function of the layer;

[0038] (2) Introduction and calculation of node information entropy:

[0039] The information complexity of each node feature in the graph is quantitatively evaluated by information entropy;

[0040] Normalize the feature vectors of the nodes to convert them into probability distributions; calculate the initial entropy value of each node based on the information entropy formula, and normalize the entropy values of all nodes to obtain normalized information entropy of a unified scale;

[0041] (3) Design of adaptive discard rate:

[0042] The message drop rate of each node is determined based on the normalized information entropy. The drop rate distribution is proportional to the entropy value. The global maximum drop rate is used as a hyperparameter to control the upper limit of the drop intensity.

[0043] (4) Dynamic Control of Message Passing Graph Neural Networks:

[0044] Training phase: Each message is randomly sampled and discarded based on the adaptive discard rate of the source node. If not discarded, it is scaled proportionally to keep the overall expected value stable; if discarded, it is directly set to zero;

[0045] Testing phase: directly use the complete original message matrix without performing any discarding operation.

[0046] In a second aspect, in order to achieve the above-mentioned purpose, the present invention discloses a graph neural network adaptive discarding system based on information entropy, comprising:

[0047] The probability acquisition module is used to receive the node's feature vector, perform Softmax normalization on the node's feature vector, calculate the entropy value corresponding to the node based on the information entropy, normalize the entropy value corresponding to the node, and multiply it by the preset global maximum drop rate to obtain the node's personalized drop probability;

[0048] The sampling module is used to map the personalized dropout probability of nodes through the edge index mechanism in the PyTorch Geometric framework to obtain mapped edges, and then perform sampling based on the mapped edges to obtain the Bernoulli mask;

[0049] The adaptive discarding module is used to perform scaling based on the Bernoulli mask and the personalized discarding probability of the node to obtain the perturbed message matrix, and input the perturbed message matrix into the pre-established graph neural network model, thereby realizing the adaptive discarding of the graph neural network.

[0050] In another aspect of the present invention, in order to achieve the above-mentioned purpose, a terminal device is disclosed, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The memory stores a computer program capable of running on the processor, and when the processor loads and executes the computer program, it adopts the above-mentioned graph neural network adaptive discarding method based on information entropy.

[0051] Beneficial effects of the present invention:

[0052] By analyzing the impact of disturbances on node representation, the present invention can provide a deeper understanding of the mechanism and performance differences of various random drop mechanisms in the model training process. The adaptive drop strategy based on information entropy can effectively reduce sample variance by focusing on discarding information in the message matrix, thereby improving the stability of the training process and accelerating the convergence of the model. The dynamic drop rate strategy based on dynamic evaluation of node information entropy is significantly better than the fixed drop rate strategy while ensuring the same average drop rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0054] Figure 1 It is a schematic flow chart of the method of the present invention;

[0055] Figure 2 Schematic diagram of the principles of three discarding methods of the present invention;

[0056] Figure 3 This is a schematic diagram of the principle of the graph neural network adaptive discarding strategy based on dynamic evaluation of node information entropy of the present invention;

[0057] Figure 4 Schematic diagram of variance calculation for random discarding of the present invention;

[0058] Figure 5 It is a comparison diagram of disturbances with different discarding rates in the dataset of the present invention;

[0059] Figure 6 Schematic diagram of the degree of MADGap over-smoothing in training of the three methods of the present invention;

[0060] Figure 7 This is a schematic diagram of the message transmission principle of the graph neural network of the present invention;

[0061] Figure 8 It is a schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0062] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0063] Example 1:

[0064] like Figure 1 As shown, a graph neural network adaptive discarding method based on information entropy includes the following steps:

[0065] S101: Receive a feature vector of a node, perform Softmax normalization on the feature vector of the node, calculate an entropy value corresponding to the node based on information entropy, normalize the entropy value corresponding to the node, and multiply it by a preset global maximum drop rate to obtain a personalized drop probability of the node;

[0066] The process of performing Softmax normalization on the feature vector of the node:

[0067] Assume that the node feature matrix is , among which OK Representation node The eigenvector of , uses the Softmax function to normalize the eigenvector into a probability distribution

[0068]

[0069] Where, Representative Node The probability of belonging to a particular category, represents the original input logits, The exponential operation is part of the Softmax function. is the step of normalizing all categories, An index representing different categories or features; Indicates the Node No. Features, Indicates the Node No. Features.

[0070] The calculation formula for calculating the entropy value corresponding to the node based on information entropy is as follows:

[0071]

[0072] Where, is a node The information entropy of is a node After the Softmax function normalization, The probability components of the dimensions, is an index variable representing each dimension in the node feature vector. is the total number of dimensions of node features, Is to prevent constant.

[0073] Normalizing the entropy values corresponding to the nodes includes:

[0074]

[0075] Where, is the normalized information entropy, indicating that the node The normalized information entropy is used to unify the scale, making the entropy values of all nodes comparable and avoiding unstable model training due to large differences in node features. It is the maximum value of all node information entropies, that is, the maximum information entropy of all node features in the graph. By dividing the information entropy of each node by the maximum value, the information entropy can be normalized to a smaller range, making its value more uniform and comparable.

[0076] Assume that there are three nodes A, B, and C in the network training, and their eigenvectors are: eigenvector of node A: 0.5, 0.3, 0.2; eigenvector of node B: 0.1, 0.7, 0.2; eigenvector of node C: 0.6, 0.2, 0.2

[0077] Node feature matrix for:

[0078]

[0079] Next, Softmax normalization is applied to the feature vector of each node. For the feature vector of node A [0.5, 0.3, 0.2], the calculation formula of Softmax is:

[0080]

[0081] First, calculate arrive Value:

[0082] Then, calculate the Softmax normalization constant:

[0083]

[0084] So the normalized probability of node A is:

[0085]

[0086] The same process is applied to the feature vectors of nodes B and C.

[0087] The formula for information entropy is:

[0088]

[0089] For node A, its normalized probability Substitute into the calculation:

[0090]

[0091]

[0092] The same approach applies to Node B and Node C.

[0093] Next, the information entropy is normalized. The information entropy of node A is , the information entropy of node B and node C are and .

[0094] Normalize the information entropy of each node, the formula is:

[0095]

[0096] Normalized information entropy of node A:

[0097]

[0098] Normalized information entropy of node B:

[0099]

[0100] Normalized information entropy of node C:

[0101]

[0102] S102: Using the edge indexing mechanism in the PyTorch Geometric framework, the personalized drop probability of a node is mapped to obtain mapped edges. Sampling based on the mapped edges yields a Bernoulli mask. PyTorch Geometric (PyG) is a graph neural network (GNN) library built on PyTorch, specifically designed for processing graph data. The library provides flexible graph data representations and a series of efficient graph convolution operations. PyG primarily relies on the message passing mechanism, making message passing between graph nodes a core function of GNNs. In PyG, graph data is stored in the Data class, with graph edge and node features stored in attributes such as edge_index and x. edge_index is a 2×E tensor, where E is the number of edges in the graph, storing edge connectivity information.

[0103] The personalized drop probability of the node is defined by the following formula: for:

[0104] Where, is the global maximum drop rate hyperparameter, which indicates the highest drop probability that the node with maximum entropy can reach.

[0105] The process of scaling based on the Bernoulli mask and the personalized drop probability of the node to obtain the perturbed message matrix is as follows:

[0106] For each element in the message matrix , according to the source node Adaptive drop rate , performs Bernoulli sampling as follows, where Represents a slave node To Node Message:

[0107]

[0108] Then generate the perturbed message matrix , where the element calculation formula is as follows:

[0109]

[0110] Where, This is the message after the disturbance, indicating that under the discard strategy, the node To Node The value of the message after adjustment. This value is the original message A weighted version of , where the weights depend on the dropping strategy. is a node To Node The original message of . In the absence of a discard policy, the message Represents a slave node To Node The amount or characteristics of information conveyed.

[0111] when When , the message is retained and scaled; when When , the message is completely discarded; the scaling factor Ensure that the perturbed message is consistent with the original value in expectation.

[0112] According to the normalized information entropy, the adaptive drop rate of each node is calculated , assuming the maximum dropout rate max_dropout=0.5, then: the dropout rate of node A:

[0113]

[0114] Drop rate of Node B:

[0115]

[0116] Drop rate of node C:

[0117]

[0118] Assume that the present invention calculates the message from node A to node B , based on the drop rate of node A , perform Bernoulli sampling:

[0119]

[0120] According to the perturbation formula:

[0121]

[0122] S103: Based on the Bernoulli mask and the node's personalized drop probability, the perturbed message matrix is obtained and input into the pre-established graph neural network model to achieve adaptive drop of the graph neural network. Calculating the perturbed message matrix from node A to node B completes the random drop step based on information entropy. This regularization can improve the robustness and generalization ability of the model.

[0123] The pre-established graph neural network model is as follows:

[0124] Graph neural networks are a type of deep learning model designed specifically for processing graph-structured data. Their core mechanism is message passing. Graph neural networks pass information between nodes at each layer, gradually updating the feature representation of the nodes. Figure 7 As shown in the figure, in each layer of a graph neural network, each node receives feature information from neighboring nodes, integrates this information through aggregation, and performs nonlinear transformations based on its own features, thereby continuously optimizing the node's representation. This mechanism enables graph neural networks to achieve tasks such as node classification, graph classification, and link prediction.

[0125] Suppose an undirected graph is denoted by ,in Represents a node collection represents the edge set. The node feature matrix is ,in Representation node The eigenvector of Is the characteristic dimension of the node. The edge describes the relationship between nodes using the adjacency matrix Indicates that The adjacency matrix represents OK, Representation node and If the relationship between , then it means the node and There are edge connections. The node degree vector is recorded as ,in Representation and Node The number of connected edges, i.e. . Compute and Node The sum of the weights of all connected edges. Meanwhile, the total degree of the graph is given by Calculation. When applying the message passing graph neural network on the graph G, the message matrix can be expressed as ,in It is the message transmitted between nodes. is the total number of messages passed in the graph, It is the dimension of the message.

[0126] from Figure 7 As can be seen from the figure, the nodes are connected by edges, forming an undirected graph. Figure 2 As shown in the input graph in Its neighboring nodes 、 、 Nodes are connected by edges. Each node has a feature vector ,in Representative Node , Represents the current level (or time step). Edges represent the relationship between nodes, usually through the adjacency matrix To indicate that the element Representation node and The specific steps of the message passing graph neural network are as follows:

[0127] Neighbor node message generation. The characteristics of each node are generated by the message generation function The features of its neighboring nodes and edges work together to generate a message. The message generation formula is:

[0128]

[0129] Where, It is in Slave nodes in the layer To Node the message delivered; and Represents nodes respectively and nodes In the The feature vector of the layer, Indicates in Slave nodes in the layer To Node The edge, It is The message generation function of the layer generates messages based on the characteristics of nodes and edges. Figure 2 The dotted box in the middle represents the message generation function ,Each edge with an arrow represents the generation and transmission of a message, and the direction of the arrow points from the neighbor node to the target node.

[0130] Aggregation: After receiving the message, the central node The information from its neighboring nodes needs to be aggregated. There are many ways to aggregate, common ones include summing, averaging, or maximizing. Through the aggregation operation, a node can combine the messages from all neighboring nodes to generate a new representation. The formula is as follows:

[0131]

[0132] Node update: The aggregated result will be used as the input for the next node update. The central node Based on its current status and aggregated messages to update its own features. The updating process is done through the transformation function Complete, generate new node representation , the formula is as follows:

[0133]

[0134] Where, Representation node In the Feature representation of the layer; Is a transformation function that combines the characteristics of the current node and aggregate messages from neighboring nodes to generate updated node features. Figure 2 In the figure, the arrow goes from the transformation function Points to the updated node feature , which represents the updating process of node features.

[0135] By continuously generating and aggregating messages and updating nodes, graph neural networks optimize node feature representations layer by layer. Information transfer at each layer allows nodes to not only utilize their own unique features but also gradually incorporate the global structure of the graph through interactions with neighboring nodes. After these multiple layers of information transfer, node features become more accurate and representative, making them more suitable for graph analysis tasks.

[0136] This model, based on graph neural networks (GNNs) and an entropy-driven adaptive drop strategy, aims to optimize information flow within GNNs by dynamically adjusting the drop rate of node information. This approach effectively avoids overfitting, oversmoothing, and information redundancy, improving the model's performance on complex graph data. This invention innovates on traditional GNNs by dynamically controlling the information flow of each edge using entropy. Model Architecture Overview: GNNs update node representations by aggregating features from neighboring nodes. Typically, nodes update their features by exchanging messages with their neighbors via the adjacency matrix. This model, based on traditional GNNs, introduces an entropy-driven adaptive drop strategy to dynamically adjust the message drop probability. Specifically, the message drop probability for each node is determined based on the complexity of the node's features (i.e., entropy). Nodes with higher entropy have a greater message drop probability.

[0137] To quantitatively assess the information complexity of each node's features, this paper introduces node information entropy as a metric. Information entropy effectively reflects the uncertainty and diversity of node feature distribution, thereby characterizing the information richness of nodes in the graph structure. By introducing information entropy, it not only helps identify key nodes but also provides a theoretical basis and quantitative support for subsequent feature selection and representation learning. It is defined as follows:

[0138]

[0139] Where, is a node The eigenvector of After softmax normalization, dimensional components; Is to prevent A small constant of

[0140] A node's information entropy reflects the balanced distribution of its features across all dimensions and is an important indicator of the complexity and uncertainty of a node's information. A higher entropy value indicates a more even distribution across all feature dimensions, more complex and richer the information carried, and also greater information redundancy. Therefore, such nodes are more tolerant of feature perturbations or random dropouts during message transmission. Conversely, nodes with lower entropy values tend to have more concentrated and structured key information, and their feature distribution is highly biased, indicating a more deterministic and compact semantic expression. In such cases, their messages must be preserved as completely as possible to avoid loss of critical information. Feature analysis based on information entropy enables more refined message control strategies, thereby preserving valid information while improving the model's robustness and generalization capabilities.

[0141] Specifically, the present invention will be further described below through examples:

[0142] Based on the implicit regularization effect introduced by the random dropout strategy in the graph neural network, this paper focuses on how to influence the representation learning and generalization ability of the model by regulating the intensity of disturbances during message propagation. To facilitate theoretical derivation and analysis, the present invention discusses this under a simplified setting, assuming that the graph neural network model used is a single-layer structure and the downstream task is a binary classification problem. In this setting, the representation of each node is obtained by aggregating and transforming the features of its neighboring nodes, thereby forming an embedded expression of the local graph structure. By analyzing the impact of disturbances on node representation, we can have a deeper understanding of the action mechanism and performance differences of various random dropout mechanisms in the model training process. Specifically, it can be expressed as:

[0143]

[0144] Where, is the updated feature matrix of the node, is the node adjacency matrix (normalized), Message Matrix is the transformation matrix. Next, the sigmoid activation function is used for nonlinear transformation, which is expressed as:

[0145]

[0146] When training a model, the cross entropy loss function is usually used to measure the difference between the prediction and the actual label. The expression of the cross entropy loss function is:

[0147]

[0148] Where, and Represents nodes respectively and The output features of and is the label of the node.

[0149] During training, when applying the random dropout method, the original message matrix is no longer used. , but use the perturbed message matrix , that is, some elements of the message matrix are masked by the discard operation. Under the framework of the random discard method driven by information entropy, the perturbed message matrix is obtained by applying a Bernoulli mask to each element To generate. Specifically expressed as:

[0150]

[0151] Where, is the global drop rate, It is a mask generated by Bernoulli distribution, which is used to control whether each message is discarded. Apply the mask to each element in , can realize the random discard of message matrix, the formula is a very small constant (such as ), used to avoid numerical instability, can be ignored in the inference formula. The message matrix after perturbation is Substituted into the objective function, the new expected loss function is recorded as The expected value of the objective function can be expressed as:

[0152]

[0153] Simplified to:

[0154] in:

[0155]

[0156]

[0157] right and Taylor expansion of the two functions:

[0158]

[0159] Will and Substitute the second-order Taylor expansion of The formula is:

[0160]

[0161] Used in the text Approximate substitution , since the true label The variance characteristics of show a similar behavior pattern to the predicted probability—the variance reaches its maximum when the probability value is close to 0.5, and gradually decreases when it approaches 0 or 1. This correspondence not only facilitates theoretical understanding, but also provides a simplified basis for formula derivation.

[0162] because , The formula is finally simplified to:

[0163]

[0164] After introducing random dropout perturbations and performing Taylor expansion on the loss function, the final expected loss function is obtained:

[0165]

[0166] As can be seen from the above formula, this loss function contains a regularization term, which can suppress overfitting and oversmoothing, thereby enhancing the model's generalization and robustness. By reducing the variance of node features, the regularization term encourages the model to focus more on important features, reduces the impact of redundant information, and makes node representations more stable and reliable. In this way, the random dropout method can improve the performance of graph neural network models when processing complex graph data, especially in high-dimensional tasks, improving the model's robustness and generalization.

[0167] This paper theoretically analyzes the adaptive message discarding strategy based on information entropy and demonstrates its advantages over other discarding strategies from two aspects: (1) By comparing the differences in sample variance between this method and other discarding rate methods, it is proved that the adaptive discarding strategy based on information entropy can effectively reduce the sample variance by focusing on the information in the discarded message matrix, thereby improving the stability of the training process and accelerating the convergence of the model. (2) By deriving and comparing the variance of the dynamic discarding rate strategy and the fixed discarding rate strategy, it is proved that the dynamic discarding rate strategy based on the dynamic evaluation of node information entropy is significantly better than the fixed discarding rate strategy while ensuring the same average discarding rate.

[0168] We demonstrate the effectiveness of an entropy-based message dropping strategy in reducing sample variance. This includes establishing a mathematical model for the strategy and deriving theoretical bounds on its variance reduction. Furthermore, through comparative analysis with existing random dropout methods, we theoretically demonstrate the effectiveness of the proposed strategy in improving model training stability and accelerating convergence.

[0169] The random drop method is essentially to Perform masking operations to convert to corresponding discarding methods. Message Matrix It is usually used to represent the communication process between nodes in graph neural networks. Therefore, the present invention evaluates the impact of these methods by comparing the message matrices in different training cycles and calculating the sample variance through the norm of the message matrix.

[0170] like Figure 4 As shown, assuming that the original message matrix is a size of is a matrix of all 1s, that is, the value of each element is 1. Based on this assumption, the sample variance can be calculated by the 1-norm of the message matrix. To simplify the analysis, it is assumed that each directed edge corresponds to a row vector in the message matrix, the graph is assumed to be undirected, and the degree of each node is , that is, each node is connected to the four surrounding nodes through undirected edges. Therefore, there are two edges between every two connected nodes. As shown in the figure, there are two edges between node 1 and node 11, one from node 1 to node 11 and the other from node 11 to node 1. From the figure, we can see that the total number of edges, that is, the total number of rows in the message matrix, is: .

[0171] The random dropout method can be regarded as multiple independent Bernoulli samplings. The whole process conforms to the binomial distribution, so the message matrix can be calculated The variance of the message matrix of different random drop methods and this method will be calculated below. variance.

[0172] Dropout: This method randomly discards features equal to the number of adjacent nodes after sampling the features on each node. Figure 4 There are nodes, and the feature dimension of each node is and There is an adjacent node. After Bernoulli sampling, each time an element is dropped, it masks the elements.

[0173] Therefore, the method message matrix The variance of is:

[0174]

[0175] Substitute the values into:

[0176]

[0177] DropEdge: This method randomly drops all feature elements on an edge after sampling all edges. Figure 4 There are edges (assuming it is an undirected graph, so there are two edges between every two nodes), the characteristic dimension of each edge is Therefore, after After Bernoulli sampling. Every time an element is dropped, it masks the message matrix elements.

[0178] Therefore, the method message matrix The variance of is:

[0179]

[0180] Substitute the values into:

[0181]

[0182] (3) DropNode: This method randomly drops all feature elements on a node after sampling all nodes. Figure 4 There are nodes, and the feature dimension of each node is and There is an adjacent node. After Bernoulli sampling, each time an element is dropped, it masks the elements.

[0183] Therefore, the method message matrix The variance of is:

[0184]

[0185] Substitute the values into:

[0186]

[0187] (4) Adaptive message discarding strategy based on information entropy: This method samples all feature elements on all nodes and discards one element. Figure 4 There are nodes, and the feature dimension of each node is and There is an adjacent node. After 100 Bernoulli samples, each time an element is dropped, it masks 1 element in the message matrix.

[0188] Therefore, the method message matrix The variance of is:

[0189]

[0190] Substitute the values into:

[0191]

[0192] in, is the discard rate, is the drop probability dynamically calculated based on node feature entropy, is the number of nodes, is the feature dimension, is the degree of each node

[0193] Based on the above variance analysis results, we can quantitatively compare the variance characteristics of different random dropout methods. The variances of each sample have the following relationship:

[0194]

[0195] In actual application scenarios, the number of nodes in the graph , average node degree and feature dimensions It is much larger than the simplified conditions set in the theoretical analysis of this invention, usually several times or even dozens of times higher. Under such high-dimensional, large-scale graph structures, the variance introduced by the existing random dropout method during the training process will be significantly amplified, thereby exacerbating the instability and performance fluctuations of the model. In contrast, the method proposed in this invention has better performance in controlling the intensity of disturbances and suppressing the growth of variance. Therefore, in actual large-scale graph data, the gap in variance control ability between it and other methods will also be significantly widened, further highlighting the advantages of this method in stability and generalization performance.

[0196] Systematic experiments were conducted on four benchmark graph datasets: Cora, CiteSeer, PubMed, and Flicker, to verify the effectiveness and universality of an adaptive message drop strategy based on node information entropy. Using GCN and GAT as the foundational framework, the experiments compared the strategy with mainstream dropout strategies, including Dropout, DropEdge, and DropNode. The performance was evaluated specifically on two typical graph learning tasks: node classification and link prediction.

[0197] Dataset introduction and experimental parameter setting

[0198] This paper selected five widely used benchmark graph datasets: Cora, CiteSeer, PubMed, and Flicker for experimental evaluation. These datasets cover a wide range of graph structures (from academic citation networks to social networks), different scales (from thousands to tens of thousands of nodes), and various graph learning tasks (such as node classification and link prediction). This allows for a comprehensive and systematic evaluation of the performance of different message dropping strategies in various practical scenarios. Detailed data for each dataset is shown in Table 1.

[0199] Table 1 Dataset information

[0200]

[0201] Cora, CiteSeer, and PubMed are three typical academic paper citation network datasets, where nodes represent academic papers and edges represent citation relationships between papers. These datasets are often used for node classification tasks. Classification experiments based on paper content features (such as keywords and abstracts) can effectively evaluate the performance of graph neural networks in academic network analysis.

[0202] The Flicker dataset is a graph-structured dataset constructed from the social media platform Flickr. Nodes represent user-uploaded images, and edges reflect connections between images based on visual similarity or social connections. This dataset is widely used for image classification tasks. By classifying image nodes into predefined semantic categories, it effectively evaluates the representation learning and classification performance of graph neural networks in processing multimedia content in social network environments.

[0203] This experiment will use the AdamW optimizer for training on RTX5070. The experimental environment is shown in Table 2:

[0204] Table 2 Experimental environment

[0205]

[0206] The training epoch is set to 500, and the model consists of a two-layer graph convolution module, and the hidden layer dimension of the two-layer graph convolution is set to 16. The learning rate is 0.005. The regularization rate is , The value of is set to .

[0207] On four benchmark datasets, Cora, CiteSeer, PubMed, and Flickr, the proposed method was systematically compared with traditional random dropout strategies (including DropNode, DropEdge, and Dropout) in the node classification task. The results are shown in Tables 3 and 4. As can be seen from the table, compared with the baseline graph neural network model without a dropout mechanism, all methods that introduced a dropout strategy achieved performance improvements in classification accuracy; however, the performance of different strategies on different datasets varied significantly. For example, DropEdge performed best on the Cora dataset, but showed a significant performance decline on the Flickr dataset.

[0208] Table 3 Comparison of the accuracy of this method and other dropout methods in the GCN backbone network

[0209]

[0210] Table 4 Comparison of the accuracy of this method and other discarding methods in the GAT backbone network

[0211]

[0212] In contrast, the node information entropy-based adaptive drop strategy proposed in this paper achieved optimal performance in all test scenarios, demonstrating superior stability and adaptability, validating its broad applicability and robustness in graph neural networks. Specifically: 1) In terms of classification accuracy, this paper achieved an average improvement of approximately 5% compared to traditional drop strategies; 2) In terms of stability, it effectively balanced information retention and noise suppression by dynamically adjusting the drop rate; and 3) In terms of adaptability, it consistently maintained leading performance across diverse graph structures and task scenarios.

[0213] In summary, the present invention demonstrates significant advantages in accuracy, stability, and generalization ability, verifying the effectiveness and advancement of the adaptive message discarding strategy based on information entropy in graph neural networks.

[0214] Robustness analysis

[0215] This paper evaluates the classification performance of different dropping strategies on perturbed graphs, analyzing the robustness of each strategy through comparison. To ensure the cleanliness of the initial data, four benchmark datasets, Cora, CiteSeer, PubMed, and Flickr, were selected as experimental subjects. Graph structure perturbations were simulated by randomly adding edges at varying ratios (with the perturbation rate increasing from 0% to 30%) to the original graph. Node classification was then performed on this basis.

[0216] As can be seen from Tables 5 to 8, as the disturbance rate increases, the adaptive discarding strategy of the present invention has the smallest accuracy drop, indicating that it has strong anti-interference ability. In the extreme case of a disturbance rate of 30%, the classification accuracy of the Cora, CiteSeer, PubMed, and Flickr datasets dropped by 5.71%, 2.88%, 3.13%, and 1.73%, respectively. In addition, Figure 5 It can also be intuitively observed from the visualization results that the performance degradation of the present invention at each disturbance level is significantly lower than that of other comparison methods, further verifying the significant advantages of the present method in terms of robustness and stability.

[0217] Table 5 Robustness test results and comparison in Cora dataset

[0218]

[0219] Table 6 Robustness test results and comparison in CiteSeer dataset

[0220]

[0221] Table 7 Robustness test results and comparison in PubMed dataset

[0222]

[0223] Table 8 Robustness test results and comparison in Flickr dataset

[0224]

[0225] Oversmoothness analysis

[0226] In order to quantitatively evaluate the over-smoothing phenomenon, this paper uses MADGap as a measurement indicator. MADGap (Mean Absolute Difference Gap) is a quantitative indicator for measuring the difference in node representation between layers in a graph neural network, which is defined as the feature matrix of nodes in adjacent layers. Norm difference mean:

[0227]

[0228] in is the total number of neural network layers, is the total number of nodes in the graph, It is Tier The feature vector of each node, yes Norm, that is, the sum of the absolute values in the feature dimension.

[0229] This metric reflects the model's resistance to oversmoothing by calculating the absolute differences in node representations layer by layer and averaging them globally. A low MADGap value indicates that node features across different layers are becoming homogenized, posing a risk of oversmoothing. Conversely, a low MADGap value indicates that the model is effectively maintaining the discriminability of features across layers. Therefore, MADGap serves as an important measure of a model's resistance to oversmoothing, reflecting the degree of disparity in node representations across layers. A smaller value indicates highly consistent node representations, suggesting a tendency toward oversmoothing. A larger value indicates good discriminability between layer representations, indicating that the model is effectively maintaining node discriminability during training.

[0230] Figure 6 The results show that the MADGap value of our method increases steadily with increasing training rounds and eventually stabilizes, indicating that the model maintains a high degree of node representation differentiation during training, significantly alleviating the oversmoothing problem. In contrast, the MADGap values of the GCN-DropNode and GCN-DropEdge methods fluctuate significantly during training, and the differences in representation between layers are unstable, indicating a more pronounced oversmoothing phenomenon.

[0231] like Figure 2-Figure 7 As shown,

[0232] Figure 2The message matrix in the figure represents the feature information between nodes. In the figure, the connections between nodes 1, 2, 3, 4, and 5 represent nodes 2 through 5, which are adjacent to node 1. Each matrix element represents the feature information between two nodes: m12 represents the feature information between nodes 1 and 2; m13 represents the feature information between nodes 1 and 3; m14 represents the feature information between nodes 1 and 4; and m15 represents the feature information between nodes 1 and 5. These feature information of adjacent nodes constitute the message matrix in the figure. The color depth in the matrix represents the level of information entropy; darker colors indicate higher entropy and more complex information flow. Stripes in the matrix represent discarded information. The subfigures (a), (b), and (c) below the figure illustrate three different random dropout strategies: (a) represents the Dropout method, (b) represents the DroNode method, and (c) represents the DroEdget method. The color bar below represents the information entropy value, with a color gradient (light colors indicate low entropy, dark colors indicate high entropy). Striped squares represent discarded information.

[0233] Figure 3 The message matrix in Figure 1 represents the characteristic information between nodes. In the figure, node 1 is connected to nodes 2, 3, 4, and 5, representing nodes 2 through 5 adjacent to node 1. Each matrix element represents the characteristic information between two nodes: m12 represents the characteristic information between nodes 1 and 2; m13 represents the characteristic information between nodes 1 and 3; m14 represents the characteristic information between nodes 1 and 4; and m15 represents the characteristic information between nodes 1 and 5. The color depth in the matrix indicates the level of information entropy, with darker colors indicating higher entropy. Stripes in the matrix represent discarded information. The right side of the matrix shows the information discarding method in the message matrix of this method. The color bar below represents the color gradient of the information entropy value (from low to high) (light colors indicate low entropy, dark colors indicate high entropy); striped squares indicate discarded information.

[0234] Figure 4 (b) in the figure indicates a size of The message matrix of the all-1 matrix , that is, the value of each element is 1. n represents the number of nodes, that is Figure 4Nodes 1, 2, 3, and 4 in (c) are adjacent nodes. 1, 2, 3, and 4 represent the feature information on nodes 1, 2, 3, and 4, respectively. c represents the feature dimension, i.e., the number of information dimensions between each node. d represents the node degree, i.e., the number of adjacent nodes to each node. Node 1 is adjacent to nodes 11, 12, 13, and 14. Node 2 is adjacent to nodes 21, 22, 23, and 24. Node 3 is adjacent to nodes 31, 32, 33, and 34. Node 4 is adjacent to nodes 41, 42, 43, and 44. Figure 4The k in (a) represents the total number of feature information on the edge, which is also the total number of edges. 1,11 represents the edge between node 1 and node 11 from node 1 to node 11, and 11,1 represents the edge between node 1 and node 11 from node 11 to node 1; 1,12 represents the edge between node 1 and node 12 from node 1 to node 12, and 12,1 represents the edge between node 1 and node 12 from node 12 to node 1; 1,13 represents the edge between node 1 and node 13 from node 1 to node 13, and 13,1 represents the edge between node 1 and node 13 from node 13 to node 1; 1,14 represents the edge between node 1 and node 14 from node 1 to node 14, and 14,1 represents the edge between node 1 and node 14 from node 14 to node 1; 2,21 indicates the edge between node 2 and node 21 from node 2 to node 21, and 21,2 indicates the edge between node 2 and node 21 from node 21 to node 2; 2,22 indicates the edge between node 2 and node 22 from node 2 to node 22, and 22,2 indicates the edge between node 2 and node 22 from node 22 to node 2; 2,23 indicates the edge between node 2 and node 23 from node 2 to node 23, and 23,2 indicates the edge between node 2 and node 23 from node 23 to node 2; 2,24 indicates the edge between node 2 and node 24 from node 2 to node 24, and 24,2 indicates the edge between node 2 and node 24 from node 24 to node 2; 3,31 indicates the edge between node 3 and node 31 from node 3 to node 31, and 31,3 indicates the edge between node 3 and node 31 from node 31 to node 3; 3,32 indicates the edge between node 3 and node 32 from node 3 to node 32, and 32,3 indicates the edge between node 3 and node 32 from node 32 to node 3; 3,33 indicates the edge between node 3 and node 33 from node 3 to node 33, and 33,3 indicates the edge between node 3 and node 33 from node 33 to node 3; 3,34 indicates the edge between node 3 and node 34 from node 3 to node 34, and 34,3 indicates the edge between node 3 and node 34 from node 34 to node 3; 4,41 indicates the edge between node 4 and node 41 from node 4 to node 41, and 41,4 indicates the edge between node 4 and node 41 from node 41 to node 4; 4,42 indicates the edge between node 4 and node 42 from node 4 to node 42, and 42,4 indicates the edge between node 4 and node 42 from node 42 to node 4; 4,43 indicates the edge between node 4 and node 43 from node 4 to node 43, and 43,4 indicates the edge between node 4 and node 43 from node 43 to node 4; 4,44 indicates the edge between node 4 and node 44 from node 4 to node 44, and 44,4 indicates the edge between node 4 and node 44 from node 44 to node 4.

[0235] Figure 5Four subgraphs are shown, representing experimental results on four different datasets: Cora, CiteSeer, PubMed, and Flickr. GCN-DropNode represents experiments integrating DropNode into a GCN network; GCN-DropEdge represents experiments integrating DropEdge into a GCN network; GCN-Invention represents experiments integrating this method into a GCN network; and GCN-Dropout represents experiments integrating Dropout into a GCN network. Each subgraph shows how the performance of different dropout strategies changes with the dropout rate. The horizontal axis in each subgraph represents the amount of perturbation added to the dataset: 10% represents the addition of 10% perturbation; 20% represents the addition of 20% perturbation; and 30% represents the addition of 30% perturbation. The vertical axis represents the accuracy under the corresponding perturbation. The vertical axis of the experimental results for the Cora dataset is marked with 0.675, 0.700, 0.725, 0.750, 0.775, 0.800, and 0.825. The accuracy ranges from 0.675 to 0.825 in intervals of 0.025. The experimental results for the CiteSeer dataset are marked on the vertical axis as 0.580, 0.600, 0.620, 0.640, 0.660, 0.680, 0.700, and 0.720. The accuracy ranges from 0.580 to 0.720 in intervals of 0.020. The experimental results for the PubMed dataset are marked on the vertical axis as 0.730, 0.740, 0.750, 0.760, 0.770, 0.780, and 0.790. The accuracy ranges from 0.730 to 0.790 in intervals of 0.010. The experimental results for the Flickr dataset are marked on the vertical axis as 0.420, 0.440, 0.460, 0.480, 0.500, and 0.520. The accuracy ranges from 0.420 to 0.520 in intervals of 0.020. GCN represents experiments using various dropout methods integrated into this network. The yellow line shows the experimental results of the DropNode method in the GCN network; the blue line shows the experimental results of the DropEdge method in the GCN network; the red line shows the experimental results of the Dropout method in the GCN network; and the black line shows the experimental results of this method in the GCN network.

[0236] Figure 6The over-smoothness of DropNode, DropEdge and the present invention in different training groups is shown. GCN-DropNode means integrating DropNode into the GCN network for experiments; GCN-DropEdge means integrating DropEdge into the GCN network for experiments; GCN-present invention means integrating this method into the GCN network for experiments. The horizontal axis is the number of training groups, from 0 to 500 times, with an interval of 100 times. The vertical axis is the MADGap over-smoothness value, from 0 to 0.6 with an interval of 0.1. GCN means integrating various drop methods on this network for experiments. The blue line is the trend of change in the MADGap over-smoothness value of the DropEdge method under the GCN network as the number of training groups increases; the orange line is the trend of change in the MADGap over-smoothness value of the DropEdge method under the GCN network as the number of training groups increases; the green line is the trend of change in the MADGap over-smoothness value of the DropEdge method under the GCN network as the number of training groups increases.

[0237] Figure 7 The processing flow of the graph neural network is shown. Part (a) shows the input graph, which contains nodes , , , , and the connection relationship between them: and 、 、 Adjacent nodes are connected by edges to form an undirected graph. Part (b) shows the processing flow of the graph neural network. Each node has a feature vector ,in Representative Node , Indicates the current level (or time step). Representation node In the The feature vector of the layer; Representation node In the The feature vector of the layer; Representation node In the The feature vector of the layer; Representation node In the The feature vector of the layer. Represents the relationship connecting these two nodes. Represents a slave node To Node edge; Represents a slave node To Node edge. Represents a slave node To Node edge. Is the message generation function, through Functions can generate messages. It is a slave node To Node The message delivered. 、 and Generated by message generation function ; 、 and Generated by message generation function ; 、 and Generated by message generation function . Indicates message aggregation; Represents a transformation function. 、 and After the message is aggregated and the node In the The feature vector of the layer Through the transformation function Update nodes to generate new node feature representations .

[0238] It can be seen that the method proposed in the present invention performs better in suppressing over-smoothing, improving node representation discriminability and maintaining model stability, which further verifies the effectiveness and advantages of the present invention.

[0239] Specifically, in order to verify the effect of the present invention, an experiment on social network anomaly detection by integrating graph convolutional neural network was conducted in combination with this application.

[0240] Data collection

[0241] 1. Node Data: User behavior data (such as posting frequency, number of likes, number of comments, and social interaction behavior) is a core node feature. This data is typically obtained from social platforms through APIs. This includes: each user's posting, like, and comment frequency; and user interaction records (such as which users they interact with and how often they interact).

[0242] 2. Graph structure data: Structural information of social networks (such as friendships and followings between users) is used to construct the adjacency matrix of the graph. Specifically, the relationships between users determine the edges in the graph. Common social relationships include:

[0243] Whether user A and user B follow each other;

[0244] Whether user A has interacted with user B (e.g., commenting, liking, etc.).

[0245] Data preprocessing

[0246] After collecting data, it needs to go through preprocessing steps to ensure that it is suitable for subsequent model training and inference.

[0247] 1. Data cleaning: Remove invalid or missing data, such as missing user behavior records or incomplete user information.

[0248] 2. Feature normalization and standardization: To ensure that the model can handle features of different scales, each user's behavioral characteristics need to be normalized or standardized. For example, post frequency, number of likes, and number of comments can be normalized to be in the range of [0, 1].

[0249] 3. Convert node features to probability distribution: Apply Softmax normalization to the behavioral features of each node to convert them into probability distribution form:

[0250]

[0251] 4. Information entropy calculation: Calculate the entropy value of each node. The higher the entropy value, the richer the information.

[0252] Feature selection and dropout rate calculation

[0253] 1. Feature selection: Select features useful for model training by calculating entropy. Nodes with higher entropy values typically have richer behavioral information, and a lower dropout rate can be used to retain more information.

[0254] 2. Drop rate calculation: Calculate the adaptive drop rate of each node based on its entropy value. The drop rate of a node is positively correlated with its entropy value. The higher the entropy, the higher the drop rate. The drop rate calculation formula is:

[0255]

[0256] Issues that need attention in specific applications

[0257] Identification of abnormal users

[0258] Abnormal user characteristics: Abnormal users often exhibit behavioral patterns that differ from the majority of users. For example, they may post at an unusually high frequency, have a disproportionate number of likes and comments, or exhibit highly monotonous behavior. A discard strategy based on information entropy can accurately identify these abnormal users by dynamically adjusting the discard rate.

[0259] The relationship between entropy value and discard rate: nodes with lower entropy values have simpler behaviors, while nodes with higher entropy values have more complex behaviors. Therefore, nodes with higher entropy values should retain more information to avoid the loss of important behavioral features.

[0260] Data volume and computational overhead

[0261] Computational overhead of large-scale social networks: As social networks grow in size, the number of nodes and feature dimensions increase significantly. This can lead to significant overhead in calculating entropy and adjusting the dropout rate. To address this, a distributed computing framework can be used to parallelize graph data processing, reducing computational bottlenecks.

[0262] Adjust the dropout rate: Adjust the maximum dropout rate and entropy threshold based on the data characteristics and application scenarios. If the dropout rate is set too high, key information may be discarded, resulting in degraded model performance; if the dropout rate is too low, it may not effectively reduce redundant information.

[0263] Example: Node Abnormal users may post frequently but rarely interact with other users. Their behavior is less complex, so their entropy is lower and the dropout rate is lower, allowing the model to focus on their main behavioral characteristics.

[0264] Reduce redundant information dissemination

[0265] Balance between information retention and discarding: Through the calculation of entropy values and dynamic adjustment of the discard rate, the model can retain more information on information-rich nodes and discard unnecessary features on information-redundant nodes, thus avoiding information overload and over-smoothing problems.

[0266] Example: For a node (active users), whose behavioral characteristics are rich and the information entropy is high. The model retains more information through a low discard rate, which helps to improve the behavioral learning of normal users.

[0267] Improve the stability and robustness of the model

[0268] Better generalization capability: The adaptive dropout mechanism helps the model avoid overfitting during training, reduces the impact of noise and redundant information, and improves the robustness of the model in the face of complex and dynamically changing data.

[0269] Accurate identification of abnormal user behavior

[0270] Combining entropy with a discard strategy: When detecting abnormal users, users with low information entropy (such as those who frequently post but don't interact) are more likely to have unnecessary information discarded, ensuring the model focuses on important nodes. This strategy allows the model to more accurately identify users with abnormal behavior and improve detection accuracy.

[0271] Using the social network data shown in Table 9:

[0272] Node feature matrix:

[0273] Table 9 Social network data

[0274]

[0275] Adjacency matrix (representing the interaction relationship between users):

[0276]

[0277] Calculate the entropy value of each node

[0278] Softmax normalizes the features of each node:

[0279] Apply the Softmax function to the behavioral characteristics of each node (such as posting frequency, number of likes, and number of comments) to convert them into probability distribution.

[0280] For example, for the node , the formula for Softmax normalization is:

[0281]

[0282] Hypothetical Node The characteristics of , the present invention calculates the normalized probability of each feature:

[0283]

[0284] So, the node The normalized probability is: [0.3935, 0.2187, 0.3878]

[0285] Calculate information entropy:

[0286] Use the information entropy formula:

[0287]

[0288] For Node , calculate its entropy:

[0289]

[0290] node The entropy value is 1.0383.

[0291] Perform the same calculation for other nodes, assuming the entropy values are as follows:

[0292] node The entropy value is 1.0383

[0293] node The entropy value is 0.9060

[0294] node The entropy value is 0.8887

[0295] node The entropy value is 1.0532

[0296] node The entropy value is 0.9300

[0297] node The entropy value is 0.4020 (abnormal user)

[0298] Normalized entropy

[0299] Calculate the normalized entropy value of each node:

[0300]

[0301] The normalized entropy of each node is as follows:

[0302] node :1.0383 / 1.0532=0.9859

[0303] node :0.9060 / 1.0532=0.8603

[0304] node :0.8887 / 1.0532=0.8430

[0305] node :1.0532 / 1.0532=1.0000

[0306] node :0.9300 / 1.0532=0.8833

[0307] node :0.4020 / 1.0532=0.3813 (abnormal users)

[0308] Calculating the discard rate

[0309] The drop rate of each node is calculated based on the normalized entropy value. Assume that the global maximum drop rate is 0.5.

[0310]

[0311] For example, for the node :

[0312]

[0313] Calculation results:

[0314] node The discard rate =0.4929

[0315] node The discard rate =0.4302

[0316] node The discard rate =0.4215

[0317] node The discard rate =0.5000

[0318] node The discard rate =0.4417

[0319] node The discard rate =0.1906 (abnormal user)

[0320] Bernoulli sampling and message dropping

[0321] Bernoulli sampling: Based on the drop rate of each node, the present invention uses Bernoulli distribution for sampling to determine whether each message should be retained:

[0322]

[0323] Among them: When , the message is retained and scaled; when , the message is completely discarded.

[0324] Message scaling: Scale the retained messages. The formula is as follows:

[0325]

[0326] Among them: When , the message is retained and scaled; when When , the message is completely discarded; the scaling factor Ensure that the perturbed message is consistent with the original value in expectation.

[0327] Messaging and feature updates

[0328] After being discarded and scaled, the messages will participate in the next step of node feature aggregation, thereby updating the node features. The perturbation (discarding messages) during information transmission helps to enhance the robustness of the model and reduce the propagation of redundant information.

[0329] Final result:

[0330] node ,Due to its low discard rate, messages are almost never discarded, ,retaining more information and ensuring that the model makes full use of this ,information.

[0331] node ,Due to its high discard rate, messages will be discarded, reducing the interference of redundant information and helping the model identify abnormal users.

[0332] Example 2: Figure 8 As shown, in order to achieve the above purpose, the present invention discloses a graph neural network adaptive discarding system based on information entropy, comprising:

[0333] The probability acquisition module 11 is used to receive the feature vector of the node, perform Softmax normalization on the feature vector of the node, calculate the entropy value corresponding to the node based on the information entropy, normalize the entropy value corresponding to the node, and multiply it by the preset global maximum drop rate to obtain the personalized drop probability of the node;

[0334] A sampling module 12 is used to map the personalized drop probability of the node through the edge index mechanism in the PyTorch Geometric framework to obtain a mapping edge, and perform sampling based on the mapping edge to obtain a Bernoulli mask;

[0335] The adaptive discarding module 13 is used to perform scaling based on the Bernoulli mask and the personalized discarding probability of the node to obtain a perturbed message matrix, and input the perturbed message matrix into a pre-established graph neural network model, thereby realizing adaptive discarding of the graph neural network.

[0336] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is used to implement one or more instructions, specifically for loading and executing one or more instructions in a computer storage medium to implement the above method.

[0337] It should be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium having a computer program stored thereon, which, when executed by a processor, performs the above-described method. The storage medium may be any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0338] Throughout this specification, references to terms such as "one embodiment," "example," or "specific example" indicate that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present disclosure. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0339] The above shows and describes the basic principles, main features and advantages of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited to the above embodiments. The above embodiments and descriptions are merely illustrative of the principles of the present disclosure. Various changes and improvements may be made to the present disclosure without departing from the spirit and scope of the present disclosure, and such changes and improvements shall fall within the scope of the present disclosure.

Claims

1. A graph neural network adaptive discarding method based on information entropy, characterized in that: The method comprises the following steps: Receive the node's feature vector, perform Softmax normalization on the node's feature vector, calculate the node's corresponding entropy value based on information entropy, normalize the node's corresponding entropy value, and multiply it by the preset global maximum drop rate to obtain the node's personalized drop probability; The process of performing Softmax normalization on the feature vector of the node: Assume that the node feature matrix is , among which OK Representation node The eigenvector of , uses the Softmax function to normalize the eigenvector into a probability distribution: Where, Representative Node The probability of belonging to a particular category, represents the original input logits, The exponential operation is part of the Softmax function. is the step of normalizing all categories, An index representing different categories or features; Indicates the Node No. Features, Indicates the Node No. Features The calculation formula for calculating the entropy value corresponding to the node based on information entropy is as follows: Where, is a node The information entropy of is a node After the Softmax function normalization, The probability components of the dimensions, is an index variable representing each dimension in the node feature vector; is the total number of dimensions of node features, Is to prevent constant; Normalizing the entropy values corresponding to the nodes includes: Where, is the normalized information entropy, indicating that the node The information entropy of is the normalized value. It is the maximum value of the information entropy of all nodes; The personalized drop probability of the node is defined by the following formula: for: in is the global maximum drop rate hyperparameter, which indicates the highest drop probability that the node with maximum entropy can reach; The personalized drop probability of the node is mapped through the edge index mechanism in the PyTorch Geometric framework to obtain the mapped edge, and sampling is performed based on the mapped edge to obtain the Bernoulli mask; Based on the Bernoulli mask and the personalized drop probability of the node, the perturbed message matrix is obtained and input into the pre-established graph neural network model to achieve adaptive drop of the graph neural network. The process of scaling based on the Bernoulli mask and the personalized drop probability of the node to obtain the perturbed message matrix is as follows: For each element in the message matrix , according to the source node Adaptive drop rate , performs Bernoulli sampling, where Represents a slave node To Node Message: Where, is a random variable, is a Bernoulli distribution, is the probability of success from the Bernoulli distribution, Is with the node Related constants; Then generate the perturbed message matrix , where the element calculation formula is as follows: Where, This is the disturbed news. is a node To Node The original message; described when When , the message is retained and scaled; when When , the message is completely discarded; the scaling factor Ensure that the perturbed message is consistent with the original value in expectation.

2. The graph neural network adaptive discarding method based on information entropy according to claim 1 is characterized in that: The pre-established graph neural network model is as follows: In each layer of the graph neural network, each node receives feature information from neighboring nodes, integrates neighbor information through aggregation operations, and performs nonlinear transformations based on its own features, enabling the graph neural network to achieve node classification, graph classification, and link prediction. Suppose the undirected graph is denoted by ,in Represents a node collection represents a set of edges; The node feature matrix is ,in Representation node The eigenvector of It is the characteristic dimension of the node. The edge describes the relationship between nodes using the adjacency matrix Indicates that The adjacency matrix represents OK, Representation node and If the relationship between , then it means the node and There is an edge connection, and the node degree vector is recorded as ,in Representation and Node The number of connected edges, i.e. , Compute and Node The sum of the weights of all connected edges, the total degree of the graph is given by Calculation; When applying the message passing graph neural network, the message matrix can be expressed as ,in It is the message transmitted between nodes. is the total number of messages passed in the graph, It is the dimension of the message; The specific steps of the message passing graph neural network are as follows: (1) Message generation of neighbor nodes: The characteristics of each node are generated by the message generation function The message generation formula is: Where, It is in Slave nodes in the layer To Node the message delivered; and Represents nodes respectively and nodes In the The feature vector of the layer, Indicates in Slave nodes in the layer To Node The edge, It is The message generation function of the layer; (2) Introduction and calculation of node information entropy: The information complexity of each node feature in the graph is quantitatively evaluated by information entropy; Normalize the feature vectors of the nodes to convert them into probability distributions; calculate the initial entropy value of each node based on the information entropy formula, and normalize the entropy values of all nodes to obtain normalized information entropy of a unified scale; (3) Design of adaptive discard rate: The message drop rate of each node is determined based on the normalized information entropy. The drop rate distribution is proportional to the entropy value. The global maximum drop rate is used as a hyperparameter to control the upper limit of the drop intensity. (4) Dynamic Control of Message Passing Graph Neural Networks: Training phase: Each message is randomly sampled and discarded based on the adaptive discard rate of the source node. If not discarded, it is scaled proportionally to keep the overall expected value stable; if discarded, it is directly set to zero; Testing phase: directly use the complete original message matrix without performing any discarding operation.

3. A graph neural network adaptive discarding system based on information entropy, which adopts the graph neural network adaptive discarding method based on information entropy according to any one of claims 1 to 2, characterized in that: include: The probability acquisition module is used to receive the node's feature vector, perform Softmax normalization on the node's feature vector, calculate the entropy value corresponding to the node based on the information entropy, normalize the entropy value corresponding to the node, and multiply it by the preset global maximum drop rate to obtain the node's personalized drop probability; The sampling module is used to map the personalized dropout probability of nodes through the edge index mechanism in the PyTorch Geometric framework to obtain mapped edges, and then perform sampling based on the mapped edges to obtain the Bernoulli mask; The adaptive discarding module is used to perform scaling based on the Bernoulli mask and the personalized discarding probability of the node to obtain the perturbed message matrix, and input the perturbed message matrix into the pre-established graph neural network model, thereby realizing the adaptive discarding of the graph neural network.

4. A terminal device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: The memory stores a computer program that can be run on a processor. When the processor loads and executes the computer program, it adopts a graph neural network adaptive discarding method based on information entropy as described in any one of claims 1 to 2.

Images