Robustness classification method based on graph neural network in edge environment

By using Drop Message random discarding method in edge environments, the problems of overfitting and instability in GNN training are solved, and more stable training and faster convergence are achieved, improving the robustness and information diversity of the model reserve.

CN118916742AInactive Publication Date: 2024-11-08FUZHOU UNIV

Patent Information

Application Number
CN202411002341.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

When training graph neural networks (GNNs) in edge environments, there are problems of overfitting, oversmoothing and instability, and the random discarding method is difficult to converge, the training process is unstable, and there is a lack of theoretical guarantee.

Method used

A random discarding method based on 'Drop Message' (Drop Message) is proposed. During the information transmission process, nodes, edges or messages are randomly selected for discarding, allowing the same node to spread different information to different neighbors, and the model is improved through the Drop Message random discarding mechanism generalization ability and stability.

Benefits of technology

It achieves more stable training effects, faster convergence speed and better information diversity retention, reduces sample variance, and improves the robustness and generalization ability of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916742A_ABST
    Figure CN118916742A_ABST
Patent Text Reader

Abstract

The invention provides a robustness classification method based on a graph neural network in an edge environment, which comprises the following steps: in a classification task, in each layer of graph neural network, repeating the process of aggregating and updating information of nodes and edges through a message passing mechanism until the last layer; for each node, calculating new node features by using information of neighbor nodes and edges; before each message transmission step, part of nodes, edges or messages are randomly selected according to Drop Message to be discarded; the method comprises the following specific steps of: node discarding: randomly selecting nodes in a certain proportion, and setting feature vectors of the nodes to be zero; edge discarding: randomly selecting edges with a certain proportion, and temporarily removing the connection relationship; and message discarding: in a message transmission process, randomly discarding a part of message transmission paths so as to reduce an over-fitting risk.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of edge computing and graph neural network technology, and specifically relates to a robust classification method based on graph neural network in an edge environment. Background Art

[0002] In recent years, with the vigorous development of communication technology and artificial intelligence, various new intelligent applications (such as facial recognition, AR / VR, autonomous driving, etc.) have emerged. These intelligent applications have shown characteristics such as computational intensiveness and delay sensitivity, while the limited computing power of mobile devices has restricted their further development and popularity. In order to alleviate the computing load caused by these intelligent applications on mobile devices, edge computing deploys computing resources at the edge of the network, effectively improving the service quality while greatly reducing data transmission delay.

[0003] Graphs play an important role in these intelligent applications and are also used to represent complex relationships in many fields such as social media (social networks), finance (trade networks), and biology (biological networks). First, the graph neural network contains a position attention mechanism, which can better capture the spatial dependencies of traffic nodes in urban road networks. Then, a gated recurrent neural network with Transformer is used to capture the local and global information of traffic flow sequences in the time dimension. Finally, an improved grid search optimization method is used to optimize the introduced parameters of the model and obtain the global optimal solution with high time efficiency. At the same time, with the rapid increase in the number of various services and APIs on the Internet and the Web, it is becoming more and more challenging for developers to quickly and accurately find APIs that meet their needs, so an efficient recommendation system is needed. At present, the application of graph neural networks in the field of service recommendation has achieved great success, but many such methods are still limited to simple interactions and ignore the intrinsic relationship between mashups and API calls. To solve this problem, graph neural networks have recently attracted widespread attention as a powerful tool for graph representation learning. In particular, the Graph Neural Network (GNN) adopts a message passing mode in which each node can gain information from its neighbor nodes in each convolutional layer, and has been widely used in various downstream tasks such as node classification, link prediction, vertex clustering, and recommendation systems. In terms of node classification, the influence of uncertainty in node feature representation on node classification is studied. The radial basis function is used to calculate the distance between nodes. After obtaining the centroids of various nodes, the class label of the nearest centroid to the unlabeled node is assigned according to the distance to improve the node classification performance. At the same time, the distance between the unlabeled node and the centroid is defined as the uncertainty of the model output, and the gradient penalty loss is used to enhance the detectability of input changes, which can effectively detect out-of-distribution node samples. In the recommendation system, since graph data is a special form of data consisting of nodes and edges. In this data, entities are modeled as nodes, and there may be edges between nodes to represent the relationship between entities. By analyzing and mining this data, people can obtain a lot of valuable information. Therefore, for each node in the graph, it also brings the risk of privacy information leakage.

[0004] However, despite rapid development, training GNNs in edge environments faces several challenges such as overfitting, oversmoothing, and instability. In fact, compared with other data forms, collecting labels for graph data is both laborious and inherently biased, which in turn limits the generalization performance of GNNs due to overfitting. In addition, the representations of different nodes in GNNs are often indistinguishable due to the recursive aggregation of neighbor node information, which prevents GNNs from effectively modeling high dependencies from multi-hop neighbors. The recursive aggregation pattern makes GNNs vulnerable to the quality of the input graph. In other words, noisy graphs and adversarial attacks can easily degrade the performance of GNNs.

[0005] The above problems can be solved by using random dropout methods, which integrate augmented data into the model by randomly masking part of the input data. These methods focus on randomly dropping or sampling existing information, and can also be regarded as a data augmentation technique. Benefiting from the advantages of being unbiased, adaptive, and parameter-free, random dropout methods have greatly contributed to improving the performance of most GNNs.

[0006] However, some unresolved issues related to the random dropout method for GNNs remain. First, a common and critical problem with the current random dropout method is that the augmented data introduced in GNNs makes it difficult for parameters to converge and the training process is unstable. Moreover, it is challenging to find an optimal dropout method that works for all graphs and models, because different graphs and models have their own characteristics, and using different dropout strategies can have a great impact on model performance. In addition, the answer to how to choose an appropriate dropout rate when dealing with these methods remains unclear, and so far no theoretical guarantee has been provided to explain why the random dropout method can improve the performance of GNNs. Summary of the invention

[0007] Therefore, in order to overcome the defects and shortcomings of the related art, the present invention proposes a robust classification method based on graph neural network in edge environment, based on the proposed "Drop Message" random drop method, which can be applied to all information transmission GNN. The existing random drop method performs a drop operation on the node feature matrix or adjacency matrix, while Drop Message performs a drop operation on the propagation information, allowing the same node to propagate different information to different neighbors.

[0008] The technical solution specifically adopted by the present invention to solve the technical problem is:

[0009] A robust classification method based on graph neural network in edge environment:

[0010] In the classification task, in each layer of the graph neural network, the process of aggregating and updating the information of nodes and edges through the message passing mechanism is repeated until the last layer; for each node, the new node features are calculated using the information of neighboring nodes and edges;

[0011] Before each message delivery step, some nodes, edges or messages are randomly selected and dropped according to Drop Message. The specific steps are as follows:

[0012] Node dropout: randomly select a certain proportion of nodes and set their feature vectors to zero;

[0013] Edge discarding: randomly select a certain proportion of edges and temporarily remove their connection relationships;

[0014] Message Dropping: During the message passing process, part of the message passing path is randomly dropped to reduce the risk of overfitting.

[0015] Furthermore, before training the graph neural network, the architecture parameters of the message passing graph neural network model are first initialized, including the feature dimensions of nodes and edges, the number of network layers, and the activation function; the parameters for random discarding are initialized, including the discarding rate and the discarding method; the input graph data is preprocessed, including the standardization of node features and edge features, and the sparse matrix representation of the graph structure; and it is ensured that the graph data has been converted into a suitable input format before entering the model.

[0016] Furthermore, during the training process of the graph neural network, the graph data that has been randomly discarded by Drop Message is input into the graph neural network model for training; the model parameters are updated using the back propagation algorithm to minimize the loss function; during the training process, the Drop Message random discarding mechanism is used to improve the generalization ability of the model and reduce overfitting.

[0017] Furthermore, the trained graph neural network model is evaluated on the validation set and its performance indicators are calculated; the model parameters or discard rate are adjusted according to the evaluation results to further optimize the model performance.

[0018] And, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the robust classification method based on graph neural network in the edge environment as described above are implemented.

[0019] A non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the robust classification method based on a graph neural network in an edge environment as described above.

[0020] Compared with the prior art, the present invention and its preferred solution have better and more stable training effect, faster convergence speed and can retain diversified information. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0022] Figure 1 This is a schematic diagram of the Drop Message implementation mechanism of an embodiment of the present invention;

[0023] Figure 2 This is a comparison chart of the test accuracy of different random dropout methods in the embodiments of the present invention on the GAT backbone model and the Cora dataset;

[0024] Figure 3 This is a comparison chart of the test accuracy of different random drop methods in the embodiments of the present invention on the GCN backbone model and the Cora dataset;

[0025] Figure 4 This is a comparison chart of the test accuracy of different random discarding methods in the embodiments of the present invention on the APPNP backbone model and the Cora data set;

[0026] Figure 5 A comparison chart of training losses of different random discarding methods on the GAT backbone model and the Cora dataset according to an embodiment of the present invention;

[0027] Figure 6 This is a comparison chart of the test accuracy of different random discarding methods in the embodiments of the present invention on the Citeseer dataset. DETAILED DESCRIPTION

[0028] In order to make the features and advantages of this patent more obvious and easy to understand, the following embodiments are specifically described in detail as follows:

[0029] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0030] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0031] The embodiment of the present invention considers integrating the existing dropout method into the framework proposed by the present invention, and theoretically proves that the random dropout method on GNN is equivalent to introducing an additional regularization term in its loss function, which makes the model more robust. The superiority of the proposed Drop Message is proved, and its sample variance is smaller and the training process is more stable. From the perspective of information theory, Drop Message maintains the characteristics of information diversity and can be considered as the upper limit of other random dropout methods in theory.

[0032] like Figure 1 As shown, there is no need to explicitly generate a complete information matrix, because each row in the information matrix represents a directed edge in the graph, and the Drop Message proposed in the present invention can be applied independently to each directed edge. This feature allows Drop Message to be easily parallelized, such as by edge, by node, or by batch, and applied to the message passing backbone model without increasing the time or space complexity. In addition, Drop Message provides a unified framework for the three random dropping methods, and proves at the theoretical level that random dropping in GNN is equivalent to providing an additional regularization term. From the perspective of information theory, Drop Message retains the most information diversity and is the upper bound of other random dropping methods.

[0033] The Drop Message of the embodiment of the present invention is different from the existing method in that it directly processes information rather than graphs. However, intuitively, discarding features, edges, nodes or information will eventually act on the information matrix. First, it is proved in Table 1 that Drop out, Drop Edge, Drop Node and Drop Message can all be expressed as Bernoulli sampling processes.

[0034] Table 1 summarizes different random dropout methods from the perspective of the Bernoulli sampling process.

[0035]

[0036] Considering the information propagated by the central node, Drop Message allows different information to be propagated to different neighboring nodes, and its induced information matrix can be arbitrary. The induced information matrix of other methods obeys some explicit constraints and can be regarded as a special form of Drop Message. Dropout, Drop Edge, Drop Node and DropMessage randomly mask the information matrix according to certain rules. The equivalent operation of each method on the information matrix will be provided below.

[0037] Drop out: discard the elements X in the feature matrix X drop ={X i,j |∈ i,j = 0} is equivalent to shielding the element M in the information matrix M drop ={M i,j |source(M i,j )∈X drop}, where source(M i,j ) indicates M i,j Which element in the corresponding feature matrix. (1) Randomly delete half of the hidden neurons in the network, and the input and output neurons remain unchanged. (2) Forward propagate the input x through the modified neural network, and then backpropagate the loss result through the modified network to update the parameter weights of the neural network. After a small batch of training samples complete the process, update the corresponding parameters (w, b) on the retained neurons according to the stochastic gradient descent method. (3) Continue to repeat this process: restore the deleted neurons (at this time, the deleted neurons remain the same, and the parameters of the neurons that have not been deleted have been updated).

[0038] Drop Edge: discard element E in the adjacency matrix A drop ={E i,j |A i,j =1and∈ i,j = 0} is equivalent to shielding the element M in the information matrix M drop ={M i |edge(M i )∈E drop}, where edge(M i ) indicates M iCorresponding edges. (1) The Drop-Edge model is an edge and relationship model, which is mainly used to model the edges and relationships in text data and calculate the weights between them. (2) The Drop-Edge model requires some parameters to set, including the weight of the edge to be used, the weight range of the edge, etc. (3) Implementation method: The Drop-Edge model can be implemented by using some special operations in the model layer, such as multiplying the weight of the edge by a score, and then adding these scores to get a vector representing the edge. (4) Advantages: The Drop-Edge model can process the edges and relationships in text data, and can better capture the complex relationships in text data. At the same time, the parameters of the Drop-Edge model are relatively simple, easy to implement and debug. (5) Disadvantages: The effect of the Drop-Edge model is affected by the weight of the edge. If the selected edge weight is inappropriate, the model may not perform well. In addition, the Drop-Edge model can only process the weight of the edge and cannot process the relationships in the text data. Therefore, for some text data with complex relationships, the effect may not be good.

[0039] Drop Node: Drop element V in feature matrix X drop ={X i |∈ i = 0} is equivalent to shielding the element M in the information matrix M drop ={M i |node(M i )∈V drop}, where node(M i ) indicates M i Which row in the corresponding feature matrix. (1) Function: The Drop Node model is a text modeling model that is mainly used to model text data and predict the probability of the next word or tag. (2) Parameters: The Drop Node model requires some parameters to set, including the word vector to be used, the attention mechanism, etc. (3) Implementation method: The Drop Node model can be implemented by using some special operations in the model layer, such as converting words in the text into vectors, using the attention mechanism, etc. (4) Advantages: The Drop Node model can model text data and predict the probability of the next word or tag. It has a good effect on some tasks that need to predict text context information. (5) Disadvantages: The parameters of the Drop Node model need to be set more complicated, and the effect may not be good for some complex text data. In addition, the Drop Node model can only predict the probability of the next word or tag, and cannot determine the specific content of the text.

[0040] Drop Message: Directly perform random masking on the information matrix M.

[0041] According to the above description, it is found that Drop Message performs the most fine-grained shielding on the information matrix. The other three random dropping methods drop rows or columns of the information matrix, while Drop Message randomly drops each unit of the information matrix independently, which makes it the most flexible dropping method.

[0042] An illustration of the effectiveness of the Random Dropping method

[0043] The Random Dropping method of GNN is equivalent to introducing an additional regularization term in the objective function to make the model more robust. To simplify the analysis, it is assumed that the downstream task is binary classification and a single-layer GCN is used as the backbone model, expressed as H = Where M represents the message matrix, W represents the transformation matrix, and B∈R n*k Indicates which messages each node should aggregate, is its normalized form. Using sigmoid as the nonlinear function, the result is expressed as Z = sigmoid (H). When cross entropy is used as the loss function, the objective function is expressed as

[0044]

[0045] When performing RandomDropping on a graph, use the perturbation information matrix Instead of the original information matrix M. The expected objective function can be expressed as

[0046]

[0047] As shown in the above equation, the Random Dropping method of the graph introduces an additional regularization term to the objective function. For the binary classification problem, this regularization makes the classification probability approach 0 or 1, so as to obtain a clearer judgment on whether it belongs to this category and facilitate the calculation of the loss function. The random dropping method encourages the model to extract more important high-level representations, which enhances the robustness of the model. By expanding the model output dimension, the above equation can be well generalized to multi-classification tasks. Formally, when dealing with multi-classification tasks, the final objective function can be expressed as

[0048]

[0049] Advantages of Drop Message

[0050] Drop Message has advantages over other Random Dropping methods in two aspects: stabilizing the training process and retaining diverse information. All random dropping methods face the problem of unstable training process. Existing studies have shown that it is caused by random noise in each training set. These noises increase the difficulty of parameter coverage and the instability of the training process. In general, sample variance can be used to measure the degree of stability. The input of each training set can be regarded as a random sample of the entire graph, and the sample variance is calculated by the average difference between every two independent samples. Compared with other random dropping methods, Drop Message effectively alleviates the above problems by reducing sample variance.

[0051] On a message passing GNN with the same drop rate δ, Drop Message has the smallest sample variance among existing random drop methods. Simply put, Drop Message independently determines whether an element in the information matrix is ​​masked, which means the minimum Bernoulli trajectory for randomly dropping the information matrix. By reducing the sample variance, Drop Message reduces the difference in the information matrix between different training sets, thereby stabilizing the training process and accelerating convergence. The reason why Drop Message has the smallest sample variance is that it is the most fine-grained random drop method in the GNN model. When using Drop Message, each element M of the information matrix i,j They will be independently judged whether they should be blocked.

[0052] From the perspective of information theory, the three existing random drop methods and Drop Message loss are compared in terms of information diversity. Information diversity includes feature diversity and topological diversity. Feature diversity is defined as

[0053]

[0054] Topological diversity is defined as TD G =card({||M j ||0≥1}), where j∈[0,k). M∈R k*c represents the information matrix, ||·||0 computes the zero norm of the input vector, and card(·) computes the number of elements in a set. In other words, feature diversity is defined as the total number of retained feature dimensions from different source nodes; topological diversity is defined as the total number of directed edges that propagate at least one dimension of information. According to the definitions of feature diversity and topological diversity, the method has the ability to maintain information diversity only when both feature diversity and topological diversity are not reduced after random dropout.

[0055] when When , Drop Message can maintain the desired information diversity, where δ i is node v i The discard rate, d i Yes i In order to maintain the diversity of topological information, the Drop Message method hopes that at least one element in each row of the information matrix M can be retained in the expected value:

[0056]

[0057] In order to maintain the diversity of feature information, it is hoped that for each element in the feature matrix X, at least one element in the corresponding information matrix M is retained in the expected value:

[0058]

[0059] In order to maintain information diversity, the loss rate δ i Formula (4-2) and formula (4-3) should be satisfied:

[0060]

[0061] From the perspective of information theory, the random drop method with the ability to maintain information diversity can save more information and theoretically perform better than the drop method without this ability. Therefore, this can also explain why the proposed method performs better than the existing drop method. In fact, only a drop rate δ can be set for the entire graph instead of each node. Therefore, both Drop Message and other methods may lose some information. But even in this case, Drop Message still saves more information than other methods with the same drop rate. It turns out that in practical applications, DropMessage still has advantages.

[0062] Method evaluation:

[0063] The effectiveness and adaptability of the proposed Drop Message will be verified through empirical research. In particular, the following questions are explored: (1) Is Drop Message superior to other random drop methods on GNNs? (2) Can Drop Message further improve the robustness and training efficiency of GNNs? (3) Is information diversity important in GNNs?

[0064] Datasets: Five public graph datasets were used in the experiments, including Cora, CiteSeer, PubMed, ogbn-arxiv, and Flickr. Cora, CiteSeer, PubMed, and ogbn-arxiv: These four different citation networks are widely used as graph benchmarks. Flickr: It is provided by the largest photo sharing website Flickr. A node in the graph represents a picture uploaded to Flickr. If two pictures have certain common properties (such as the same geographic location, the same gallery, or comments from the same user), an edge will appear between the nodes of these two pictures. The purpose of the node classification task is to classify these pictures into 7 categories based on their labels.

[0065] Baseline Methods: The proposed Drop Message random dropout method is compared with other existing random dropout methods, including Dropout, Drop Edge, and Drop Node. These dropout methods are adopted as backbone models on various GNNs and their performance on different datasets is compared.

[0066] Backbone network model: Three mainstream GNNs are used as backbone models: GCN, GAT, and APPNP. The formal practices of these methods are borrowed, while some minor modifications are made. All these backbone models have random dropout modules in different steps of the model implementation. The GAT model performs random dropout after the self-attention calculation, while the APPNP model performs random dropout at the beginning of each iteration. For fair comparison, the implementation of random dropout modules in the same step of different backbone network models is unified. Dropout, Drop Edge, and Drop Node are fixed on the initial input, and Drop Message is fixed at the starting point of the information propagation process.

[0067] Table 2 Experimental results

[0068]

[0069]

[0070] As shown in Table 2, the random drop method outperforms the GNN without random drop most of the time in terms of node classification and link prediction. In addition, the effect of the random drop method varies depending on the dataset, backbone network model, and downstream tasks. For example, the random drop method on APPAP improves the average accuracy by 1.5% on CiteSeer and 0.2% on PubMed. At the same time, the random drop method improves the accuracy of GCN on Cora by 2.6% and 4.4% on GAT.

[0071] The proposed Drop Message works well in most scenarios, showing its strong adaptability to various scenarios. Overall, there are 9 settings under the node classification task, and each setting is a combination of different backbone network models and datasets. The results show that Drop Message achieved the best results in 3 settings, and suboptimal results in the remaining 4 settings. The test accuracy is very close to the optimal test accuracy. In addition, compared with the three existing random drop methods, DropMessage's stable performance on all datasets is very impressive. Due to the use of the most fine-grained drop strategy, DropMessage obtains a smaller inductive bias. Therefore, compared with other methods, Drop Message is more suitable for most scenarios, such as Figure 2-Figure 4 shown.

[0072] Next, we experimentally analyze the loss during training when using different random dropout methods. Figure 5 The figure shows the change in loss during the training of the GCN backbone network using different random drop methods on CiteSeer. Similar training loss curves can be obtained under other experimental settings. The experimental results show that as the number of training rounds increases, the training loss of all random drop methods decreases. Among all methods, the training loss of Drop Node is the largest, while the training loss of Drop Message gradually becomes the smallest, and the sample variance of Drop Message is the smallest, so the convergence speed is the fastest and the performance is the most stable. This is consistent with the theoretical results in this section.

[0073] Oversmoothing is a common problem in GNNs, which means that as the network depth increases, node representations become indistinguishable. We will evaluate the impact of various random dropout methods on this problem, and measure the degree of oversmoothing by the change in test accuracy. It should be noted that smaller values ​​indicate more indistinguishable node representations, and vice versa. The experiments are conducted on CiteSeer with GCN as the backbone model. Figure 6 The relative growth of the test accuracy of the final node representation compared to the original model without any random dropout technique is shown. The results show that all these random dropout methods can alleviate the over-smoothing phenomenon by increasing the test accuracy when the model depth increases. Among all the random dropout methods, the proposed DropMessage performs well in terms of consistency. When the number of layers is ≥ 3, it improves the test accuracy by 0.2 on average compared to other random dropout methods. The reason for this result is that Drop Message can generate more different information than other methods, which to some extent prevents nodes from converging to the same representation.

[0074] Experiments are conducted to evaluate the importance of information diversity to the message passing GNN model. CiteSeer is set as the experimental dataset, which contains 3327 nodes and 4732 edges. The average node degree in CiteSeer is close to 3. According to the formula The upper limit of the drop rate is calculated based on the node degree and feature dimension. The feature dimension of CiteSeer is 3703, which is much larger than the node degree. Therefore, the upper limit depends only on the node degree. In the backbone node setting, the packet loss rate of each node is set to be less than its upper limit δ i =0.66<0.667. In the backbone average setting, set the drop rate δ i =0.66+∈ i , where ∈ i ~Uniform(-0.15, 0.15). Both settings use Drop Message. The average random drop rate of all nodes is almost the same in both settings, but only the former can maintain the expected information diversity. The following table lists the results. Regardless of which backbone model is selected, the backbone node average setting is better than the backbone average setting.

[0075] The present invention proposes Drop Message, a general random dropping method for the message passing GNN model. In this embodiment, all random dropping methods are unified into the overall algorithm framework by first dropping the message matrix and analyzing its effect. Then, the superiority of Drop Message in stabilizing the training process and maintaining information diversity is theoretically verified. Because the information matrix is ​​dropped at a fine-grained level, Drop Message shows greater applicability in most cases. The effectiveness and versatility of the proposed method are demonstrated by conducting multi-task experiments on three public datasets.

[0076] Based on the above design and verification of the solution of this embodiment, the robust classification method based on graph neural network in the edge environment is constructed as follows:

[0077] (1) Initializing the model and data preprocessing

[0078] Initialize the architecture parameters of the message passing GNN model, including the feature dimensions of nodes and edges, the number of network layers, activation functions, etc. At the same time, initialize the parameters for random dropout, including the dropout rate and the dropout method (node, edge, or message). Preprocess the input graph data, including the standardization of node and edge features, the sparse matrix representation of the graph structure, etc. Ensure that the graph data has been converted to a suitable input format before entering the model.

[0079] (2) Application random discard

[0080] Before each message delivery step, some nodes, edges or messages are randomly selected and dropped according to the proposed Drop Message. The specific steps are as follows:

[0081] Node dropout: Randomly select a certain proportion of nodes and set their feature vectors to zero.

[0082] Edge discarding: Randomly select a certain proportion of edges and temporarily remove their connections.

[0083] Message Dropping: During the message passing process, part of the message passing path is randomly dropped to reduce the risk of overfitting.

[0084] (3) Message passing and updates

[0085] In each layer of GNN, the information of nodes and edges is aggregated and updated through the message passing mechanism. For each node, the new node features are calculated using the information of neighboring nodes and edges. This process is repeated until the last layer.

[0086] (4) Model training and evaluation

[0087] Input the graph data that has been randomly discarded into the GNN model for training. Use the back propagation algorithm to update the model parameters and minimize the loss function. During the training process, the random discard mechanism is used to improve the generalization ability of the model and reduce overfitting. Evaluate the trained GNN model on the validation set and calculate its performance indicators. Adjust the model parameters or discard rate based on the evaluation results to further optimize the model performance.

[0088] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0089] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0090] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0092] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0093] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

[0094] This patent is not limited to the above-mentioned optimal implementation mode. Anyone can derive various other forms of a robust classification method based on graph neural network in an edge environment under the inspiration of this patent. All equal changes and modifications made according to the scope of the patent application of the present invention should be covered by this patent.

Claims

1. A robust classification method based on graph neural network in edge environment, characterized by: In the classification task, in each layer of the graph neural network, the process of aggregating and updating the information of nodes and edges through the message passing mechanism is repeated until the last layer; for each node, the new node features are calculated using the information of neighboring nodes and edges; Before each message delivery step, some nodes, edges or messages are randomly selected and dropped according to Drop Message. The specific steps are as follows: Node dropout: randomly select a certain proportion of nodes and set their feature vectors to zero; Edge discarding: randomly select a certain proportion of edges and temporarily remove their connection relationships; Message Dropping: During the message passing process, part of the message passing path is randomly dropped to reduce the risk of overfitting.

2. The robust classification method based on graph neural network in edge environment according to claim 1 is characterized in that: Before training the graph neural network, first initialize the architecture parameters of the message passing graph neural network model, including the feature dimensions of nodes and edges, the number of network layers, and the activation function; initialize the parameters for random discarding, including the discarding rate and the discarding method; preprocess the input graph data, including the standardization of node features and edge features, and the sparse matrix representation of the graph structure; ensure that the graph data has been converted into a suitable input format before entering the model.

3. The robust classification method based on graph neural network in edge environment according to claim 1, characterized in that: During the training process of the graph neural network, the graph data that has been randomly discarded by Drop Message is input into the graph neural network model for training; the model parameters are updated using the back propagation algorithm to minimize the loss function; during the training process, the Drop Message random discarding mechanism is used to improve the generalization ability of the model and reduce overfitting.

4. The robust classification method based on graph neural network in edge environment according to claim 1, characterized in that: Evaluate the trained graph neural network model on the validation set and calculate its performance indicators; adjust the model parameters or discard rate based on the evaluation results to further optimize the model performance.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the steps of the robust classification method based on graph neural network in an edge environment as described in any one of claims 1-4.

6. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the robust classification method based on graph neural network in an edge environment as described in any one of claims 1 to 4 are implemented.

Citation Information

Patent Citations

  • Graph neural network method, device and equipment capable of keeping similarity transformation invariance

    CN114239802A

  • Particle orbit discovery method based on graph neural network cellular automaton

    CN115563841A

Cited By

  • Graph neural network self-adaptive discarding method and system based on information entropy

    CN120337986A