A graph neural network architecture optimization method based on syntax genetic programming

Abstract

The application relates to the technical field of graph neural networks, in particular to a graph neural network architecture optimization method based on syntax genetic programming, which comprises the following steps: S1, constructing a syntax rule library of a graph neural network architecture GNN; S2, generating an initial GNN architecture population based on the syntax rule library through genetic programming; S3, predicting the performance of individual GNN architectures in the population by using a Gaussian process surrogate model; S4, searching for an optimal GNN architecture in a syntax tree space through a genetic algorithm; S5, for individual GNN architectures in the population, calculating a training probability based on an expected improvement function, if the training probability is higher than a preset evaluation threshold, selecting the individual GNN architecture for actual training evaluation to obtain a real performance, and updating the Gaussian process surrogate model by using the real performance; S6, judging whether a preset optimization termination condition is met, if yes, outputting an optimized GNN architecture, and if not, returning to step S3. The application can improve the automation degree, efficiency and model performance of GNN model architecture search.

Description

Technical Field

[0001] This invention relates to the field of graph neural network technology, and more specifically to a graph neural network architecture optimization method based on syntactic genetic programming. Background Technology

[0002] Graph Neural Networks (GNNs), as an emerging deep learning model, have demonstrated powerful capabilities in processing graph-structured data. By effectively combining graph computation with neural networks, GNNs have achieved remarkable results in numerous fields such as node classification, graph classification, link prediction, recommender systems, and risk control.

[0003] However, the practical application and widespread deployment of GNNs still face many challenges. First, designing high-performance GNN architectures is a time-consuming process reliant on expert experience, especially in fields requiring the handling of complex relationships and sophisticated patterns, such as predicting molecular activity in drug discovery, analyzing protein-protein interaction networks in bioinformatics, or identifying complex fraud rings in financial risk control. Traditional manual design methods require researchers to repeatedly debug and optimize the network structure and hyperparameters according to specific problems, which is inefficient and makes it difficult to guarantee finding the optimal architecture. Second, the hyperparameter optimization process for GNN architectures is complex. Manually adjusting hyperparameters is not only time-consuming but also prone to getting trapped in local optima, making it difficult to fully exploit the performance potential of GNNs. Furthermore, existing automated graph machine learning techniques generally lack knowledge-driven guidance in the GNN architecture search process, resulting in a large search space, excessively long search time, low optimization efficiency, and a tendency to overfit to specific datasets, leading to insufficient model generalization ability. This is unacceptable in critical applications requiring high-precision prediction and reliability, such as drug development, precision medicine, and financial security.

[0004] In recent years, Neural Architecture Search (NAS) technology has provided a new approach for the automated design of neural network architectures. Several NAS methods have also emerged in the field of graph neural networks. However, these methods still have limitations in search efficiency and generalization ability. For example, reinforcement learning-based NAS methods suffer from unstable training processes and low search efficiency; while evolutionary algorithm-based NAS methods can find well-performing architectures, they lack effective guidance for the search process, are prone to blind searching, and have high computational costs.

[0005] Therefore, there is an urgent need for a method that can efficiently, automatically, and knowledge-drivenally optimize GNN architectures to reduce reliance on expert experience, shorten the development cycle, and improve the performance and generalization ability of GNN models. Summary of the Invention

[0006] In view of this, the present invention provides a graph neural network architecture optimization method based on syntactic genetic programming, which can solve the problems of GNN architecture design relying on expert experience, low optimization efficiency, and insufficient generalization ability in the prior art. Especially in complex application scenarios that require high-performance graph neural network models, such as drug discovery, bioinformatics, and financial risk control, it can improve the automation, efficiency, and model performance of GNN model architecture search.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] A graph neural network architecture optimization method based on syntactic genetic programming includes the following steps:

[0009] S1. Construct the syntax rule base for the graph neural network architecture GNN;

[0010] S2. Based on the syntax rule base, an initial GNN architecture population is generated through genetic programming. The GNN architecture of each individual in the population is represented as a syntax tree. The syntax tree is recursively generated according to the production rules, and the network structure and hyperparameters of each layer of the network are encoded synchronously.

[0011] S3. Predict the performance of individual GNN architectures in the population using a Gaussian process surrogate model;

[0012] S4. Based on the prediction performance of individual GNN architectures, the optimal GNN architecture is searched in the syntax tree space using a genetic algorithm.

[0013] S5. For an individual GNN architecture in the population, calculate the training probability P based on its expected improvement function EI(x). train (x), if the training probability P train If (x) is higher than the preset evaluation threshold, then the individual GNN architecture is selected for actual training and evaluation to obtain its true performance, and the true performance is used to update the Gaussian process proxy model.

[0014] S6. Determine whether the preset optimization termination condition is met. If yes, output the optimized GNN architecture; otherwise, return to step S3. The optimized GNN architecture is used to process graph structure data to perform classification tasks.

[0015] Furthermore, in S1, the syntax rule base is defined based on the BNF normal form and includes the set of nonterminal symbols N, the set of terminal symbols T, the set of production rules P, and the start symbol S;

[0016] The set of nonterminal symbols N contains scalable network modules and is divided into a set of hyperparameter nonterminal symbols N. H and the set of module nonterminal symbols N M ;

[0017] The set of terminal symbols T contains specific network operators and hyperparameter values, and is divided into the set of hyperparameter terminal symbols T. H and the set of module terminators T M ;

[0018] The set of production rules P defines the derivation relations from non-terminal symbols to terminal symbols and / or non-terminal symbols;

[0019] The starting symbol S represents the initial derivation symbol for the network architecture.

[0020] Furthermore, each production rule p∈P in the production rule set P is of the form l→r, where l∈N is the derived symbol, and r=r H r M To derive the result, r H ∈(N H ∪T H * indicates the hyperparameter part, r M ∈(N M ∪T M ) * This indicates the submodule section. For the set of hyperparameter nonterminal symbols, For the set of hyperparameter terminal symbols, For the set of nonterminal symbols of a module, This is the set of module terminators.

[0021] Furthermore, in S2, the syntax tree generation process includes:

[0022] Initialization: Create an initial syntax tree with the start symbol S as the root node;

[0023] Recursive expansion: For each non-terminal leaf node l in the current syntax tree k ∈N∪S, from the production rule subset P(l k )={p:l k →r k A production rule is randomly selected from {p∈P} for expansion, where r k The right-hand side of the production rule is a sequence of hyperparameter symbols and module symbols;

[0024] Termination condition: Repeat the recursive expansion steps until all leaf nodes of the syntax tree are terminal symbols.

[0025] Furthermore, S3 includes:

[0026] S31. Define the Gaussian process surrogate model based on the following formula:

[0027] f~GP(m(x),k(x,x′))

[0028] Where f represents the fitness function, m(x) represents the mean function, and k(x,x') represents the covariance function. The mean function is calculated based on the following formula:

[0029]

[0030] in, denoted by ; f(x) represents the expected value; f(x) represents the stochastic function in the Gaussian process surrogate model, used to approximate the true fitness function;

[0031] The covariance function k(x,x') is calculated based on the tree kernel function defined below:

[0032]

[0033] Among them, D Tree (x,x') represents the tree edit distance between syntax trees x and x', and σ is the kernel width parameter;

[0034] S32. Train the Gaussian process surrogate model based on the evaluated training data (X,y), where X is the set of evaluated GNN architectures and y is the corresponding real performance.

[0035] S33. Predict new, unevaluated GNN architectures X using a trained Gaussian process surrogate model. * fitness function f * The distribution of the results follows a normal distribution:

[0036]

[0037] Where, μ * To predict the mean; σ * The standard deviation of the forecast represents the uncertainty of the forecast.

[0038] Furthermore, in S4, the genetic algorithm includes selection, crossover, and mutation operations;

[0039] The selection operation adopts the tournament selection method, in which a certain number of individuals are randomly selected from the population for comparison each time, and the individual with the best prediction performance is selected as the parent. The selection operation is repeated until the number of parent individuals meets the requirements for crossover and mutation.

[0040] The crossover operation includes: randomly selecting a non-terminal symbol node from each of the two parent individual syntax trees, ensuring that the non-terminal symbol types of the two nodes are the same; swapping the subtrees rooted at the two selected nodes to generate two new child individual syntax trees;

[0041] The mutation operation includes: randomly selecting a non-terminal symbol node in the individual syntax tree; randomly generating a new subtree based on the production rules in the syntax rule base that take the non-terminal symbol as the left; and replacing the selected node with the newly generated subtree to complete the mutation operation.

[0042] Furthermore, in S5, the expected improvement function EI(x) is used to evaluate the potential improvement value of an individual GNN architecture, and its calculation result is used to determine the training probability P. train (x) is also used to guide the search direction of the genetic algorithm or to select from multiple individuals that meet the evaluation conditions; wherein, guiding the search direction of the genetic algorithm includes: giving priority to individuals with higher improvement value in the selection operation of the genetic algorithm, or selecting the individual with the highest improvement value from multiple individual GNN architectures that meet the actual training and evaluation conditions for actual training and evaluation.

[0043] The expected improvement function EI(x) is defined based on the following formula:

[0044]

[0045] Where Imp represents the improvement amount, Z represents the standardized improvement amount, and Imp = μ(x) - y best -ξ, μ(x) and σ(x) are the mean and standard deviation of the Gaussian process surrogate model predictions, respectively. best Let ξ be the optimal performance among the currently evaluated GNN architectures, Φ be the exploration factor, and φ be the finite element. These are the cumulative distribution function and probability density function of the standard normal distribution, respectively.

[0046] Furthermore, S5 also includes:

[0047] Calculate the prediction probability P predict (x) and training probability P train (x), and based on the training probability P train (x) Decide whether to perform actual training and evaluation on the individual GNN architecture, where the prediction probability P predict (x) and training probability P train The formula for calculating (x) is:

[0048] P predict (x)=max(P0,min(1,EI(x)))

[0049] P train (x)=1-P predict (x)

[0050] Where P0 is the lower bound of the prediction probability, if the training probability P trainIf (x) is higher than the preset value, then the individual GNN architecture x is actually trained and evaluated to obtain its true performance y on the validation set, and the Gaussian process agent model is updated with new data points (x,y), that is, the new data points are added to the evaluated dataset and the Gaussian process agent model is retrained.

[0051] Furthermore, in S6, the optimization termination condition includes at least one of the following:

[0052] The genetic algorithm has reached its preset maximum number of iterations.

[0053] The predictive performance of the population-optimal individual GNN architecture improves by less than a preset threshold within a preset number of consecutive generations;

[0054] The preset computing resource limit has been reached.

[0055] Furthermore, in S6, the optimized GNN architecture is used for node classification in citation networks or for graph classification of chemical compounds to predict molecular properties. In the node classification task in citation networks, the input is the graph structure data of the citation network, where nodes represent documents and edges represent citation relationships, and the output is a category label for each node. In the graph classification task of chemical compounds, the input is a set of graph structure data representing chemical compounds, where nodes represent atoms and edges represent chemical bonds, and the output is a category label for each graph (e.g., indicating whether the compound has specific biological activity or toxicity properties).

[0056] As can be seen from the above technical solution, compared with the prior art, the present invention has the following beneficial effects:

[0057] 1. This invention realizes the structured representation and automatic generation of GNN architecture by constructing a grammar rule base, reducing the dependence on expert experience and improving the automation level of GNN architecture design.

[0058] 2. This invention introduces a Gaussian process surrogate model for GNN architecture performance prediction, avoiding the expensive actual training and evaluation of each candidate architecture, significantly improving architecture search efficiency and shortening optimization time.

[0059] 3. This invention effectively guides the genetic algorithm to perform efficient searches in the GNN architecture space by exploring and utilizing the expected improvement (EI) function balance search process, realizing knowledge-driven efficient GNN architecture optimization, accelerating the discovery of the optimal GNN architecture, and improving the quality of search results. This enables the optimized GNN model to exhibit superior performance in key tasks such as chemical molecule property prediction (e.g., toxicity or activity prediction in figure classification tasks) and biological network analysis (e.g., protein function prediction in node classification tasks).

[0060] 4. This invention accelerates evaluation through automated search and proxy models, significantly shortening the development cycle of customizing and deploying high-performance GNN models for specific complex applications (such as new drug screening, precision medicine analysis, and dynamic financial risk control), effectively reducing dependence on computing resources and expert experience, and improving the model's generalization ability and application value on real-world complex graph data. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0062] Figure 1 A flowchart illustrating the graph neural network architecture optimization method based on syntactic genetic programming provided by this invention;

[0063] Figure 2 A flowchart for evaluating the fitness of an individual GNN architecture based on a Gaussian process surrogate model provided by this invention;

[0064] Figure 3 A diagram illustrating the syntax rules used in the search space of the GNN architecture provided by this invention. Detailed Implementation

[0065] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] like Figure 1 As shown in the figure, this invention discloses a graph neural network architecture optimization method based on syntactic genetic programming, characterized by the following steps:

[0067] S1. Construct the syntax rule base for the graph neural network architecture GNN;

[0068] S2. Based on the syntax rule base, an initial GNN architecture population is generated through genetic programming. The GNN architecture of each individual in the population is represented as a syntax tree. The syntax tree is recursively generated according to the production rules, and the network structure and hyperparameters of each layer of the network are encoded synchronously.

[0069] S3. Predict the performance of individual GNN architectures in the population using a Gaussian process surrogate model;

[0070] S4. Based on the prediction performance of individual GNN architectures, the optimal GNN architecture is searched in the syntax tree space using a genetic algorithm.

[0071] S5. For an individual GNN architecture in the population, calculate the training probability P based on its expected improvement function EI(x). train (x), if the training probability P train If (x) is higher than the preset evaluation threshold, then the individual GNN architecture is selected for actual training and evaluation to obtain its true performance, and the Gaussian process surrogate model is updated using the true performance; whereby the expected improvement function EI(x) is used to calculate P train In addition to (x), it can also help guide the search direction of the genetic algorithm or select from multiple individuals that meet the evaluation criteria; the relative uncertainty index P train (x) comprehensively reflects the predictive performance improvement potential and predictive uncertainty of architecture x, and is calculated by the Gaussian process model based on the evaluated data and the expected improvement function EI(x).

[0072] S6. Determine whether the preset optimization termination condition is met. If yes, output the optimized GNN architecture; otherwise, return to step S3. The optimized GNN architecture is used for graph data classification tasks.

[0073] The core idea of ​​this invention is to construct a GNN architecture syntax rule base, automatically generate the syntax tree representation of the GNN network architecture using Grammar-based Genetic Programming (GBGP), and simultaneously encode the network structure and hyperparameters. To improve search efficiency, this invention introduces a Gaussian Process (GP) surrogate model to predict the performance of the GNN architecture, and combines it with a genetic algorithm to efficiently search for the optimal architecture in the syntax tree space. A knowledge-driven, efficient GNN architecture optimization is achieved by balancing exploration and utilization in the search process through an Expected Improvement (EI) function.

[0074] The following is a further explanation of each of the above steps.

[0075] S1 constructs the grammar rule library for the graph neural network architecture GNN.

[0076] The syntax rule base is defined based on the BNF normal form and includes a set of nonterminal symbols N, a set of terminal symbols T, a set of production rules P, and a start symbol S, which is formally represented as a quadruple G = (N, T, P, S).

[0077] The set of nonterminal symbols N contains scalable network modules, such as... <net>(Network module) <gconv>(Graph Convolution Module) <residual>(Residual Module) <attention>(Attention module) <fc>(fully connected layer) <conv>(convolutional layer) <gat>(Attention layer in diagram) <act>(Activation function), etc. The set of non-terminal symbols is further divided into the set of hyperparameter non-terminal symbols N. H (For example <dim> 、、 <k>and the set of module nonterminal symbols N M (For example <net> 、 <gconv> 、 <fc>).

[0078] The set of terminal symbols T contains specific network operators and hyperparameter values, such as GCNConvLayer, ChebConvLayer, GATConvLayer, ReLU, Sigmoid, Tanh, and specific values ​​of the hyperparameters (e.g., 16, 32, 64, 128, 0.4, 0.5, 0.6, 1, 2, 4, 1, 2, 3). The set of terminal symbols is further divided into the set of hyperparameter terminal symbols T. H (e.g., 16,32,64,128,1,2,4,1,2,3,0.4,0.5,0.6) and the set of module terminators T M (e.g., GCNConvLayer, Relu).

[0079] The set of production rules P defines the derivation relations from non-terminal symbols to terminal symbols and / or non-terminal symbols; each production rule p∈P in the set of production rules P has the form l→r, where l∈N is the derivation symbol, and r=r H r M To derive the result, r H ∈(N H ∪T H ) * This refers to the hyperparameter part, r M ∈(N M ∪T M ) * This indicates the submodule section. For the set of hyperparameter nonterminal symbols, For the set of hyperparameter terminal symbols, For the set of nonterminal symbols of a module, This is the set of module terminators. For example, the production rules for graph convolution modules could be... <gconv>\rightarrow <dim> <conv> <fc>This indicates that a graph convolutional module consists of hidden layer dimensions. <dim>Convolutional layer <conv>and fully connected layer <fc>composition.

[0080] The starting symbol S represents the initial derivation symbol for the network architecture, for example... <net>, represents the initial derivation symbol of the network architecture.

[0081] The syntax rule base constructed by this invention has good flexibility and scalability. Users can easily add new network modules (non-terminal symbols), specific operations (terminal symbols), or modify production rules based on prior knowledge or specific application needs, so as to guide the search space to tilt in a more expected direction, and further combine expert experience with automated search.

[0082] S2. Based on a syntax rule base, an initial GNN architecture population is generated through genetic programming. Each individual's GNN architecture in the population is represented as a syntax tree. The syntax trees are recursively generated according to production rules, and the network structure and hyperparameters of each layer are encoded synchronously. The syntax tree generation process includes representing each individual's GNN architecture in the population as a syntax tree. The syntax tree generation process includes:

[0083] Initialization: Create an initial syntax tree with the start symbol S as the root node;

[0084] Recursive expansion: For each non-terminal leaf node l in the current syntax tree k ∈N∪S, from the production rule subset P(l k )={p:l k →r k A production rule is randomly selected from {p∈P} for expansion, where r k The right side of the production rule is shown in the reference. Figure 3 It is a sequence composed of hyperparameter symbols and module symbols. For example, if the current node is <gconv>Then from P(· <gconv>Randomly select a rule from ) for example <gconv>\rightarrow <dim> <conv> <fc>,Will <gconv>Node expansion to <dim> 、 <conv>and <fc>The subtree of the child node.

[0085] Termination condition: Repeat the recursive expansion steps until all leaf nodes of the syntax tree are terminal symbols.

[0086] Through the above process, each individual GNN architecture is encoded into a syntax tree, which not only describes the structure of the network, but also contains hyperparameter information of each layer of the network.

[0087] S3. Predict the performance of individual GNN architectures in the population using a Gaussian process surrogate model. The specific process is as follows: Figure 2 As shown, it includes:

[0088] S31. To avoid the huge computational overhead of actually training and evaluating each GNN architecture, this invention introduces a Gaussian process surrogate model to predict the performance (fitness) of the GNN architecture. The Gaussian process (GP) surrogate model is defined based on formula (1):

[0089] f~GP(m(x),k(x,x')) (1);

[0090] Where f represents the fitness function (e.g., the classification accuracy of the GNN architecture on the validation set), m(x) represents the mean function, and k(x,x') represents the covariance function, used to measure the similarity between two GNN architectures x and x'. The mean function is calculated based on the following formula:

[0091]

[0092] in, This represents the expected value. Let f(x) be the expected value of the fitness function; f(x) represents a random function in the Gaussian process surrogate model used to approximate the true fitness function, such as the classification accuracy of the GNN architecture on the validation set.

[0093] The covariance function k(x,x') is calculated based on the tree kernel function defined below:

[0094]

[0095] Among them, D Tree (x, x') represents the tree edit distance between syntax trees x and x', where σ is the kernel width parameter, controlling the width of the radial basis functions of the kernel. Tree edit distance D Tree The calculation of (x,x') is based on the minimum cost of editing operations, including node insertion, node deletion, and node replacement, and is obtained using existing tree edit distance algorithms (such as the ZSS algorithm).

[0096] S32. Train the Gaussian process surrogate model based on the evaluated training data (X,y), where X is the set of evaluated GNN architectures and y is the corresponding real performance (validation set accuracy).

[0097] S33. Predict new, unevaluated GNN architectures X using a trained Gaussian process surrogate model. * fitness function f * The distribution of the results follows a normal distribution:

[0098]

[0099] Where, μ * To predict the mean; σ * The standard deviation of the forecast represents the uncertainty of the forecast.

[0100] Using the covariance matrix ∑ of the training dataset, the covariance matrix ∑ between the training set and the test set. * And the autocovariance matrix ∑ of the test set ** To improve computational stability, the covariance matrix ∑ is decomposed using the Cholesky decomposition shown in formula (5):

[0101] ∑=LL T (5);

[0102] Here, L represents the lower triangular matrix obtained by the Cholesky decomposition of the covariance matrix ∑. ∑ is the covariance matrix of the training set in the Gaussian process model, and the lower triangular property of L ensures the uniqueness and computational stability of the decomposition.

[0103] Predicted mean μ * The variance represents the performance value of the prediction. This reflects the uncertainty in the forecast. Among them, the forecast mean μ * and prediction variance It is calculated using the following formula:

[0104] μ * =∑ * L -T L -1 y (6);

[0105]

[0106] Based on a small number of evaluated GNN architectures and their real performance data, a Gaussian process surrogate model is trained. For unevaluated individual GNN architectures in the population, the trained Gaussian process surrogate model is used to predict their performance, and the predicted mean μ(x) and predicted variance σ are output. 2 (x). Prediction variance σ 2 (x) characterizes the uncertainty of performance prediction; the larger the variance, the higher the uncertainty.

[0107] S4. Prediction performance based on individual GNN architectures: The optimal GNN architecture is searched in the syntax tree space using a genetic algorithm. The genetic algorithm includes selection, crossover, and mutation operations.

[0108] The selection operation adopts the tournament selection method. Each time, a certain number of individuals (e.g., 3) are randomly selected from the population for comparison. The individual with the best prediction performance (mean μ(x)) is selected as the parent. The selection operation is repeated until the number of parent individuals meets the requirements for crossover and mutation.

[0109] Crossover operations include subtree crossover. A non-terminal symbol node is randomly selected from each of the two parent individual syntax trees, ensuring that the two nodes have the same non-terminal symbol type (e.g., both are 0). <conv>Swap the subtrees rooted at the two selected nodes to generate two new child syntax trees.

[0110] The mutation operation includes: randomly selecting a non-terminal symbol node in the individual syntax tree; randomly generating a new subtree based on the production rules in the syntax rule base that take the non-terminal symbol as the left; and replacing the selected node with the newly generated subtree to complete the mutation operation.

[0111] S5. For an individual GNN architecture in the population, calculate the training probability P based on its expected improvement function EI(x). train (x); if the training probability P train If (x) exceeds the preset evaluation threshold, then the individual GNN architecture is selected for actual training and evaluation to obtain its true performance, and this true performance is used to update the Gaussian process surrogate model; where the expected improvement function EI(x) is used to calculate P train In addition to (x), it can also help guide the search direction of the genetic algorithm or select from multiple individuals that meet the evaluation criteria;

[0112] Regarding the determination of the preset evaluation threshold, through cross-validation or preliminary experiments, 0.5 or 0.6 is a reasonable initial trial value and a moderate threshold.

[0113] Specifically, to balance exploration and utilization in the search process, this invention employs an expected improvement (EI) function to assist in guiding the genetic algorithm's search direction or selecting from multiple individuals that meet the evaluation criteria. The expected improvement function EI(x) is defined based on the following formula:

[0114]

[0115] Where EI(x) represents Expected Improvement, used to evaluate the improvement at point x.

[0116] The performance improvement that sampling may bring is represented by Imp, where Imp is the improvement and Z is the improvement due to normalization. Imp = μ(x) - y best -ξ, μ(x) and σ(x) are the mean and standard deviation of the Gaussian process surrogate model predictions, respectively. best The optimal performance among the currently evaluated GNN architectures is represented by ξ, which is the exploration factor (used to balance exploration and exploitation, e.g., ξ = 0), and Φ and These are the cumulative distribution function and probability density function of the standard normal distribution, respectively. The value calculated by EI(x) represents the magnitude of the expected performance improvement; a larger value indicates that sampling at that point may bring a greater performance boost. In genetic algorithms, EI(x) is used to guide the search direction: by prioritizing individuals with higher EI(x) for crossover and mutation, regions more likely to improve performance are explored.

[0117] Calculate the prediction probability P predict (x) and training probability P train (x), and based on the training probability P train (x) Decide whether to perform actual training and evaluation on the individual GNN architecture, where the prediction probability P predict (x) and training probability P train The formula for calculating (x) is:

[0118] P predict (x)=max(P0,min(1,EI(x))) (9)

[0119] P train (x)=1-P predict (x) (10)

[0120] Where P0 is the lower bound of the predicted probability (e.g., P0 = 0.1), if the training probability P train If (x) is higher than the preset value (e.g., 0.5), then the individual GNN architecture x is actually trained and evaluated to obtain its true performance y on the validation set, and the Gaussian process surrogate model is updated with new data points (x,y), that is, the new data points are added to the evaluated dataset and the Gaussian process surrogate model is retrained.

[0121] S6. Determine whether the preset optimization termination condition is met. If yes, output the optimized GNN architecture (i.e., the GNN architecture with the highest prediction performance) as the optimization result. Otherwise, return to step S3 and proceed with the next generation of genetic evolution.

[0122] The optimization termination conditions include at least one of the following:

[0123] The genetic algorithm has reached the preset maximum number of iterations (e.g., 50 generations).

[0124] The predictive performance of the population-optimal individual GNN architecture improves by less than a preset threshold (e.g., 0.001) within a preset number of generations (e.g., 10 generations).

[0125] The preset computing resource limit is reached (e.g., total training time exceeds 24 hours).

[0126] The optimized GNN architecture can be applied to graph data analysis tasks with high performance and efficiency requirements. Typical application scenarios include, but are not limited to:

[0127] Drug discovery and molecular property prediction: In this scenario, the input is graph data representing the molecular structure of chemical compounds (nodes represent atoms, edges represent chemical bonds, and nodes and edges can have features such as atom type and chemical bond type). The optimized GNN architecture is used to perform graph classification tasks and predict specific molecular properties, such as biological activity, toxicity, solubility, or druggability. This method can automatically discover the optimal GNN architecture suitable for capturing complex molecular structure-property relationships, accelerating the candidate drug screening process and reducing R&D costs. For example, in the graph classification task of chemical compounds, the output is a category label for each graph (molecule) (e.g., whether it has mutagenicity or high activity).

[0128] Bioinformatics and Precision Medicine: The input is biological network graph data, such as protein-protein interaction (PPI) networks (nodes are proteins, edges are interactions) or gene regulation networks. The optimized GNN architecture can be used for node classification tasks (such as predicting protein function, identifying disease-related genes) or graph classification tasks (such as classifying cancer subtypes based on gene expression networks). This method helps to discover GNN models that can effectively integrate biological network topology and node features, advancing precision medicine and biological research. For example, in node classification tasks in citation networks (analogous to certain biological networks), the input is the graph structure data of the network, and the output is the category label for each node.

[0129] Financial Risk Control and Fraud Detection: The input is a financial transaction network (nodes are accounts / users, edges are transactions / transfers) or a user relationship network. The optimized GNN architecture is used for node classification (e.g., identifying fraudulent accounts, money laundering accounts, or high-risk users) or subgraph detection (identifying fraud groups). This method can automatically optimize a GNN architecture that can effectively capture complex and dynamic fraud patterns, improving financial institutions' risk identification capabilities and anti-fraud efficiency.

[0130] Citation network node classification takes a graph structure of the citation network as input, where nodes represent documents and edges represent citation relationships, and the output is a category label (e.g., research topic) for each node.

[0131] Next, a specific example of the method of the present invention will be described.

[0132] S1. Constructing the GNN architecture grammar rule base: The GNN architecture grammar rule base is based on... Figure 3 The rules shown are used to construct this syntax rule base. This rule base is defined using BNF normal form and includes non-terminal and terminal symbols, as well as production rules.

[0133] For example, nonterminal symbols <net>The representative network module, its production rules

[0134] <net> → <dim> <gconv> <fc>This indicates that a network can be represented by a single-dimensional hyperparameter. <dim>A graph convolution module <gconv>and a fully connected layer <fc>It is formed by concatenation. The terminator GCNConv represents the specific GCN convolution operation, ReLU represents the ReLU activation function, and 16, 32, 64, 128, etc. represent the specific values ​​of the hidden layer dimension.

[0135] S2. Generate the initial GNN architecture population: based on Figure 3 The syntax rule base is used to generate an initial population with a population size of 50 through the genetic programming tool of the DEAP library. Each individual GNN architecture is represented as a syntax tree, and the syntax tree generation process is as described above, starting with the initial symbol... <net>The root node is used, and then, according to the syntax rules, non-terminal nodes are recursively expanded until all leaf nodes are terminal symbols. For example, according to the rules... <net> → <dim> <gconv> <fc> ， <net>The node expands to three child nodes. <dim> 、 <gconv>and <fc>.for <gconv>Nodes can also be based on rules

[0136] <gconv> → <dim> <conv> <fc>or <gconv> → <conv>The process continues to expand until all leaf nodes are terminal symbols such as GCNConvLayer, ReLU, and 16, eventually forming a complete syntax tree. This syntax tree synchronously encodes the network structure and hyperparameter information of the GNN.

[0137] S3. Performance Prediction Based on Gaussian Process Agent Model: Using the Gaussian process agent model to predict the performance of the GN N architecture, with a process framework as follows: Figure 2 As shown. The Gaussian process surrogate model is defined based on formula (1), and its covariance function k(x,x') is calculated based on the tree kernel function defined by formula (3). The Gaussian process surrogate model is constructed using the scik-learn and SciPy libraries in Python. The tree kernel function is implemented using a custom TreeKernel class, which calculates the similarity between syntax trees based on the tree edit distance algorithm provided by the zss library. The kernel function width parameter σ is set to 1.0, and the regularization parameter is set to 1e. -6 .

[0138] In the initial stage, five GNN architectures are randomly selected for actual training and evaluation. Their classification accuracy on the validation set is used as the true performance to initialize the Gaussian process surrogate model. Then, for subsequently generated GNN architectures, the trained Gaussian process surrogate model is used to predict their performance and estimate the prediction uncertainty (prediction variance).

[0139] S4. Genetic Algorithm Optimization: The genetic algorithm tools provided by the DEAP library are used to perform an optimization search in the syntax tree space. Tournament selection is used for the selection operation, with a tournament size of 3. Subtree crossover is used for the crossover operation, with a crossover probability of 0.5. Subtree mutation is used for the mutation operation, with a mutation probability of 0.2.

[0140] S5. Actual training evaluation and model update: Calculate the expected improvement value EI(x) and the predicted probability P using formulas (8), (9) and (10). predict (x) and training probability P train (x). The exploration factor ξ is set to 0, and the lower bound of the prediction probability P0 is set to 0.1. If the training probability P train If x is greater than 0.5, the PyTorchGeometric library is used to train and evaluate the GNN architecture. The Adam optimizer is used, with a learning rate of 0.01, a cross-entropy loss function, and 50 or 100 training epochs. Experiments are conducted on the Cora and Citeseer node classification datasets, as well as the MUTAG and PTC graph classification datasets. The experimental hardware environment is a 64-bit system with an Intel(R) Core(TM) i5-11320H processor, 16GB of RAM, and an NVIDIA GeForce MX450 graphics card.

[0141] The actual performance of the GNN architecture obtained from the actual training evaluation is used as a new data point and added to the training dataset of the Gaussian process surrogate model. The Gaussian process surrogate model is then retrained using all the evaluated data to improve prediction accuracy.

[0142] S6. Termination of Optimization: The genetic algorithm terminates after 50 generations of iteration. The best-performing GNN architecture in the last generation is output as the final optimization result. This optimized GNN architecture can then be applied to specific graph data analysis tasks. For example, in the graph classification task of chemical compounds, it can be used to predict the mutagenicity of molecules (e.g., using the MUTAG dataset for validation). The input is a molecular graph, nodes are atoms, edges are chemical bonds, and the model outputs a binary classification result indicating whether the molecule is mutagenic. The architecture optimized by this method aims to improve the accuracy of such molecular attribute prediction tasks, demonstrating its potential in drug discovery-assisted design. Similarly, the optimized architecture can also be applied to citation network node classification (e.g., the Cora dataset) or other graph data analysis tasks in fields such as bioinformatics and financial risk control.

[0143] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0144] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.< / conv> < / gconv> < / fc> < / conv> < / dim> < / gconv> < / gconv> < / fc> < / gconv> < / dim> < / net> < / fc> < / gconv> < / dim> < / net> < / net> < / fc> < / gconv> < / dim> < / fc> < / gconv> < / dim> < / net> < / net> < / conv> < / fc> < / conv> < / dim> < / gconv> < / fc> < / conv> < / dim> < / gconv> < / gconv> < / gconv> < / net> < / fc> < / conv> < / dim> < / fc> < / conv> < / dim> < / gconv> < / fc> < / gconv> < / net> < / k> < / dim> < / act> < / gat> < / conv> < / fc> < / attention> < / residual> < / gconv> < / net>

Claims

1. A graph neural network architecture optimization method based on syntactic genetic programming, characterized in that, Includes the following steps: S1. Construct the syntax rule base for the graph neural network architecture GNN; S2. Based on the syntax rule base, an initial GNN architecture population is generated through genetic programming. The GNN architecture of each individual in the population is represented as a syntax tree. The syntax tree is recursively generated according to the production rules, and the network structure and hyperparameters of each layer of the network are encoded synchronously. S3. Predict the performance of individual GNN architectures in the population using a Gaussian process surrogate model; S4. Based on the prediction performance of individual GNN architectures, the optimal GNN architecture is searched in the syntax tree space using a genetic algorithm. S5. For individual GNN architectures in the population, based on their expected improvement function... Calculate training probability If training probability If the performance exceeds the preset evaluation threshold, the individual GNN architecture is selected for actual training and evaluation to obtain its true performance, and the true performance is used to update the Gaussian process proxy model. Expected Improvement Function The computational results are used to determine the training probabilities and to evaluate the potential improvement value of individual GNN architectures. It is also used to guide the search direction of the genetic algorithm or to select from multiple individuals that meet the evaluation conditions; among them, guiding the search direction of the genetic algorithm includes: giving priority to individuals with higher improvement value in the selection operation of the genetic algorithm, or selecting the individual with the highest improvement value from multiple individual GNN architectures that meet the actual training and evaluation conditions for actual training and evaluation. Expected Improvement Function Based on the following formula definition: ； in, Indicates the amount of improvement. Indicates the amount of improvement in standardization. , , and These are the mean and standard deviation predicted by the Gaussian process surrogate model, respectively. For the best performance among the currently evaluated GNN architectures, To explore factors, and These are the cumulative distribution function and probability density function of the standard normal distribution, respectively. S6. Determine whether the preset optimization termination condition is met. If yes, output the optimized GNN architecture; otherwise, return to step S3. The optimized GNN architecture is used to process graph structure data to perform classification tasks. The optimized GNN architecture is used for node classification in citation networks or for graph classification of chemical compounds to predict molecular properties. In the node classification task in citation networks, the input is the graph structure data of the citation network, where nodes represent documents and edges represent citation relationships, and the output is the category label for each node. In the graph classification task of chemical compounds, the input is a set of graph structure data representing chemical compounds, where nodes represent atoms and edges represent chemical bonds, and the output is the category label for each graph.

2. The graph neural network architecture optimization method based on syntactic genetic programming according to claim 1, characterized in that, In S1, the syntax rule base is defined based on the BNF normal form and includes the set of nonterminal symbols N, the set of terminal symbols T, the set of production rules P, and the start symbol S; The set of nonterminal symbols N contains scalable network modules and is divided into a set of hyperparameter nonterminal symbols. and module nonterminal symbol set ; The set of terminal symbols T contains specific network operators and hyperparameter values, and is divided into the set of hyperparameter terminal symbols. and module terminator set ; The set of production rules P defines the derivation relations from non-terminal symbols to terminal symbols and / or non-terminal symbols; The starting symbol S represents the initial derivation symbol for the network architecture.

3. The graph neural network architecture optimization method based on syntactic genetic programming according to claim 2, characterized in that, Each production rule in the production rule set P The form is ,in For the derived symbol, To derive the result, This refers to the hyperparameter part. This indicates the submodule section. For the set of hyperparameter nonterminal symbols, For the set of hyperparameter terminal symbols, For the set of nonterminal symbols of a module, This is the set of module terminators.

4. The graph neural network architecture optimization method based on syntactic genetic programming according to claim 3, characterized in that, In S2, the process of generating the syntax tree includes: Initialization: Create an initial syntax tree with the start symbol S as the root node; Recursive expansion: For each non-terminal leaf node in the current syntax tree From the subset of production rules A production rule is randomly selected from the data and expanded accordingly. The right-hand side of the production rule is a sequence of hyperparameter symbols and module symbols; Termination condition: Repeat the recursive expansion steps until all leaf nodes of the syntax tree are terminal symbols.

5. The graph neural network architecture optimization method based on syntactic genetic programming according to claim 1, characterized in that, S3 include: S31. Define the Gaussian process surrogate model based on the following formula: ； in, Represents the fitness function. Represents the mean function, The covariance function and the mean function are defined and calculated based on the following formula: ； in, Indicates the expected value; The stochastic function in the Gaussian process surrogate model is used to approximate the true fitness function; the covariance function Calculations are based on the tree kernel function defined below: ； in, Representation of syntax tree and Tree edit distance between them This is the kernel function width parameter; S32, based on the evaluated training data The Gaussian process surrogate model is trained, where For the set of GNN architectures that have been evaluated, This corresponds to the actual performance. S33. Predict new, unevaluated GNN architectures using a trained Gaussian process surrogate model. fitness function The distribution of the results follows a normal distribution: ； in, To predict the mean; The standard deviation of the forecast represents the uncertainty of the forecast.

6. The graph neural network architecture optimization method based on syntactic genetic programming according to claim 1, characterized in that, In S4, the genetic algorithm includes selection, crossover, and mutation operations; The selection operation adopts the tournament selection method, in which a certain number of individuals are randomly selected from the population for comparison each time, and the individual with the best prediction performance is selected as the parent. The selection operation is repeated until the number of parent individuals meets the requirements for crossover and mutation. The crossover operation includes: randomly selecting a non-terminal symbol node from each of the two parent individual syntax trees, ensuring that the non-terminal symbol types of the two nodes are the same; swapping the subtrees rooted at the two selected nodes to generate two new child individual syntax trees; The mutation operation includes: randomly selecting a non-terminal symbol node in the individual syntax tree; randomly generating a new subtree based on the production rules in the syntax rule base that take the non-terminal symbol as the left; and replacing the selected node with the newly generated subtree to complete the mutation operation.

7. The graph neural network architecture optimization method based on syntactic genetic programming according to claim 1, characterized in that, S5 also includes: Calculate the predicted probability and training probability And based on the training probability The decision to perform actual training and evaluation on individual GNN architectures, including the prediction probability. and training probability The calculation formula is: ； ； in, To provide a lower bound for the prediction probability, if the training probability... If the value is higher than the preset value, then the individual GNN architecture will be affected. Conduct actual training and evaluation to obtain its real performance on the validation set. and using new data points Update the Gaussian process surrogate model by adding new data points to the evaluated dataset and retraining the Gaussian process surrogate model.

8. The graph neural network architecture optimization method based on syntactic genetic programming according to claim 1, characterized in that, In S6, the optimization termination condition includes at least one of the following: The genetic algorithm has reached its preset maximum number of iterations. The predictive performance of the population-optimal individual GNN architecture improves by less than a preset threshold within a preset number of consecutive generations; The preset computing resource limit has been reached.

Images