Graph neural network architecture search method and apparatus

Abstract

The application provides a graph neural network architecture search method and device, and relates to the technical field of artificial intelligence and deep learning, and the method comprises the following steps: constructing a super network based on an application requirement and a graph neural network search space, the search space comprising a function space and an operation space; performing search on the function space to determine a function combination with the highest accuracy of the super network; fixing the function of each position on the super network according to the function combination with the highest accuracy; and performing search on the operation space to determine an optimal graph neural network architecture satisfying the hardware efficiency requirement and the accuracy requirement in the application requirement. The application divides the search space into the function space and the operation space, and organizes the search space in the form of the super network, so that the search time can be effectively reduced. In addition, the application combines a hardware perception device, so that the actual hardware efficiency of the searched graph neural network architecture on a target device can be effectively improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and deep learning technology, and in particular to a method and apparatus for searching graph neural network architectures. Background Technology

[0002] Graph Neural Networks (GNNs) are a newly emerging deep learning method in recent years, capable of processing non-Euclidean space data. Traditional deep learning methods, such as Convolutional Neural Networks (CNNs), can effectively extract features from regularized data (images, speech, text). However, in the real world, many practical application scenarios generate data from non-Euclidean spaces, making traditional deep learning methods unsuitable for processing such data. Therefore, Graph Neural Networks were proposed to process unstructured data and have made significant progress in various fields. Currently, Graph Neural Networks have strong application potential in various research areas such as social network analysis, knowledge graphs, recommender systems, and point cloud processing, and are gradually becoming mainstream methods. The essence of Graph Neural Networks is a graph + neural network; it can perform neural network operations on irregular graph data, possesses powerful representation learning capabilities, and can effectively extract spatial topological features from irregular data. Each layer of a basic Graph Neural Network contains structural components such as node aggregation functions, node update functions, node sampling functions, and activation functions.

[0003] Manually designed graph neural network (GNN) models often rely on expert experience to select different structural components for the GNN layers. This requires significant time for retraining and optimization when faced with different tasks and datasets. Furthermore, it's difficult for humans to consider all bottlenecks in the model, making it challenging to efficiently design high-performance GNN models for various scenarios. Therefore, automated design techniques for neural network architecture search have emerged.

[0004] Neural architecture search is an automated machine learning technique (AutoML) whose main task is to search for an optimal network model structure within a given design space of neural network models, given optimization objectives (such as model accuracy), search strategies (such as evolutionary algorithms and reinforcement learning), and evaluation methods. However, the applicant has found that existing neural network architecture search techniques have the following shortcomings:

[0005] 1. The search objective is solely based on maximizing accuracy. The optimization function of the search system does not incorporate hardware efficiency metrics, resulting in a graph neural network architecture that fails to meet the computational efficiency requirements of real-world applications.

[0006] 2. There is a lack of effective means to evaluate the hardware efficiency of graph neural network architectures. The low computing performance and high communication latency of many target deployment devices make it infeasible to deploy the sub-architectures to the target devices in real time after sampling for hardware efficiency testing.

[0007] 3. The search strategy uses simple reinforcement learning or evolutionary algorithms. After each sampling of a sub-architecture, training is required to obtain the accuracy performance of the sub-architecture, resulting in a large search time overhead. Summary of the Invention

[0008] In view of this, the present invention provides a graph neural network architecture search method and apparatus to solve at least one of the aforementioned problems.

[0009] To achieve the above objectives, the present invention adopts the following solution:

[0010] According to a first aspect of the present invention, a graph neural network architecture search method is provided, the method comprising: constructing a supernet in a graph neural network search space based on application requirements, the search space including a function space and an operation space; performing a search in the function space to determine a combination of functions that maximizes the accuracy of the supernet; fixing the function at each position in the supernet according to the combination of functions with the highest accuracy; and performing a search in the operation space to determine an optimal graph neural network architecture that satisfies the hardware efficiency requirements and accuracy requirements in the application requirements.

[0011] As an embodiment of the present invention, the above method further includes: obtaining the application requirements of the target application, wherein the application requirements include: target dataset, target deployment device, accuracy requirements and hardware efficiency requirements.

[0012] As an embodiment of the present invention, the method described above for searching the function space to determine a function combination that maximizes the accuracy of the supernet includes: sampling the function space to generate a function space sub-architecture, the function space sub-architecture including two sets of function settings corresponding to the upper and lower halves of the supernet respectively; training the function space sub-architecture based on the function settings to obtain the accuracy of the function settings; repeating the sampling of the function space and the training of the function space sub-architecture until a preset maximum number of iterations is reached; and determining a set of function combination settings that maximizes the accuracy of the supernet.

[0013] As an embodiment of the present invention, after fixing the function at each position on the supernet according to the function combination with the highest accuracy, the above method further includes: performing supernet pre-training for a preset number of rounds to obtain supernet pre-training weights.

[0014] As an embodiment of the present invention, the method described above for performing a search of the operation space to determine the optimal graph neural network architecture that meets the hardware efficiency and accuracy requirements of the application includes: randomly sampling the operations at each position on the supernet to obtain an operation space sub-architecture; inputting the operation space sub-architecture into a hardware sensing device, which obtains a hardware efficiency score corresponding to the operation space sub-architecture; determining whether the hardware efficiency score meets the hardware efficiency requirements; in response to the hardware efficiency meeting the hardware efficiency requirements, assigning supernet pre-training weights to the operation space sub-architecture to test it on the validation set of the target dataset to obtain an accuracy score; performing the next round of population iteration based on an evolutionary algorithm until the set maximum number of iterations is reached, and outputting the optimal graph neural network architecture.

[0015] As an embodiment of the present invention, the hardware efficiency score obtained by the hardware sensing device in the above method corresponding to the operation space sub-architecture includes: taking each operation of the operation space sub-architecture as a node, the data flow direction between operations as an edge, and introducing a global node to store the graph attribute information of the input target dataset. The global node connects all other nodes, thereby abstracting the operation space sub-architecture into a directed graph; converting the directed graph into an adjacency matrix; assigning a feature vector to each node based on a one-hot code to form a feature matrix; and using the encoding of the target device, the adjacency matrix, and the feature matrix as inputs to a graph neural network predictor to output the hardware efficiency score through the graph neural network predictor.

[0016] As an embodiment of the present invention, the graph neural network predictor in the above method is trained in the following manner: N graph neural network architectures are randomly sampled from the graph neural network search space; the corresponding hardware efficiency is obtained by testing on different target deployment devices, thereby forming a graph neural network architecture-hardware efficiency dataset; the graph neural network predictor is trained for a preset number of rounds based on the dataset to improve its prediction accuracy.

[0017] According to a second aspect of the present invention, a graph neural network architecture search apparatus is provided, the apparatus comprising: a supernet construction unit, configured to construct a supernet in a graph neural network search space based on application requirements, the search space including a function space and an operation space; a function space search unit, configured to perform a search in the function space to determine a combination of functions that maximizes the accuracy of the supernet; a supernet function fixing unit, configured to fix the function at each position on the supernet according to the combination of functions with the highest accuracy; and an operation space search unit, configured to perform a search in the operation space to determine an optimal graph neural network architecture that satisfies the hardware efficiency requirements and accuracy requirements in the application requirements.

[0018] As an embodiment of the present invention, the above-mentioned apparatus further includes a requirement acquisition unit, used to acquire the application requirements of the target application, the application requirements including: target dataset, target deployment device, accuracy requirements and hardware efficiency requirements.

[0019] As an embodiment of the present invention, the function space search unit is specifically used for: sampling the function space to generate a function space sub-architecture, the function space sub-architecture including two sets of function settings corresponding to the upper and lower halves of the supernet respectively; training the function space sub-architecture based on the function settings to obtain the accuracy of the function settings; repeating the sampling of the function space and the training of the function space sub-architecture until a preset maximum number of iterations is reached; and determining the set of function combination settings that makes the supernet have the highest accuracy.

[0020] As an embodiment of the present invention, the above-mentioned device further includes a pre-training unit for performing a preset number of rounds of supernet pre-training to obtain supernet pre-training weights.

[0021] As an embodiment of the present invention, the above-mentioned operation space search unit specifically includes:

[0022] A random sampling module is used to randomly sample the operations at each location on the supernet to obtain the operation space sub-architecture;

[0023] An input module is used to input the operation space sub-architecture into a hardware sensing device, and the hardware sensing device obtains the hardware efficiency score corresponding to the operation space sub-architecture.

[0024] The judgment module is used to determine whether the hardware efficiency score meets the hardware efficiency requirements.

[0025] The architecture output module is used to assign supernet pre-training weights to the operation space sub-architecture in response to the hardware efficiency requirements, and to test it on the validation set of the target dataset to obtain an accuracy score; and to perform the next round of population iteration based on the evolutionary algorithm until the set maximum number of iterations is reached, and output the optimal graph neural network architecture.

[0026] As an embodiment of the present invention, the hardware sensing device obtains the hardware efficiency score corresponding to the operation space sub-architecture by: taking each operation of the operation space sub-architecture as a node, the data flow between operations as an edge, and introducing a global node to store the graph attribute information of the input target dataset. The global node connects all other nodes, thereby abstracting the operation space sub-architecture into a directed graph; converting the directed graph into an adjacency matrix; assigning a feature vector to each node based on a one-hot code to form a feature matrix; and using the encoding of the target device, the adjacency matrix, and the feature matrix as inputs to a graph neural network predictor to output the hardware efficiency score through the graph neural network predictor.

[0027] As an embodiment of the present invention, the above-mentioned graph neural network predictor is trained in the following manner: N graph neural network architectures are randomly sampled from the graph neural network search space; the corresponding hardware efficiency is obtained by testing on different target deployment devices, thereby forming a graph neural network architecture-hardware efficiency dataset; the graph neural network predictor is trained for a preset number of rounds based on the dataset to improve its prediction accuracy.

[0028] According to a third aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0029] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0030] As can be seen from the above technical solutions, the graph neural network architecture search method and apparatus provided by this invention adopt a fine-grained design approach, dividing the search space into a function space and an operation space, and organizing the search space through a supernet, which can effectively reduce the search time and thus solve the problem of efficient deployment and real-time operation of graph neural networks on edge devices. Furthermore, this application, combined with the proposed hardware-aware device, can effectively improve the actual hardware efficiency of the searched graph neural network architecture on the target device, simultaneously meeting the accuracy and hardware efficiency requirements of the application scenario for the graph neural network model. This has significant advantages compared to traditional manual design and automated design technologies without hardware-aware means. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0032] Figure 1 This is a flowchart illustrating a graph neural network architecture search method provided in an embodiment of this application;

[0033] Figure 2 This is a schematic diagram of the function space search process provided in an embodiment of this application;

[0034] Figure 3 This is a flowchart illustrating a graph neural network architecture search method according to another embodiment of this application;

[0035] Figure 4 This is a schematic diagram of the working process of the hardware sensing device provided in the embodiments of this application;

[0036] Figure 5 This is a schematic diagram of the hardware sensing device provided in an embodiment of this application;

[0037] Figure 6 This is a schematic diagram of the training process of the GNN predictor provided in the embodiments of this application;

[0038] Figure 7 This is a complete flowchart illustrating the graph neural network architecture search method provided in the embodiments of this application;

[0039] Figure 8 This is a schematic diagram of a hardware-aware graph neural network search system architecture provided in an embodiment of this application;

[0040] Figure 9 This is a schematic diagram of the structure of a graph neural network architecture search device provided in an embodiment of this application;

[0041] Figure 10 This is a schematic diagram of the structure of a graph neural network architecture search device provided in another embodiment of this application;

[0042] Figure 11 This is a schematic diagram of the operation space search unit provided in an embodiment of this application;

[0043] Figure 12 This is a schematic block diagram of the system configuration of the electronic device provided in the embodiments of this application. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0045] First, a brief description of some of the technical terms used in this application:

[0046] The term "Graph Neural Networks" (GNN) is a type of deep learning model used to process graph-structured data. It can learn node representations using information from the nodes and edges in a graph, and perform tasks such as classification, clustering, and generation.

[0047] The term "hardware awareness" refers to the awareness of the execution efficiency of a neural network model on a target hardware computing platform. It does not require the actual deployment of the model on the computing platform. Instead, it uses a certain strategy to obtain its hardware-related performance and to guide the neural network architecture design process.

[0048] The term "Neural Architecture Search" (NAS) is an automated design and optimization method for neural network models. It searches for the optimal network structure for a specific task.

[0049] Considering that the deployment requirements of graph neural network models in real-world applications include not only high accuracy but also high computational efficiency on target devices and other hardware performance requirements, existing solutions have failed to provide a method for incorporating hardware efficiency targets into the search process for multi-objective search, nor a method for evaluating the hardware efficiency of sampled sub-architectures on target devices. Furthermore, existing search strategies are too inefficient and time-consuming. Therefore, the purpose of this application is to propose a graph neural network architecture search method and apparatus. On the one hand, it can introduce hardware efficiency indicators, enabling the final graph neural network architecture to simultaneously meet the accuracy and hardware efficiency requirements of the application scenario; on the other hand, it can improve search efficiency and reduce search time.

[0050] like Figure 1 The diagram shown is a flowchart of a graph neural network architecture search method provided in an embodiment of this application. The method includes the following steps:

[0051] Step S101: Construct a supernet for the graph neural network search space based on application requirements. The search space includes a function space and an operation space.

[0052] In this embodiment, the search space of the graph neural network is first hierarchically divided into a function space and an operation space. These will be described separately below:

[0053] 1. Function Space

[0054] The function space is composed of the specific functions corresponding to each operation performed by a graph neural network. For example, for the aggregation operation in a graph neural network, the specific functions during its execution include the aggregation operator type, the aggregation message structure, etc. Unlike a specific graph neural network operation, the function space includes the specific function types corresponding to the operation. All components contained in the function space can be shown in Table 1 below:

[0055] Table 1

[0056]

[0057] Of course, the function types included in the function space of this embodiment are not limited to those in Table 1. More or fewer function types can be added based on factors such as the emergence of other specific new applications, new network structures, and limitations of computing resources.

[0058] 2. Operating space

[0059] The operation space consists of operations performed by each specific graph neural network. Because operations and their specific execution functions are broken down hierarchically, the operation space contains only operation types, not specific function options. The components included in the operation space can be: aggregation operations, combination operations, sampling operations, and connection operations. Aggregation operations perform message aggregation between nodes, combination operations update node features, sampling operations sample neighbors during node aggregation, and connection operations connect different operation components. Similarly, the operation types included in the operation space in this embodiment are not limited to the above four types; more or fewer operation types can be added depending on other specific applications, the emergence of new network structures, and limitations of computing resources.

[0060] In this embodiment, the supernet consists of N positions, such as position 1, position 2, ..., position N. Each position contains paths formed by combinations of all operation functions. During sampling, one path can be randomly selected at each position to construct a complete graph neural network architecture. This supernet's search space organization decouples the search and training processes, allowing weights to be reused in subsequent training, thereby improving search efficiency.

[0061] Since the search space of the graph neural network in this embodiment is hierarchically divided and organized in the form of a supernet, the subsequent architecture search process is also divided into a multi-stage search of the function space and the operation space, that is, the hierarchical design space is explored by a multi-stage search of the function space first and then the operation space.

[0062] Step S102: Perform a search of the function space to determine a combination of functions that maximizes the accuracy of the supernet.

[0063] To reduce search time during the function space search, the supernet can be divided into two parts: the upper part, for example, includes positions 1 to N / 2, and the lower part includes positions N / 2+1 to N. This allows for function reuse between the upper and lower parts of the supernet. Specifically, for the upper part, a set of function options is searched and reused at every position within the upper part of the supernet. The same method is used for the lower part. Ultimately, the function search no longer requires independent searching at every position, but only searches the two sets of functions. During the function search, accuracy is optimized. Each time, two sets of function settings are sampled, and then the functions are fixed at each position in the supernet. Mini-batch training of the supernet is then performed to obtain the supernet accuracy under these function settings, which is then fed back to the evolutionary algorithm for iterative processing.

[0064] Based on the above description, such as Figure 2 As shown, step S102 may further include the following sub-steps:

[0065] Step S1021: Sample the function space to generate a function space sub-architecture, the function space sub-architecture including two sets of function settings corresponding to the upper and lower halves of the supernet respectively.

[0066] Step S1022: Train the function space sub-architecture based on the function settings to obtain the accuracy of the function settings.

[0067] Training the function space subarchitecture here only requires small-batch training, such as 50 rounds of small-batch training.

[0068] Step S1023: Repeat the sampling of the function space and the training of the function space sub-architectures until the preset maximum number of iterations is reached. That is, repeat steps S1021-S1022 above until the preset maximum number of iterations is reached, such as performing 1000 iterations of function space search.

[0069] Step S1024: Determine the set of function combinations that maximizes the accuracy of the supernet.

[0070] In this embodiment, through the above steps S1021-S1024, it is no longer necessary to search for a set of function settings for each position of the supernet, thereby significantly reducing the search time overhead.

[0071] Step S103: Fix the function at each position on the supernet according to the function combination with the highest accuracy.

[0072] Step S104: Perform a search of the operation space to determine the optimal graph neural network architecture that meets the hardware efficiency and accuracy requirements of the application.

[0073] In this embodiment, after fixing the function combination at each position of the supernet, the operation space sub-architecture is obtained by searching the operation space, that is, sampling each position of the supernet. These operation space sub-architectures need to meet not only the accuracy requirements but also the hardware efficiency requirements. Then, the optimal graph neural network architecture that meets the above requirements is obtained based on the evolutionary algorithm.

[0074] As described above, the graph neural network architecture search method provided by this invention employs a fine-grained design approach, dividing the search space into a function space and an operation space, and organizing the search space through hypernetting. This effectively reduces search time, thereby solving the problem of efficient deployment and real-time operation of graph neural networks on edge devices. Furthermore, this application, combined with the proposed hardware-aware device, can effectively improve the actual hardware efficiency of the searched graph neural network architecture on the target device, simultaneously meeting the accuracy and hardware efficiency requirements of application scenarios for graph neural network models. This method has significant advantages compared to traditional manual design and automated design techniques without hardware-aware means.

[0075] like Figure 3 The diagram shown is a flowchart of a graph neural network architecture search method according to another embodiment of this application. The method includes the following steps:

[0076] Step S301: Obtain the application requirements of the target application, including: target dataset, target deployment device, accuracy requirements, and hardware efficiency requirements.

[0077] The target dataset is used to score the accuracy and hardware efficiency of the sub-architecture in the subsequent search of the operating space. The target deployment device indicates the target device to be deployed, including information such as the encoding of the target deployment device. The accuracy requirement and hardware efficiency requirement represent the accuracy and hardware efficiency that the final neural network architecture must meet.

[0078] Step S302: Construct a supernet for the graph neural network search space based on application requirements. The search space includes a function space and an operation space.

[0079] Step S303: Perform a search of the function space to determine a combination of functions that maximizes the accuracy of the supernet.

[0080] Step S304: Fix the function at each position on the supernet according to the function combination with the highest accuracy.

[0081] Steps S302-S304 are the same as steps S101-S103 in the previous embodiment, and will not be repeated here.

[0082] Step S305: Perform a preset number of rounds of supernet pre-training to obtain the supernet pre-training weights.

[0083] The pre-trained weights of the supernet trained in this step are used to prepare for subsequent evaluation of the accuracy scores of the operation space sub-architecture.

[0084] Step S306: Randomly sample the operations at each location on the supernet to obtain the operation space sub-architecture.

[0085] In this embodiment, during the aforementioned function space search phase, the supernet does not fix the operations at each position during sampling; instead, it completes the forward propagation of each training round through random sampling. However, during the operation space search in this step, the functions at each position on the supernet are fixed. Sampling is performed by randomly sampling the operations that should be connected at each position on the supernet each time, thereby constructing the operation space sub-architecture.

[0086] Step S307: Input the operation space sub-architecture into the hardware sensing device, and obtain the hardware efficiency score corresponding to the operation space sub-architecture from the hardware sensing device.

[0087] In this embodiment, the aforementioned hardware sensing device is mainly based on a GNN predictor built from a three-layer graph convolutional neural network to predict the hardware efficiency of a given graph neural network architecture.

[0088] like Figure 4 As shown, the hardware efficiency score obtained by the hardware sensing device in this step, corresponding to the operating space sub-architecture, may specifically include the following steps S401-S404:

[0089] Step S401: Using each operation of the operation space sub-architecture as a node and the data flow between operations as an edge, a global node is introduced to store the graph attribute information of the input target dataset. The global node connects to all other nodes, thereby abstracting the operation space sub-architecture into a directed graph.

[0090] Step S402: Convert the directed graph into an adjacency matrix.

[0091] Step S403: Assign a feature vector to each node based on the one-hot code form to form a feature matrix.

[0092] Step S404: Use the target device's encoding, the adjacency matrix, and the feature matrix as inputs to a graph neural network predictor to output a hardware efficiency score.

[0093] Steps S401-S404 above can be combined Figure 5 The schematic diagram of the hardware sensing device shown illustrates that the directed graph abstracted from the graph neural network architecture (operation space sub-architecture) includes input nodes, output nodes, operation nodes Op1, Op2, Op3, Op4, Op5, and a global node. Then, an adjacency matrix and a feature matrix are generated. The target device, the adjacency matrix, and the feature matrix are used as inputs to the GNN predictor. The GNN predictor outputs the hardware efficiency of the given graph neural network architecture on the target device.

[0094] Preferred, such as Figure 6 As shown, the GNN predictor is trained using the following steps S601-S603.

[0095] Step S601: Randomly sample N graph neural network architectures from the graph neural network search space. Here, the value of N should be as large as possible to improve the prediction accuracy of the GNN predictor; for example, 20,000 can be used.

[0096] Step S602: Test on different target deployment devices to obtain the corresponding hardware efficiency, thereby constructing a dataset of graph neural network architecture-hardware efficiency.

[0097] Different target deployment devices can collect various deployment devices currently on the market, such as Raspberry Pi, Jetson, Intel CPU, Nvidia GPU, etc. Of course, the training of the GNN predictor can also be retrained as the deployment devices on the market are updated to adapt to the needs of the new deployment devices.

[0098] Step S603: Train the graph neural network predictor for a preset number of rounds based on the dataset to improve its prediction accuracy. The preset number of rounds can be set as needed and has no fixed value. It can be based on expert experience, such as training for 300 rounds.

[0099] Step S308: Determine whether the hardware efficiency score meets the hardware efficiency requirements. If it does, proceed to step S309; ​​otherwise, return to step S306.

[0100] Step S309: Assign pre-trained weights to the operation space sub-architecture to test on the validation set of the target dataset to obtain an accuracy score.

[0101] Step S310: Determine whether the set maximum number of iterations has been reached. If it has, output the optimal graph neural network architecture. If it has not, return to step S306.

[0102] In this embodiment, the graph neural network architecture search method described above employs a multi-objective optimization function during the search process. This function considers not only accuracy but also the hardware efficiency of the architecture on the target device. The specific optimization function is as follows:

[0103]

[0104]

[0105] in The weights representing the model, It is the target deployment device. It's a delay constraint, acc train Accuracy represents the model's accuracy on the training set. val The lat represents the model's accuracy on the validation set, and the lat represents the accuracy on the target deployment device. Model inference latency, These are candidate GNN architectures, where α and β are modulating factors used to adjust the optimization bias between accuracy and hardware efficiency. By adjusting the relative values ​​of α and β, a GNN architecture that satisfies both accuracy and hardware efficiency requirements can be searched.

[0106] To better understand the above steps S301-S310, this embodiment also provides the following: Figure 7 The flowchart shown is a diagram of the graph neural network architecture search method. The two descriptions are consistent. The application of the optimization function is specifically reflected in the entire process of function space sampling, operation space sampling and output graph neural network architecture, which will not be elaborated further here.

[0107] Based on the above method, a specific embodiment is given below to further describe this application. This embodiment is based on the following... Figure 8 The hardware-aware graph neural search system architecture shown is based on, and... Figure 8As can be seen, the system comprises three modules: a search space, a search controller, and a performance evaluation module. The search space is divided into a function space and an operation space, which are organized through a hypernet. The search controller continuously samples and generates sub-architectures within the designed fine-grained search space to iteratively obtain the optimal graph neural network architecture. Specifically, the search controller is responsible for exploring the search space by sampling and generating sub-architectures from the hypernet formed by the search space. Then, it uses a training device and an evaluation device to return the accuracy and hardware efficiency scores of the sub-architectures, updates the population based on an evolutionary algorithm, and iterates until a stopping condition (such as the maximum number of iterations) is met.

[0108] based on Figure 8 The search system in this specific embodiment uses the following search method:

[0109] s1. System startup and initialization.

[0110] s2. System input application requirements.

[0111] s3. The search device constructs a supernet in the GNN search space, fixing 12 supernet positions, and setting 4 different types of operation paths for each position.

[0112] s4. The search controller begins performing a search of the function space.

[0113] s5. The search controller samples the function space, obtains the function space substructure, and then inputs it into the training device for 50 rounds to obtain its accuracy, and feeds it back to the search controller.

[0114] s6. The search controller uses an evolutionary algorithm to sample and obtain the next batch of candidate architectures based on the accuracy of the sub-architectures, forming a new population.

[0115] s7. After 1000 iterations of function space search, input the set of optimal function settings for the upper and lower halves that maximizes the supernet accuracy.

[0116] s8. Fix the optimal function setting and complete 200 rounds of pre-training on the supernet based on the training device to obtain the pre-trained weights.

[0117] s9. The search controller performs the search space search process.

[0118] s10. Sample the search space sub-architecture and send it to the hardware sensing device for evaluation of the search space sub-architecture.

[0119] s11. The hardware sensing device constructs an abstract graph of the sub-architecture and uses a GNN predictor to predict its hardware efficiency, which is then fed back to the search controller.

[0120] s12. When hardware efficiency meets application requirements, the search controller reuses the pre-trained weights of the supernet to obtain its accuracy on the validation set of the input dataset for the sub-architecture; otherwise, the score of the architecture is directly set to 0. s13. Based on the accuracy of the sub-architecture and hardware efficiency, the architecture is scored, and the evolutionary algorithm performs the next round of candidate architecture selection, completing a new round of iteration for the population.

[0121] s14. After 1000 rounds of search space iterations, output the GNN sub-architecture with the highest score as the optimal architecture.

[0122] As described above, the graph neural network architecture search method provided by this invention employs a fine-grained design approach, dividing the search space into a function space and an operation space, and organizing the search space through hypernetting. This effectively reduces search time, thereby solving the problem of efficient deployment and real-time operation of graph neural networks on edge devices. Furthermore, this application, combined with the proposed hardware sensing device, can effectively improve the actual hardware efficiency of the searched graph neural network architecture on the target device, simultaneously meeting the accuracy and hardware efficiency requirements of application scenarios for graph neural network models. This method has significant advantages compared to traditional manual design and automated design techniques without hardware sensing capabilities. Using the method of this application can meet the requirements of real-time operation of graph neural networks on resource-constrained edge devices, facilitating the exploration of the application potential of graph neural networks in small terminal devices such as drones and autonomous vehicles, and is of great significance to the future development of multi-objective optimization technology for edge graph neural networks.

[0123] like Figure 9 The diagram shown is a schematic diagram of a graph neural network architecture search device provided in an embodiment of this application. The device includes: a supernet construction unit 910, a function space search unit 920, a supernet function fixing unit 930, and an operation space search unit 940, which are connected in sequence.

[0124] The supernet construction unit 910 is used to construct a supernet in the graph neural network search space based on application requirements. The search space includes a function space and an operation space.

[0125] The function space search unit 920 is used to perform a search of the function space to determine a combination of functions that maximizes the accuracy of the supernet.

[0126] The supernet function fixing unit 930 is used to fix the function at each position on the supernet according to the function combination with the highest accuracy.

[0127] The operation space search unit 940 is used to perform a search of the operation space to determine the optimal graph neural network architecture that meets the hardware efficiency and accuracy requirements of the application.

[0128] Preferred, such as Figure 10 As shown, the above-mentioned device also includes a requirement acquisition unit 950, which is connected to the supernet construction unit 910, and is used to acquire the application requirements of the target application. The application requirements include: target dataset, target deployment device, accuracy requirements, and hardware efficiency requirements.

[0129] Preferably, the function space search unit 920 is specifically used for: sampling the function space to generate a function space sub-architecture, the function space sub-architecture including two sets of function settings corresponding to the upper and lower halves of the supernet respectively; training the function space sub-architecture based on the function settings to obtain the accuracy of the function settings; repeating the sampling of the function space and the training of the function space sub-architecture until a preset maximum number of iterations is reached; and determining the set of function combination settings that maximizes the accuracy of the supernet.

[0130] Preferred, such as Figure 10 As shown, the above-mentioned device also includes a pre-training unit 960, which is connected to the supernet function fixing unit 930 and the operation space search unit 940 respectively, and is used to perform supernet pre-training for a preset number of rounds to obtain supernet pre-training weights.

[0131] Preferred, such as Figure 11 As shown, the aforementioned operation space search unit 940 specifically includes:

[0132] The random sampling module 941 is used to randomly sample the operation at each position on the supernet to obtain the operation space sub-architecture.

[0133] Input module 942 is used to input the operation space sub-architecture into the hardware sensing device, and the hardware sensing device obtains the hardware efficiency score corresponding to the operation space sub-architecture.

[0134] The judgment module 943 is used to determine whether the hardware efficiency score meets the hardware efficiency requirements.

[0135] The architecture output module 944 is used to, in response to the hardware efficiency meeting the hardware efficiency requirements, assign supernet pre-training weights to the operation space sub-architecture to test on the validation set of the target dataset to obtain an accuracy score; perform the next round of population iteration based on the evolutionary algorithm until the set maximum number of iterations is reached, and output the optimal graph neural network architecture.

[0136] Preferably, the hardware sensing device obtains the hardware efficiency score corresponding to the operation space sub-architecture by: taking each operation of the operation space sub-architecture as a node, the data flow between operations as an edge, and introducing a global node to store the graph attribute information of the input target dataset, wherein the global node connects all other nodes, thereby abstracting the operation space sub-architecture into a directed graph; converting the directed graph into an adjacency matrix; assigning a feature vector to each node based on a one-hot code to form a feature matrix; and using the encoding of the target device, the adjacency matrix, and the feature matrix as inputs to a graph neural network predictor to output the hardware efficiency score through the graph neural network predictor.

[0137] Preferably, the above-mentioned graph neural network predictor is trained in the following manner: by randomly sampling N graph neural network architectures in the graph neural network search space; by testing on different target deployment devices to obtain the corresponding hardware efficiency, thereby forming a graph neural network architecture-hardware efficiency dataset; and by training the graph neural network predictor for a preset number of rounds based on the dataset to improve its prediction accuracy.

[0138] As described above, the graph neural network architecture search device provided by this invention employs a fine-grained design method, dividing the search space into a function space and an operation space, and organizing the search space through a supernet, which effectively reduces search time and solves the problem of efficient deployment and real-time operation of graph neural networks on edge devices. Furthermore, this application, combined with the proposed hardware sensing device, can effectively improve the actual hardware efficiency of the searched graph neural network architecture on the target device, simultaneously meeting the accuracy and hardware efficiency requirements of application scenarios for graph neural network models. This has significant advantages compared to traditional manual design and automated design techniques without hardware sensing methods. Using the method of this application can meet the scenario requirements for real-time operation of graph neural networks on resource-constrained edge devices, which is beneficial for exploring the application potential of graph neural networks in small terminal devices such as drones and autonomous vehicles, and is of great significance to the future development of multi-objective optimization technology for edge graph neural networks.

[0139] This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described method.

[0140] This invention also provides a computer-readable storage medium storing a computer program for performing the above-described methods.

[0141] like Figure 12As shown, the electronic device 600 may also include: a communication module 110, an input unit 120, an audio processor 130, a display 160, and a power supply 170. It is worth noting that the electronic device 600 does not necessarily need to include these components. Figure 12 All components shown; in addition, the electronic device 600 may also include Figure 12 For components not shown, please refer to existing technologies.

[0142] like Figure 12 As shown, the central processing unit 100, sometimes also referred to as a controller or operating control, may include a microprocessor or other processor device and / or logic device. The central processing unit 100 receives inputs and controls the operation of various components of the electronic device 600.

[0143] The memory 140 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. It may store the aforementioned failure-related information, and also store a program for executing that information. The central processing unit 100 may execute the program stored in the memory 140 to perform information storage or processing, etc.

[0144] Input unit 120 provides input to central processing unit 100. Input unit 120 may be, for example, a keypad or touch input device. Power supply 170 provides power to electronic device 600. Display 160 displays images and text. Display may be, for example, an LCD display, but is not limited thereto.

[0145] The memory 140 can be a solid-state memory, such as a read-only memory (ROM), random access memory (RAM), a SIM card, etc. It can also be a memory that retains information even when power is off, can be selectively erased, and contains more data; examples of this type of memory are sometimes referred to as EPROMs. The memory 140 can also be some other type of device. The memory 140 includes a buffer memory 141 (sometimes referred to as a buffer). The memory 140 may include an application / function storage unit 142 for storing application programs and function programs or processes for executing the operation of the electronic device 600 via the central processing unit 100.

[0146] The memory 140 may also include a data storage unit 143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 144 of the memory 140 may include various drivers for the electronic device's communication functions and / or for performing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0147] The communication module 110 is a transmitter / receiver 110 that transmits and receives signals via antenna 111. The communication module (transmitter / receiver) 110 is coupled to the central processing unit 100 to provide input signals and receive output signals, which can be the same as in a conventional mobile communication terminal.

[0148] Based on different communication technologies, multiple communication modules 110 can be configured in the same electronic device, such as cellular network modules, Bluetooth modules, and / or wireless LAN modules. The communication module (transmitter / receiver) 110 is also coupled to a speaker 131 and a microphone 132 via an audio processor 130 to provide audio output via the speaker 131 and receive audio input from the microphone 132, thereby enabling typical telecommunications functions. The audio processor 130 may include any suitable buffer, decoder, amplifier, etc. Additionally, the audio processor 130 is coupled to a central processing unit 100, enabling on-device recording via the microphone 132 and on-device playback of stored audio via the speaker 131.

[0149] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0150] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0151] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0152] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0153] Specific embodiments have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A graph neural network architecture search method, characterized in that, The method includes: Based on application requirements, a supernet is constructed for the graph neural network search space. The search space includes a function space and an operation space. The function space consists of the specific functions corresponding to the operations performed by each graph neural network. The operation space consists of the specific operations performed by each graph neural network. Perform a search of the function space to determine a combination of functions that maximizes the accuracy of the supernet; The function at each position on the supernet is fixed based on the function combination with the highest accuracy. Perform a search of the operation space to determine the optimal graph neural network architecture that meets the hardware efficiency and accuracy requirements of the application. The process of performing a search of the operation space to determine the optimal graph neural network architecture that satisfies the hardware efficiency and accuracy requirements of the application includes: The operation space sub-architecture is obtained by randomly sampling the operation at each location on the supernet; The operation space sub-architecture is input into the hardware sensing device, and the hardware sensing device obtains the hardware efficiency score corresponding to the operation space sub-architecture. Determine whether the hardware efficiency score meets the hardware efficiency requirements; In response to the hardware efficiency requirement being met, the operation space sub-architecture is assigned supernet pre-training weights to be tested on the validation set of the target dataset to obtain an accuracy score; the next round of population iteration is performed based on the evolutionary algorithm until the set maximum number of iterations is reached, and the optimal graph neural network architecture is output. The hardware sensing device obtains a hardware efficiency score corresponding to the operating space sub-architecture, including: Each operation of the operation space sub-architecture is taken as a node, and the data flow between operations is taken as an edge. At the same time, a global node is introduced to store the graph attribute information of the input target dataset. The global node connects all other nodes, thereby abstracting the operation space sub-architecture into a directed graph. Convert the directed graph into an adjacency matrix; Each node is assigned a feature vector based on a one-hot code to form a feature matrix. The target device's encoding, the adjacency matrix, and the feature matrix are used as inputs to a graph neural network predictor, which outputs a hardware efficiency score.

2. The graph neural network architecture search method as described in claim 1, characterized in that, The method further includes: obtaining the application requirements of the target application, the application requirements including: target dataset, target deployment device, accuracy requirements, and hardware efficiency requirements.

3. The graph neural network architecture search method as described in claim 1, characterized in that, The aforementioned search of the function space to determine a combination of functions that maximizes the accuracy of the supernet includes: The function space is sampled to generate a function space sub-architecture, which includes two sets of function settings corresponding to the upper and lower halves of the supernet, respectively. The accuracy of the function setting is obtained by training the function space sub-architecture based on the function setting. Repeatedly sample the function space and train the sub-architectures of the function space until the preset maximum number of iterations is reached; Determine the set of function combinations that maximize the accuracy of the supernet.

4. The graph neural network architecture search method as described in claim 2, characterized in that, After fixing the function at each position on the supernet according to the function combination with the highest accuracy, the method further includes: performing supernet pre-training for a preset number of rounds to obtain supernet pre-training weights.

5. The graph neural network architecture search method as described in claim 1, characterized in that, The graph neural network predictor is trained in the following manner: By randomly sampling N graph neural network architectures in the graph neural network search space; Tests were conducted on different target deployment devices to obtain the corresponding hardware efficiency, thereby constructing a graph neural network architecture-hardware efficiency dataset; The graph neural network predictor is trained for a preset number of rounds based on the dataset to improve its prediction accuracy.

6. A graph neural network architecture search device, characterized in that, The device includes: The supernet construction unit is used to construct a supernet based on the application requirements of the graph neural network search space. The search space includes a function space and an operation space. The function space is composed of the specific functions corresponding to the operations performed by each graph neural network, and the operation space is composed of the specific operations performed by each graph neural network. A function space search unit is used to perform a search of the function space to determine a combination of functions that maximizes the accuracy of the supernet. A supernet function fixing unit is used to fix the function at each position on the supernet according to the function combination with the highest accuracy. An operation space search unit is used to perform a search of the operation space to determine the optimal graph neural network architecture that meets the hardware efficiency and accuracy requirements of the application. The operation space search unit specifically includes: A random sampling module is used to randomly sample the operations at each location on the supernet to obtain the operation space sub-architecture; An input module is used to input the operation space sub-architecture into a hardware sensing device, and the hardware sensing device obtains the hardware efficiency score corresponding to the operation space sub-architecture. The judgment module is used to determine whether the hardware efficiency score meets the hardware efficiency requirements. The architecture output module is used to assign supernet pre-trained weights to the operation space sub-architecture in response to the hardware efficiency requirements, and to test it on the validation set of the target dataset to obtain an accuracy score; and to perform the next round of population iteration based on the evolutionary algorithm until the set maximum number of iterations is reached, and output the optimal graph neural network architecture. The hardware sensing device obtains a hardware efficiency score corresponding to the operating space sub-architecture, including: Each operation of the operation space sub-architecture is taken as a node, and the data flow between operations is taken as an edge. At the same time, a global node is introduced to store the graph attribute information of the input target dataset. The global node connects all other nodes, thereby abstracting the operation space sub-architecture into a directed graph. Convert the directed graph into an adjacency matrix; Each node is assigned a feature vector based on a one-hot code to form a feature matrix. The target device's encoding, the adjacency matrix, and the feature matrix are used as inputs to a graph neural network predictor, which outputs a hardware efficiency score.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images